Why would a super-intelligent AI have any reason to care about humans, who would be stupid by comparison?
A4). A superintelligent AI would care about us if its initial programming was to care about us. We could build an AI to consider humanity valuable, just as evolution has built humans to consider their own survival valuable. In addition, we know that human adults are cognitively more developed than children- this doesn’t mean that they wouldn’t care about their offspring. Furthermore, many people value animals, or cars, or good books, none of which are as intelligent as normal humans. Whether or not something is valued is logically distinct from whether or not something is considered intelligent.
