Concern about existential risk from misaligned AI is not based on worries that AI systems will become conscious, turn evil, or develop motivations like revenge or hatred.

Stuart Russell (co-author of Artificial Intelligence: A Modern Approach) said: "The primary concern is not spooky emergent consciousness but simply the ability to make high-quality decisions." AI systems may become 1) extremely competent at making decisions that achieve their goals, while also being 2) indifferent to things we value highly (like human life or happiness). And unfortunately, many of a highly-competent AI’s plans would have destructive side-effects on anything the AI does not specifically value.

Here's one (highly condensed) version of the basic case for concern:

As AIs improve, they're likely to become extremely competent at producing plans for achieving their goals, and at carrying out those plans.
We don't know how to reliably give an AI particular goals. Therefore, it's very likely that an AI created using current methods will end up with some goal or set of goals that we didn't intend to give it.
For almost any goal maximized by a very powerful AI, it's very probable that the most effective plan for achieving that goal will involve actions that are very bad for us, unless we're able to get the AI to "care" specifically about not doing things that are bad for us. Unfortunately, we don't know how to do that.

....link_svg__st0{opacity:.6;enable-background:new}.link_svg__st1{fill:gray}