10: Advanced AI is a big deal even if we don’t lose control

3 min read

Suggest changes in Google Docs

So far, we’ve discussed one class of possible consequences of advanced AI: systems ending up with different values than we intended, and autonomously deciding to seize control. This is the main concern that this site is about.

But of course, introducing a highly powerful class of tools into the world, let alone a new intelligent species, will have many other kinds of implications:

Even today’s systems are already quite useful, and at the same time cause significant harms. For example, art-generating AI is generating fun art for consumers, but is also making it harder for artists to find employment. Future AI that substitutes for humans in more sectors might create much more wealth and also cause much more wide-ranging economic upheaval.
Future advances may result in catastrophic risks from deliberate misuse of AI. Misuse is a separate problem from loss of control. For example, bad actors could use sufficiently powerful AI to design bioweapons or hack computer systems. Instead of the complicated research process that’s required now, AI could allow people to do these things at the press of a button.
Advanced AI has important political implications. Such a powerful tool could result in power being concentrated in a narrow set of people controlling it, like AI companies. It could also be used by governments to control their populations, and could upset the current balance of power, perhaps resulting in tensions and even war surrounding its development. Eventually, it could be used to create a permanent totalitarianism or inadvertently lock in some set of moral values we might not have wanted to.
If AI systems attain consciousness or personhood, that carries moral implications about how we should treat them. We may want to ensure we’re nice to them so they’ll be nice to us, or because we value their welfare, or because it makes us better people. Their uninterpretable nature, and our lack of understanding of concepts around consciousness, make it especially hard to navigate these implications.
When AI becomes as inventive as humans or more, we may see a drastic speedup in the pace of technological and social change. That could be good, but it could also destabilize society — we don’t seem to be bracing for that kind of change.

All these implications — positives and negatives, current and future — can be real at the same time. There isn’t just one “real problem with AI”, nor does AI bring only problems.

In fact, if we navigate the risks, the benefits from well-aligned AI, deployed toward good ends, could be transformative.