Isn't the real concern that people will misuse AI?

Misuse is a problem with every powerful technology. Fire can be used to cook food or to burn down a village, and nuclear reactions can be used to produce electricity or to power terrible weapons.

Misuse of AI has the potential to cause harm at the individual, societal, and global levels by enabling:

• Authoritarian surveillance

• Terrorism

• Autonomous weapons

• Non-consensual intimate deepfakes

• Misinformation and propaganda

One of the aims of AI governance is to find ways to prevent such harm.

However, misuse is not the only risk from AI. Even when used with the best of intentions, technology can cause accidents: a fire can spread to burn down a city, and a reactor can melt down.

Accidents with advanced AI could be particularly bad when AI can pursue its own goals, since we currently don’t know how to specify safe goals and have highly advanced systems follow them. Future AI could be powerful enough to outmaneuver humanity, and accidents resulting from misalignment could threaten human civilization as a whole.

In other words, for advanced AI to go well, we will need to both:

1. Coordinate to prevent bad actors from misusing AI.

2. Ensure powerful AI can be safely used at all, without causing harms that no human intended, by solving the alignment problem.