"AI safety" refers to efforts to prevent artificial intelligence from causing harm^[1]. This site focuses on powerful future AI systems pursuing goals in conflict with human flourishing, because these systems may end up in control of the world and pose an existential risk to humanity.

AI safety is closely related to AI alignment, which refers to ensuring that AI systems pursue the goals we want them to.^[2] Ongoing work in AI safety can be roughly categorized into:

AI alignment research: Technical and conceptual research focused on getting AI systems to do what we want them to do
AI policy and governance: Setting up institutions and mechanisms that cause the major actors to implement good AI safety practices
AI strategy and forecasting: Building good models of how AI will develop and how our actions can make it go better

The terms “AI safety” and “AI risk” are mostly used in the context of existential risk. "AI safety" is sometimes also used more broadly to include work on reducing harms from current AI systems. While people sometimes use “AI safety” and “AI alignment” interchangeably to refer to the general set of problems around smarter-than-human AI, they occasionally use “AI existential safety” to make it clear that they mean risks to all of human civilization, or “AGI safety” to make it clear that they mean risks from future generally intelligent systems. ↩︎
While we expect misalignment to be the greatest obstacle to AI safety, alignment and safety are conceptually distinct. A misaligned system can be safe if it’s not particularly capable, if we manage to contain it, if it’s never used anywhere it could be dangerous , or if it’s never built at all. Conversely, an AI system used by terrorists to design bioweapons would be highly unsafe despite being aligned with its users. ↩︎

What is AI safety?.link_svg__st0{opacity:.6;enable-background:new}.link_svg__st1{fill:gray}