What is "p(doom)"?

The term p(doom) is a shortening of "probability of [AI] doom", i.e., how likely an AI-caused existential catastrophe is to happen.¹

In the AI risk discourse, people will often say that they "have a p(doom) of X%", which means they believe an AI-caused existential catastrophe is X% likely to occur.

Estimates of p(doom) vary greatly, and people with a high p(doom) sometimes get called doomers.

Although the word “doom” suggests inevitability, “p(doom)” is not meant to express the probability that a catastrophe is inevitable, but simply the probability that a catastrophe happens.

In this context, “probability” is meant as subjective probability. If someone gives a p(doom) of 20%, they don’t mean that doom has happened 20% of the time in some reference class²

, or that they have a formal model that justifies that exact number. Rather, they’re expressing their own degree of uncertainty. Compare this situation to watching a sports game and thinking it’s 20% likely that team A will win. That means you might be willing to bet at 4 to 1 odds against team A — even if these teams have never played each other before, and you don’t have any past results to count. For both existential risks and sports, the exact numbers people give may fluctuate over time, depending on what they happen to be thinking about, even if they have not received any new relevant information.

Why use p(doom)?

Some people share their beliefs in the form of probabilities to avoid ambiguities in ordinary language. “I’m optimistic”³

may mean “P(doom) is below 1%” to one person, and “P(doom) is below 20%” to another. Using non-quantitative terms would hide this large disagreement.

Criticisms of the term

Some people criticize the use of the term. Michael Nielsen claims that people in the AI industry use the concept fatalistically, taking the probability of doom as set in stone regardless of their own choices. Secondly, as noted by Isaac King, people mean different things by “p(doom)”:

• Some include dystopian futures, others only refer to human extinction.

• Some only include disasters that happen by, say, 2040.

• Some use doom to mean x-risks due to misaligned AI, others use it to mean all x-risks arising from powerful AI, including through misuse.

Moreover, King argues that professing a high p(doom) de facto works to mark the speaker as part of a particular social group. This mixes up how people think about AI x-risks with the social games people play.

---

1. Other definitions of doom exist — see the “Criticisms of the term” section. ↩︎

2. To a frequentist, this is what it means to say that something has a "probability". This approach cannot be used here since the number of times humanity has become extinct in the past is zero. If we widen the reference class to include either species extinctions or causes of mass human deaths, we can derive base rates, but there are good reasons to take these with a grain of salt. ↩︎

3. For instance, Lina Khan, chair of the FTC, says when asked about her p(doom) that she’s an optimist. The interviewer asks if this means her p(doom) is 0%, and she answers, “no, no, more like 15%”! ↩︎