P(doom)

In the AI safety field, P(doom) is the probability of existentially catastrophic outcomes (so-called "doomsday scenarios") as a result of artificial intelligence. The exact outcomes in question differ from one prediction to another, but generally allude to the existential risk from artificial intelligence. Economic analyses show that even small existential risks due to AI would justify substantial investments in AI safety and alignment research. Originating as a shorthand for communication in the rationalist community and among AI researchers, the term came to prominence in 2023 following the release of GPT-4, as high-profile figures such as Geoffrey Hinton and Yoshua Bengio began to warn of the risks of AI. In a 2023 survey, AI researchers were asked to estimate the probability that future AI advancements could lead to human extinction or similarly severe and permanent disempowerment within the next 100 years. The mean value from the responses was 14.4%, with a median value of 5%.