Are Google, OpenAI, etc. aware of the risk?
The major AI companies are thinking about existential risk from AI. OpenAI was founded with the intention to counter risks from superintelligenceAnthropic was founded by ex-OpenAI employees specifically to focus on safety. Google DeepMind has a Safety Research division that concentrates on misalignment. These organizations have collaborated on research endeavors.1 Leaders and employees at these companies have signed a statement saying “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
However, as of 2024, the majority of the effort these organizations put forward is towards capabilities research, rather than safety.
Further reading:
-
AI Lab Watch ranks labs on various criteria related to AI safety
-
SaferAI produces a similar ranking
The paper Concrete Problems in AI Safety was a collaboration between researchers at Google Brain (now Google Deepmind), Stanford, Berkeley, and OpenAI. ↩︎