What are some arguments why AI safety might be less important?

This is an index of arguments against AI existential safety concerns. Note that views represented by authors here are often substantially different from the views of our editors.

Notes

Some recommended pieces are in bold.

Some of these arguments are substantially better than others. Additionally, some pieces are arguing for the importance of AI safety, while discussing counterarguments. Overall, the title of this document may be misleading, as many of these pieces are simply providing some important ideas to consider, rather than giving a comprehensive and conclusive argument.

It may be a useful exercise to contemplate how these arguments interact with arguments in various introductions to AI safety.

Author classification

• ~ means the person was working ~full-time on AI existential risk

  Existential risk

  A risk of human extinction or the destruction of humanity’s long-term potential.

  View full definition

  
  reduction

• ^ means the person was at least somewhat part of the AI existential risk community and/or related communities

• The ~ or ^ applies based on the date of publication. People only have an asterisk or caret if this designation was true around the time of publication. If someone critiques AI safety and then starts working on it 5 years later, they will not have an asterisk.

• Some classifications might be incorrect.

The list

• Some of the reviews of "Is power-seeking AI an existential risk?" ( Joe Carlsmith^ ); see Carlsmith's report

• What do we think are the best arguments against this problem being pressing? + Arguments against working on AI risk to which we think there are strong responses ( Benjamin Hilton^ )

• Success without dignity: a nearcasting story of avoiding catastrophe by luck ( Holden Karnofsky~ )

• My Objections to "We're All Gonna Die with Eliezer Yudkowsky

  Eliezer Yudkowsky

  Co-founder of MIRI, known for his early pioneering work in AI alignment and his predictions that AI will probably cause human extinction.

  
  " ( Quintin Pope~ )

• Counterarguments to the basic AI x-risk case ( Katja Grace~ ); see this response

• Ben Garfinkel on scrutinising classic AI risk arguments ( Rob Wiblin^, Ben Garfinkel~ ); see this slideshow

• The Crux List ( Zvi Mowshowitz^ )

• Where I agree and disagree with Eliezer ( Paul Christiano~ )

• My thoughts on the social response to AI risk ( Matthew Barnett~ )

• Counting arguments provide no evidence for AI doom ( Nora Belrose~, Quintin Pope~ ); see the comments

• Notes on Existential Risk from Artificial Superintelligence

  Superintelligence

  An AI with cognitive abilities far greater than those of humans in a wide range of important domains.

  View full definition

  
  ( Michael Nielsen )

• Imitation Learning is Probably Existentially Safe ( Michael Cohen~, Marcus

  Gary Marcus

  Professor of psychology and neural science that is known for his skepticism towards the stance that current methods will be sufficient to lead to AGI.

  
  Hutter )

• Some arguments in the CAIS Philosophy Fellowship Midpoint Deliverables ( various CAIS Philosophy Fellows )

• AI is easy to control ( Nora Belrose~, Quintin Pope~ )

• Why I Am Not (As Much Of) A Doomer (As Some People) ( Scott Alexander^ )

• ‘Dissolving' AI Risk – Parameter

  Parameter

  Numerical values in an AI system that are modified during training to change the AI's performance.

  Uncertainty in AI Future Forecasting ( Alex Bates )

• Superintelligence Is Not Omniscience ( Jeffrey Heninger~, Aysja Johnson^ )

• Exaggerating the risks (Part 6: Introducing the Carlsmith report) + part 7 + part 8 in a series ( David Thorstad^ )

• Evolution provides no evidence for the sharp left turn

  Sharp left turn

  An event in which an AI’s capabilities suddenly generalize, but its alignment doesn’t also generalize.

  View full definition
  ( Quintin Pope~ ); see this response

• On Those Undefeatable Arguments for AI Doom ( 1a3orn^ )

• AGI

  AGI

  AI that is at least as good as humans at general problem-solving.

  View full definition

  
  and the EMH: markets are not expecting aligned or unaligned AI in the next 30 years ( Trevor Chow, Basil Halperin, J. Zachary Mazlish ); see this response

• Order Matters for Deceptive Alignment + Deceptive Alignment is <1% Likely by Default ( David Wheaton^ )

• Artificial General Intelligence and how (much) to worry about it ( Rohit Krishnan )

• AGI Catastrophe and Takeover

  AI takeover

  A hypothetical event where a powerful AI effectively takes over the world.

  View full definition
  : Some Reference Class-Based Priors ( Zach Freitas-Groff^ )

• How big of a risk is misalignment? in “Why AI alignment could be hard with modern deep learning” ( Ajeya Cotra~ )

• Does natural selection favor AIs over humans? Model this! ( Tyler Cowen )

• Inference Speed is Not Unbounded ( OneManyNone )

• The situational awareness assumption in AI risk discourse, or why people should chill ( José Luis Ricon )

• “Frequently Asked Questions” in An Overview of Catastrophic AI Risks ( Center for AI Safety~ )

• The bullseye framework: My case against AI doom ( titotal^ )

• titotal on AI risk scepticism ( Vasco Grilo^ )

• Frequent arguments about alignment ( John Schulman^ )

• grey goo is unlikely ( bhauth^ )

• Disagreements with the Yudkowskian future of AI ( Matthew Barnett^ )

• Passing the ideological Turing test + part 2 ( NinaR~ )

• A tale of 2.5 orthogonality theses ( Arepo^ )

• Why transformative artificial intelligence is really, really hard to achieve ( Arjun Ramani, Zhengdong Wang ); see this reply

• Possible Miracles ( Akash Wasil~, Thomas Larsen~ )

• My Current Thoughts on the AI Strategic Landscape ( Jeffrey Heninger~ )

• Transformative AGI by 2043 is <1% likely ( Ari Allyn-Feuer, Ted Sanders ); see this comment

• What do XPT forecasts tell us about AI risk? ( Forecasting Research Institute^, rosehadshar^ )

• Predictions of AI doom are too much like Hollywood movie plots ( Timothy B. Lee )

• Many arguments for AI x-risk are wrong ( Alex Turner~ )

• AI Doom and David Hume: A Defence of Empiricism in AI Safety ( Matt Beard^ )

• [AN #80]: Why AI risk might be solved without additional intervention from longtermists ( Rohin Shah~ summarizing other people’s views )

• But exactly how complex and fragile? ( Katja Grace~ )

• “a long thread about why I'm personally not worried yet” ( William Eden^ )

• A Critique of AI Takeover Scenarios ( James Fodor^ )

• Against a General Factor of Doom ( Jeffrey Heninger^ )

• A list of good heuristics that the case for AI x-risk fails ( David Krueger~ )

• Concrete Reasons for Hope about AI ( Zac Hatfield-Dodds~ )

• My highly personal skepticism braindump on existential risk from artificial intelligence. ( Nuño Sempere^ ); see this summary

• Is Avoiding Extinction from AI Really an Urgent Priority? ( Seth Lazar, Jeremy Howard, Arvind Narayanan )

• AI will change the world, but won’t take it over by playing “3-dimensional chess”. ( Boaz Barak, Ben Edelman ); see these comments

• Arguments for AI Risk and Arguments against AI risk sections of "AI Alignment 2018-19 Review." ( Rohin Shah~ )

• Reasons I’ve been hesitant about high levels of near-ish AI risk ( Eli Lifland~^ )

• AI Risk Skepticism ( Roman Yampolskiy~ )

• Ten Levels of AI Alignment Difficulty ( Samuel Martin )

• How sure are we about this AI stuff? ( Ben Garfinkel~ )

• On Deference and Yudkowsky's AI Risk Estimates ( Ben Garfinkel~ )

• The hot mess theory of AI misalignment: More intelligent agents behave less coherently ( Jascha Sohl-Dickstein ); see this response

• Artificial superintelligence and its limits: why AlphaZero cannot become a general agent ( Karim Jebari, Joakim Lundborg ); see this response

• Existential risk from artificial general intelligence: Skepticism ( Wikipedia )

• Reframing Superintelligence: Comprehensive AI Services as General Intelligence ( K Eric Drexler

  Eric Drexler

  Engineer known for introducing molecular nanotechnology. More recently, he has been interested in AI and promotes a tool-based view he calls Comprehensive AI Services (CAIS).

  
  ^ )

• We Aren't Close To Creating A Rapidly Self-Improving AI ( Jacob Buckman )

• AI Risk, Again + I Still Don't Get Foom + How Does Brain Code Differ? ( Robin Hanson^ )

• What Are Reasonable AI Fears? ( Robin Hanson^ ); see this response

• Agency Failure AI Apocalypse? ( Robin Hanson^ ); see this response

• “I Object!” section of Extinction Risk from Artificial Intelligence ( Michael Cohen~ )

• Blake Richards on Why he is Skeptical of Existential Risk from AI on The Inside View (Blake Richards, Michaël Trazzi~ )

• Gary Marcus and Stuart Russell

  Stuart Russell

  Computer science professor at UC Berkeley, founder of CHAI, and co-author of the textbook Artificial Intelligence: A Modern Approach.

  
  discuss AI risk on the Sam Harris podcast ( Gary Marcus, Sam Harris )

• Alignment By Default ( John Wentworth~ )

• Melanie Mitchell and Stuart Russell debate for Munk Debates ( Melanie Mitchell, Stuart Russell~ )

• Why AI is Harder Than We Think ( Melanie Mitchell ); see Richard Ngo's comment

• A shift in arguments for AI risk ( Tom Adamczewski )

• How to navigate the AI apocalypse as a sane person ( Eric Hoel )

• Debate on Instrumental Convergence between LeCun

  Yann LeCun

  AI researcher who won the Turing Award in 2018 and is the Chief AI Scientist at Meta.

  
  , Russell, Bengio
  Yoshua Bengio

  2018 Turing Award winning AI researcher who is the scientific director at MILA.

  
  , Zador, and More ( Yann LeCun, Stuart Russell~, Yoshua Bengio, Tony Zador, and more )

• My Bet: AI Solves Flubs ( Scott Alexander^ )

• AI Researchers On AI Risk ( Scott Alexander^ )

• Contra Acemoglu On...Oh God, We're Doing This Again, Aren't We? + highlights from the comments ( Scott Alexander^ responding to Daron Acemoglu )

• Maybe The Real Superintelligent AI Is Extremely Smart Computers ( Scott Alexander^ responding to Ted Chiang )

• 10 Reasons to Ignore AI Safety ( Rob Miles~ )

• A Response to Steven Pinker on AI ( Rob Miles~ )

• How much EA analysis of AI safety as a cause area exists? ( Richard Ngo~ )

• Heretical Thoughts on AI | Eli Dourado ( Cinera^ )

• Some abstract, non-technical reasons to be non-maximally-pessimistic about AI alignment ( Rob Bensinger~ )

• Can a Paperclip Maximizer Overthrow the CCP? + Pinker on Alignment and Intelligence as a "Magical Potion" ( Richard Hanania )

• Superintelligence: The Idea That Eats Smart People + video ( Maciej Ceglowski )

• How I failed to form views on AI safety ( Ada-Maaria Hyvärinen^ )

• There are no coherence theorems

  Coherence theorem

  If an agent's choices satisfy certain rationality conditions, then those choices can be represented as maximizing an expected utility function

  View full definition
  ( Elliott Thornley~ )

• The Singularity is not coming, On the Measure of Intelligence, The implausibility of intelligence explosion

  Intelligence explosion

  A hypothetical scenario where machines become more intelligent very quickly, driven by recursive self-improvement.

  View full definition

  
  ( François Chollet ); see Eliezer Yudkowsky’s reply to the last piece

• "Bad AI DontKillEveryoneism Takes" section of AI #3 ( Zvi Mowshowitz~ )

• Some abstract, non-technical reasons to be non-maximally-pessimistic about AI alignment ( Rob Bensinger~ )

• The Preference Fulfillment Hypothesis ( Kaj Sotala^ )

• How Organisms Come to Know the World: Fundamental Limits on Artificial General Intelligence ( Andrea Roli, Johannes Jaeger, Stuart Kauffman )

• Don’t Worry about Superintelligence ( Nicholas Agar )

• Where I'm at with AI risk: convinced of danger but not (yet) of doom ( Amber Dawn^ )

• How Much Should You Freak out About AI? ( Michael Huemer )

• Reasons you might think human-level AI is unlikely to happen soon ( Asya Bergal~ )

• Existential risk, AI, and the inevitable turn in human history ( Tyler Cowen ); see this response from Scott Alexander and the comments on it from Tyler and Scott + this response from Zvi Mowshowitz + this response from Leopold Aschenbrenner)

• Don’t Fear the Terminator ( Yann LeCun, Anthony Zador )

• Why I am Not An AI Doomer ( Sarah Constantin^ )

• Is Human Intelligence Simple? Part 3: Disambiguating Types of Simplicity ( Sarah Constantin^ )

• More AI debate between me and Steven Pinker! + Steven Pinker and I debate AI scaling! + Reform AI Alignment ( Scott Aaronson^ )

• The Prospect of an AI Winter ( Erich Grunewald )

• Alan Chan discussing alignment with Tim Scarfe of Machine Learning Street Talk ( Alan Chan~, Tim Scarfe )

• Against AI Doomerism, For AI Progress ( Max More )

• What are some objections to the importance of AI alignment? ( Søren Elverlin~ )

• But Have They Engaged with the Arguments? ( Philip Trammell^ )

• On The Impossibility of AI Alignment ( Kevin Lacker )

• How to know if artificial intelligence is about to destroy civilization ( Oren Etzioni ); see this summary

• The AI Messiah ( ryancbriggs^ )

• How good is humanity at coordination? ( Buck Shlegeris~ )

• The Orthogonality Thesis is Not Obviously True ( Omnizoid^ )

• Bad alignment take bingo with replies ( Rob Bensinger~ ); see the card “with some 2023 additions”; note these bingo cards are mostly designed to be funny rather than truth-seeking, although they do contain some core ideas of various arguments

• Memes that caricature some common counterarguments ( AI Notkilleveryoneism Memes^ ); note that these memes lack nuance and can be seriously misleading

• High-level hopes for AI alignment ( Holden Karnofsky~ )

• How might we align transformative AI

  Transformative AI

  An AI that is capable of transforming society, as drastically as the industrial revolution or even more so.

  View full definition
  if it's developed very soon? ( Holden Karnofsky~ )

• Response to e/acc arguments ( Dan Hendrycks~ )

• The 'Wild' and 'Wacky' Claims of Karnofsky’s ‘Most Important Century’ ( Spencer Becker-Kahn~ )

• "Diamondoid bacteria" nanobots: deadly threat or dead-end? A nanotech investigation + When do "brains beat brawn" in Chess? An experiment + Could a superintelligence deduce general relativity from a falling apple? An investigation ( titotal^ )

• The 'Don't Look Up' Thinking That Could Doom Us With AI ( Max Tegmark~ )

• Why do some AI researchers dismiss the potential risks to humanity? ( David Krueger~ )

• The “General Problem Solver” Does Not Exist: Mortimer Taube and the Art of AI Criticism ( Shunryu Colin Garvey )

• Unsavory medicine for technological civilization: Introducing ‘Artificial Intelligence & its Discontents’ ( Shunryu Colin Garvey )

• Is Superintelligence Impossible? ( Daniel Dennett and David Chalmers )

• “Literally every conversation I have on Twitter about long-term risk leaves me more worried than when I started.” ( Gary Marcus )

• Steelman arguments against the idea that AGI is inevitable and will arrive soon ( RomanS^ )

Other collections

• "Alternate Perspectives" in STS 10SI: Intro to AI Alignment ( Stanford AI Alignment )

• AI Optimism ( Nora Belrose^ and Quintin Pope^ )

• Object-level AI risk skepticism tag ( LessWrong )

• “AI Success Models” tag ( LessWrong )

• A contra AI FOOM reading list ( Magnus Vinding^ )

• Richard Ngo Critiques Collection ( Richard Ngo~ )

• AI Alignment 2018-19 Review ( Rohin Shah~ )

• Has anyone written a case against AI x-risk where it is clear they understand the case for x-risk superbly well ( Michael Nielsen )

• Who are some prominent reasonable people who are confident that AI won't kill everyone? ( LessWrong )

• Future Fund Worldview Prize tag ( EA Forum )

• Open Philanthropy AI Worldviews Contest ( Open Philanthropy )

• AI skepticism tag ( EA Forum )