Anthropic researcher estimates >10% chance AI kills humans this decade
Serge Bulaev
Evan Hubinger, a senior researcher at Anthropic, said he personally thinks there is more than a 10% chance that advanced AI could cause human extinction in the next decade. He noted that current AI models seem low risk but said there is no clear plan to safely control more powerful AI in the future. This estimate is higher than most academic guesses, which usually put the risk between three and eight percent. After his statement, some experts and former staff raised more safety concerns, and policy discussions about AI safety and oversight increased. Anthropic has not made an official comment or changed its policies since this public statement.
An Anthropic researcher estimates a greater than 10% chance that AI kills humans within the next decade, a claim that has ignited intense debate across the tech and policy sectors. The statement, from alignment science lead Evan Hubinger, appeared in a public X post in September 2024.
Hubinger clarified that while the "risk from present models is low," he personally believes the chance of human extinction from advanced AI is ">10 percent within the next decade." He argued that Anthropic currently lacks a clear roadmap for aligning a future superintelligence.
Major news outlets quickly covered the story, confirming the quote and noting Anthropic had not issued an official comment. Multiple publications reported on the statement, identifying Hubinger as a senior alignment researcher.
Quantifying AI Extinction Risk
Evan Hubinger, a senior researcher at AI safety company Anthropic, has personally estimated a greater than 10% probability of human extinction caused by advanced AI within this decade. This figure represents a significant concern among AI safety researchers.
Hubinger's statement places him among those expressing high concern for AI-related existential risk. Academic surveys and expert opinions show a wide range of estimates for AI extinction risk, with many researchers expressing significant concern about potential catastrophic outcomes.
Key points from Hubinger's assessment include:
- Extinction Risk: >10% chance within a decade.
- Current Models: Deemed "low risk."
- Primary Concern: A superintelligence emerging from recursive self-improvement.
- Alignment Plan: No clear solution has been identified.
- Corporate Response: Anthropic has issued no formal rebuttal.
Hubinger's post was prompted by the resignation of fellow researcher Jan Leike, who left Anthropic citing safety concerns. Before stating his >10% probability, Hubinger endorsed Leike's critique, writing about the validity of the concerns raised.
Industry and Policy Reactions
The incident amplified calls from policy analysts for mandatory evaluations of frontier models and stronger whistleblower protections. This aligns with a broader trend where multiple jurisdictions are implementing incident reporting and transparency requirements in response to high-profile safety disputes.
The discussion occurred as rival lab OpenAI was highlighting its own progress with advanced AI systems. OpenAI has been developing increasingly capable models, with reports of improved performance on various benchmarks. However, these developments have also raised questions about safety measures and evaluation standards. While OpenAI has described recent models as more robust against certain risks, expanded capabilities continue to generate debate over potential risks and measurement standards.
Anthropic's Internal Position and Outlook
Following the disclosure, no internal policy shifts or memos have been reported at Anthropic. Observers believe this silence is consistent with the company's public stance that aligning superintelligence remains an "unsolved technical problem." Industry focus continues to center on proposals like scaled agentic behavior evaluations, public incident logs, and safe-harbor rules for researchers who voice existential risk concerns.
Frequently Asked Questions
What exactly did Anthropic researcher Evan Hubinger say about AI existential risk?
In a post on X in September 2024, Evan Hubinger, Anthropic's alignment science lead, wrote about his concerns regarding AI existential risk. He stated his personal belief that there is a greater than 10% chance of human extinction from AI within the next decade. He further emphasized that "we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to." Hubinger clarified that "the risk from present models is low" and his concern centers on superintelligence arising from recursive self-improvement.
Has Anthropic officially responded to these statements?
Anthropic has not issued a formal corporate policy statement specifically addressing Hubinger's remarks. The company had not officially commented on either Hubinger's post or the resignation criticism that prompted it. Hubinger spoke in a personal capacity, though his statements align with Anthropic's broader safety-forward posture and internal risk framing.
What prompted Hubinger's statement?
The post came in response to researcher Jan Leike's departure from Anthropic, which Leike announced citing safety concerns. This resignation was part of a pattern of high-profile departures from leading AI labs that have made conversations around regulation and development practices more prominent in the AI safety community.
How does this connect to OpenAI's recent developments?
The remarks coincided with other significant industry developments involving OpenAI's continued advancement of AI capabilities. OpenAI has been developing increasingly sophisticated models with enhanced performance across various benchmarks. These developments have generated both excitement about AI progress and concerns about safety measures and risk management as capabilities advance.
What policy impact have these safety concerns generated?
Multiple jurisdictions including the United States and European Union have begun implementing measures to improve risk management for advanced AI systems. These include:
- Safety evaluations for frontier models
- Transparency disclosures
- Whistleblower protections
- Incident reporting mechanisms
Many companies have published or updated AI Safety Frameworks describing their risk management plans. These frameworks are expected to serve as reference points for ongoing and forthcoming regulatory initiatives.