Anthropic CEO Warns AI Now Poses Public Safety Risks
Anthropic CEO Dario Amodei has called for advanced artificial intelligence systems to be regulated in a manner similar to aircraft and pharmaceutical products, arguing that frontier AI models are rapidly reaching a stage where they present genuine public safety and national security risks.
In his latest essay, Policy on the AI Exponential, Amodei said AI capabilities are advancing far faster than governments and regulatory institutions can respond. If current trends continue, he believes the world could soon witness what he describes as “Powerful AI” systems comparable to “a country of geniuses in a datacenter.”
The warning reflects growing concern among AI leaders that existing governance frameworks may be inadequate for the next generation of artificial intelligence.
Why Amodei Believes AI Needs Stronger Oversight
According to Amodei, evidence of both AI’s transformative potential and its risks has become increasingly difficult to ignore.
He pointed to recent frontier AI systems that demonstrate advanced capabilities in areas such as cybersecurity, research, and problem solving. While these capabilities can drive innovation, they could also be exploited to threaten financial systems, critical infrastructure, and national security.
Amodei warned that cybersecurity risks may only be the beginning. Future AI systems could introduce biological risks, autonomous decision-making concerns, and other threats that current regulations are not designed to address.
He argued that transparency alone is no longer sufficient as a safety measure.
AI Should Be Regulated Like Aviation
To address these risks, Amodei proposed a regulatory framework similar to agencies such as the US Federal Aviation Administration (FAA), which oversees aircraft safety.
Under his proposal, advanced AI models crossing specific computational thresholds would be subject to mandatory testing and safety evaluations before deployment.
The framework would assess risks related to cybersecurity, biosecurity, AI autonomy, and automated research capabilities.
Governments would also have the authority to delay or block the release of models deemed unsafe, while developers would be required to implement strong security controls, red-team testing, and continuous threat monitoring.
Independent accredited evaluators or government agencies could oversee these assessments.
AI Could Reshape Jobs And The Global Economy
Beyond safety concerns, Amodei highlighted the profound economic impact AI may have on labour markets.
He believes future AI systems could eventually perform many cognitive tasks more effectively than humans, generating unprecedented productivity growth while simultaneously increasing economic inequality.
According to Amodei, the challenge in an AI-driven economy may shift from creating growth to ensuring that the benefits of that growth are distributed broadly across society.
To address potential workforce disruption, he proposed:
- Improved tracking of AI-related job displacement
- Incentives that encourage employment retention
- Long-term income support mechanisms
- Exploration of broader macroeconomic safety nets
- Universal capital accounts to distribute economic gains
He also argued that AI companies should bear responsibility for any increases in electricity costs caused by large-scale data centre expansion.
Concerns Over Democracy And Civil Liberties
Amodei warned that powerful AI could become a tool for authoritarian governments if appropriate safeguards are not established.
He said AI-enabled surveillance systems may allow governments to analyse massive amounts of public information and infer highly personal details about citizens, creating privacy risks beyond those anticipated by current legal frameworks.
He also raised concerns about autonomous weapons systems and the potential for governments to exercise power with reduced human oversight.
To protect democratic institutions, he recommended:
- Clear accountability standards for autonomous weapons
- A ban on domestic deployment of autonomous weapons systems
- Restrictions on large-scale data collection loopholes
- Broader citizen access to AI-based tools and assistance
Amodei stressed that AI should not be controlled exclusively by either governments or corporations, advocating instead for balanced oversight and accountability mechanisms.
AI Will Become A Major Geopolitical Asset
The Anthropic CEO also argued that artificial intelligence is becoming a strategic geopolitical resource.
According to Amodei, countries that successfully develop advanced AI capabilities could gain advantages comparable to those enjoyed by technologically superior militaries throughout history.
He proposed the creation of an alliance of democratic nations built around shared AI principles and cooperation.
The coalition would coordinate semiconductor supply chains, share advanced technologies, address AI-related risks collectively, and ensure member nations benefit from AI-driven innovation while resisting authoritarian uses of the technology.
Existing Regulations May Not Be Ready For AI Innovation
Amodei also highlighted concerns that existing regulatory systems are not prepared for the pace of innovation AI could unleash.
He cited healthcare and drug development as examples where AI may dramatically accelerate research and discovery, while approval systems remain structured around timelines that can stretch seven to eight years.
Without modernization, he warned, regulatory bottlenecks could prevent society from fully benefiting from AI-driven scientific breakthroughs.
He urged policymakers to begin developing standards now for evaluating AI-assisted research so that innovations can be adopted more quickly once proven safe and effective.
Why Anthropic Is Advocating Regulation
Anthropic has increasingly positioned itself as one of the strongest advocates for AI safety and governance.
The company has supported proposals related to frontier model testing, transparency requirements, and research into the economic impact of AI-driven automation.
According to Amodei, the central challenge facing governments, businesses, and society is ensuring that institutions can evolve quickly enough to manage AI’s risks while unlocking its enormous economic and scientific potential.
