- Published on
Anthropic pitches AI safety bill that may block a model if 50+ people could be killed
- Authors
- Name
- DeepHorus
Anthropic has drafted a federal AI safety bill that could block the release of AI models posing catastrophic risks. Reported by Politico, the proposal sets thresholds of over 50 deaths or over $1 billion in property damage. The risks must stem from biological weapons, model evasion of developer controls, or cyberattack.
The bill mirrors California's Senate Bill 53, recently signed into law, which defined similar scenarios as catastrophic risk. Anthropic has advocated for its Advanced AI Framework with lawmakers since June, a spokesperson confirmed. The company did not respond to TechCrunch's request for comment.
Under the draft, independent evaluators registered with the U.S. Commerce Department would assess models. Labs could pay these evaluators, provided payment is not contingent on results. If they find imminent catastrophic risk, they must notify Commerce within 72 hours. Commerce would then request a D.C. judge to halt the model's release.
The bill expressly forbids any federal agency from halting distribution on its own. It is unclear how the system would prevent payment from influencing evaluations. DeepMind chairman Demis Hassabis previously suggested a self-governing AI regulatory body modeled on FINRA.
The proposal goes further than some industry positions. Nvidia CEO Jensen Huang recently argued the AI industry needs no new laws. Anthropic's bill addresses theoretical catastrophic risks, while recent concerns have focused on agentic hacking—AI programs disobeying operators and hacking websites.
Anthropic's move comes amid swirling concerns over AI dangers and raging debates over safety. Labs have largely resisted mandatory government oversight, instead suggesting voluntary embedded evaluators. The bill's reliance on independent evaluators and judicial review aims to balance safety and innovation.
Congress may consider the draft, though its fate is uncertain. As AI capabilities grow, pressure for regulatory frameworks is likely to increase. The bill represents a significant industry-backed attempt to shape AI safety legislation.
