Anthropic drafts AI safety bill tying model release to judge-ordered blocks over catastrophic risk thresholds

Anthropic has circulated a draft AI safety bill that would tie federal intervention to judicial approval when a model poses a foreseeable, imminent risk to 50 or more people or $1 billion in property damage, according to reporting by Politico. The bill defines such risks as stemming from biological weapon creation, model control evasion, or cyberattacks.

Under the proposal, independent evaluators registered with the U.S. Commerce Department would assess models for catastrophic risk, with a 72-hour window to report findings to the department. The Commerce Department would then petition a Washington, D.C., judge to halt a model’s release if the risk threshold is met.

The draft legislation explicitly forbids federal agencies from blocking a model’s distribution without a judge’s permission, centralizing authority in the judiciary rather than executive agencies.

The risk thresholds—50 lives or $1 billion in damages—mirror those in California’s Senate Bill 53, which was signed into law and similarly defines 'catastrophic risk' in AI contexts.

Evaluators could be paid by AI labs, provided payments are not contingent on evaluation outcomes, though the mechanism to prevent influence is not detailed in the report.

Anthropic confirmed the draft bill to Politico and stated it aligns with the company’s Advanced AI Framework, which it has advocated for with federal and state lawmakers since June.