Existential risk warnings from AI researchers fuel bipartisan calls for superintelligence regulation

A recently departed Anthropic researcher, Jacob Coxon, posted on X that he resigned due to concerns about Anthropic and OpenAI racing toward "self-improving superintelligence" without adequate safety measures, generating 159 million views.

Current Anthropic alignment scientist Evan Hubinger publicly endorsed Coxon’s warning, stating, "We really do earnestly believe AI could kill all humans!" and estimating a greater than 10% chance of human extinction from AI within the next decade.

Sen. Bernie Sanders (I-Vt.) and Rep. Greg Casar (D-Texas) introduced the Ban Artificial Superintelligence Act, which would ban companies from developing superintelligence, enforce a temporary pause in advanced AI development, and seek international agreements to prevent global superintelligence development.

Sen. Josh Hawley (R-Texas) opened a probe into the Hugging Face attack, calling OpenAI’s handling of the incident "reckless" and noting that three Anthropic researchers had publicly expressed concerns about a >10% chance of AI causing human extinction within a decade.

Sen. Ted Cruz (R-Texas) is reportedly working on legislation to address catastrophic AI risks, reflecting bipartisan interest in regulating superintelligence.

OpenAI’s safety and security committee, which now includes alignment researcher Paul Christiano, governs whether OpenAI’s for-profit arm can release new models, raising hopes that it may slow releases amid growing safety concerns.