Anthropic CEO proposes third-party evaluators and coordinated safety limits to pace AI development

Anthropic CEO Dario Amodei outlined a plan to "pace the frontier" of AI development, proposing three broad strategies to slow the advancement of AI capabilities. In a blog post, Amodei argued that recent progress—particularly in AI’s ability to build the next generation of AI—has accelerated to a pace that requires deliberate caution. He emphasized the need to "slow the pace at which we improve the capabilities of AI models," while acknowledging that progress will still feel rapid.

The first strategy involves inviting third-party embedded evaluators from organizations like METR to verify that companies are adhering to pacing and safety commitments. These evaluators would have access to company resources comparable to internal risk assessment teams, with exceptions for legal or contractual constraints. Amodei framed this as a unilateral commitment by Anthropic and called on governments to require other frontier companies to adopt similar measures. OpenAI CEO Sam Altman responded in agreement, stating that OpenAI would also implement embedded evaluators and share further details soon.

Amodei’s second proposal calls for leading AI companies in democratic countries to coordinate on common safety standards and limits to unchecked AI progress. He acknowledged antitrust concerns that could deter such coordination but suggested that government mediation—specifically a narrow waiver to enable safety discussions—could mitigate these risks. Amodei also addressed concerns about Chinese AI dominance, arguing that stricter controls on chip sales and model distillation to Chinese companies could widen America’s lead over the next 3–5 years.

The third strategy involves global coordination with authoritarian governments, including China, to prohibit clearly dangerous AI uses, such as the production of biological weapons. While Amodei acknowledged the limits of such cooperation, he suggested that even narrow agreements could help mitigate risks. The post reflects Anthropic’s relative openness to regulation, though critics argue such proposals could serve as a form of regulatory capture.

Amodei’s plan comes amid growing scrutiny of AI safety practices, including the resignation of Anthropic researcher Jacob Coxon, who warned that leading AI companies are "gambling with our lives." Amodei did not directly address Coxon’s resignation in his post but reiterated his belief that AI can improve human life if developed responsibly. His proposals aim to address a "crisis of trust" by demonstrating a commitment to deliberate, cautious progress.