AEF-1 standard for third-party AI evaluators gains support from xAI, OpenAI, and Anthropic

The AI Evaluator Forum (AEF) published AEF-1, a proposed baseline for independent third-party AI evaluations. The standard outlines expectations for access, conflicts of interest, funding relationships, recusal, and transparency for evaluators.

xAI, OpenAI, and Anthropic publicly cosigned support for the AEF-1 standard. The announcement follows earlier discussions about "pacing" frontier AI development and the role of third-party evaluators in verifying safety commitments.

Anthropic committed to giving embedded third-party evaluators ongoing, employee-like access to its facilities and systems. Dario Amodei, Anthropic’s CEO, described plans to provide evaluators with desks in offices, access badges, company laptops, and access to workspaces, tools, and permissions comparable to internal risk assessment teams.

The AEF-1 standard is positioned as a step toward verifiable safety commitments. It aligns with proposals for democratic coordination among frontier AI companies and efforts to coordinate with governments on pacing and compliance.

The push for embedded evaluators and standardized baselines reflects ongoing debates about how to ensure independent oversight of frontier AI development. These debates include discussions about whether labs should pace capability progress or focus on specific mitigations and containment measures.