UK AI Security Institute to publish benchmark results using EvalEval’s open schema

The UK AI Security Institute (AISI) is adopting EvalEval’s open reporting infrastructure to publicly share verified evaluation results for five benchmarks and six frontier models. The release includes results for HealthBench, FrontierMath, Humanity’s Last Exam, SWE-Bench Pro, and Terminal-Bench 2.0, alongside two cybersecurity-related evaluations, Cyber CTFs and The Last Ones.

The published results cover six frontier models: Claude Opus 4, Claude Opus 4.5, Claude Opus 4.6, GPT-5, GPT-5.2, and GPT-5.4. AISI states that transparent reporting at the transcript level is necessary not only for reproducibility but also for deeper analysis and diagnosis of evaluation outcomes.

AISI’s release accompanies the paper “How Inference Compute Shapes Frontier LLM Evaluation,” which examines how benchmark performance varies with inference-time compute and evaluation protocols. The institute highlights that performance on Humanity’s Last Exam, for example, can change depending on evaluation setup and token usage, particularly when models receive correctness feedback from an oracle after each attempt.

The collaboration between AISI and EvalEval builds on prior joint research initiated at a NeurIPS 2025 workshop and incorporates feedback that shaped the Every Eval Ever (EEE) schema. EvalEval’s mission is to standardize evaluation reporting through a shared schema and an open platform called Evaluation Cards, which consolidates benchmark metadata, evaluation-run data, and model metadata into interpretable records.

The EvalEval Coalition describes itself as a research community focused on improving evaluation science and addressing gaps in documenting evaluation applicability and utility. Its flagship projects include the Every Eval Ever schema and Evaluation Cards, which aim to clarify when seemingly similar scores stem from different experimental conditions.

AISI is a research organization within the UK government’s Department for Science, Innovation and Technology, with a mission to provide governments with a scientific understanding of risks posed by advanced AI.