UK AISI publishes what sits behind five frontier benchmark scores

single source· 1 articles · confidence: high · first seen 2026-09-22 00:00 UTC

What this means for you

If you cite benchmark scores for these six models, the cards now carry the settings behind them, and AISI's paper shows those settings move the result: how much compute each model was allowed per answer, and whether it was told when it was right. A score quoted without them is not a comparison.

The UK AI Safety Institute has published the configuration behind reported benchmark scores for six frontier models, via EvalEval's Evaluation Cards — a shared format recording how a result was produced. Five benchmarks are covered — HealthBench, FrontierMath, Humanity's Last Exam, SWE-Bench Pro and Terminal-Bench 2.0 — across Claude Opus 4, 4.5 and 4.6, and GPT-5, 5.2 and 5.4. Two cyber evaluations, Cyber CTFs and The Last Ones, use a partly different model set. The data accompany AISI's paper on how the compute spent per answer and the evaluation protocol shift scores; on Humanity's Last Exam, being told after each attempt whether an answer was correct let models solve more tasks.

Models in this story

Key facts

  • ·AISI is releasing verified results, context and configuration for five benchmarks: HealthBench, FrontierMath, Humanity's Last Exam, SWE-Bench Pro and Terminal-Bench 2.0. source
  • ·The five-benchmark release covers six models: Claude Opus 4, Claude Opus 4.5, Claude Opus 4.6, GPT-5, GPT-5.2 and GPT-5.4. source
  • ·Two cyber evaluations, Cyber CTFs and The Last Ones, are included on a different, partially overlapping set of models. source
  • ·The data accompany AISI's paper, How Inference Compute Shapes Frontier LLM Evaluation. source
  • ·The collaboration began at a joint workshop alongside NeurIPS 2025, and AISI feedback helped shape the Every Eval Ever reporting schema. source

What the sources say

  • Hugging Face Blog — Announces the collaboration's second phase and lists exactly which benchmark and model results are now published.

Sources

The original reporting. Follow these — they did the work.

← the wire