Which is best?

No single model. 10 different models lead the 18 tests below, and none leads them all.

10 leadersacross 18 tests. Each score is on its own test's scale, so the numbers never add up to a ranking.

Also on this page, kept apart: what people prefer in blind votes, and which models will answer, and which refuse.

Who places wheretop three on each test · rows ordered by lab, not by merit

ModelChess puzzlesEBR-BenchFrontierMathFrontierMath: Erdős problemsFrontierMath Tier 4FrontierMath Tiers 1–3 (v2)Furniture assemblyGPQA DiamondMirrorCodeMystery game puzzlesSimpleQA VerifiedSpireclimbSpireclimb sts1 a20 heartSpireclimb sts2 a10SWE-bench VerifiedMATH Level 5saturatingFrontierMath Tier 4 (v2)retiredOTIS Mock AIME 2024–25retired
claude-fable-5Anthropic·················2
claude-fable-5-1Anthropic········3·········
claude-fable-5-2Anthropic11···11··11·2···2·
claude-fable-5-2-fallbackAnthropic·······3··········
claude-opus-4-7Anthropic··············1···
claude-opus-5-5Anthropic···3··2·2·········
claude-sonnet-5-5Anthropic········1·········
gdm-ai-co-mathematicianGoogle DeepMind····1·············
gemini-3.5-flashGoogle DeepMind··············3···
gemini-4-argonGoogle DeepMind·······2··3·······
gemini-4-carbonGoogle DeepMind22·····1··2·······
gpt-5-2025-08-07OpenAI···············1··
gpt-5-mini-2025-08-07OpenAI···············2··
gpt-5.4-pro-2026-03-05OpenAI··3···············
gpt-5.4-pro-2026-03-05-web-appOpenAI····3·············
gpt-5.5-pre-releaseOpenAI··2···········2···
gpt-5.5-pro-pre-releaseOpenAI··1·2············3
gpt-5.6-solOpenAI·················1
gpt-6-astraOpenAI3··2·23··2·111··3·
gpt-6.1-solOpenAI···1·3···3·233··1·
gpt-6.1-terraOpenAI·3·········3·2····
o4-mini-2025-04-16OpenAI···············3··

A dot means the model is not in that test's top three — not that it was never tested. Click a test's name for its full list, with every score's date and setting.

What people prefer — the Arena

On the Arena, people put one question to two anonymous models and vote for the better answer, and a rating is fitted to the votes. It measures what people prefer, not whether an answer was right, so it is kept apart from the tests on this page and never averaged with them. A rating means something only beside another one: a model 100 points higher is expected to win about 64% of head-to-heads, and one 20 points higher about 53%.

Each line is the model's 95% range as the Arena publishes it, and the dashed rule marks the bottom of the leader's: a model whose line crosses it cannot yet be told apart from the leader. A model with few votes has a wide line. The same model often appears more than once, at different thinking settings, because the Arena rates each setting separately. “Open” marks a model whose weights are published, on the Arena's own licence record.

Chat, all questions421 models · leaderboard of 11 Nov 2026

Every question people asked, with the advantage of a longer or more heavily formatted answer removed — the Arena's own headline.

Gemini 4 Carbon (High) is first, but 2 others are within its margin of error.

  1. 1Gemini 4 Carbon (High)1526±10
  2. 2Gemini 4 Argon (High)1522±5
  3. 3Muse Wonder (Max)1511±6
  4. 4Claude Opus 5.5 (High)1507±6
  5. 5Claude Fable 5 (High)1504±4
  6. 6Claude Opus 4.6 (High)1503±3
  7. 7Claude Fable 5.1 (Max)1502±5
  8. 8Claude Opus 4.7 (High)1501±4
  9. 9Gemini 3.8 Flash (High)1497±4
  10. 10Muse Spark 1.3 (Max)1497±5
  11. 11Claude Opus 4.61496±3
  12. 12Claude Opus 4.71494±4
  13. 13Muse Spark 1.2 (xHigh)1493±6
  14. 14Muse Spark 1.11491±4
  15. 15Gemini 4 Hydrogen (High)1490±9
  16. 16Claude Opus 5 (High)1490±4
  17. 17Gemini 3.7 Flash (High)1489±5
  18. 18Muse Spark1489±6
  19. 19Claude Opus 5 (Max)1489±5
  20. 20Kimi K3 (Max)open1487±4

The same votes, split by what was asked. A model that leads overall but not on a kind of question you care about is the case for reading these.

Chat: coding questions

  1. 1Gemini 4 Carbon (High)1568±19
  2. 2Gemini 4 Argon (High)1553±9
  3. 3Claude Fable 5 (High)1552±6
  4. 4Claude Opus 4.7 (High)1551±6
  5. 5GPT-6 Astra (Max)1548±10

Chat: maths questions

  1. 1Gemini 4 Argon (High)1536±18
  2. 2Claude Fable 5.1 (Max)1526±19
  3. 3Claude Opus 5 (High)1521±11
  4. 4Claude Fable 5 (High)1520±13
  5. 5Gemini 3.8 Flash (High)1519±11

Chat: hard questions

  1. 1Gemini 4 Carbon (High)1547±12
  2. 2Gemini 4 Argon (High)1545±6
  3. 3Muse Wonder (Max)1531±8
  4. 4Claude Opus 5.5 (High)1530±7
  5. 5Claude Opus 4.6 (High)1530±4

Chat: creative writing

  1. 1Gemini 4 Argon (High)1523±10
  2. 2Claude Opus 5.5 (High)1517±11
  3. 3Gemini 4 Carbon (High)1516±20
  4. 4Claude Fable 5 (High)1501±7
  5. 5Claude Opus 4.6 (High)1499±6

Code Arena155 models · leaderboard of 11 Nov 2026

A separate arena: each model builds a small working web app from the same request, and people vote on the result.

Claude Fable 5.2 (xHigh) is first, but one other model is within its margin of error.

  1. 1Claude Fable 5.2 (xHigh)1838±21
  2. 2Claude Opus 5.5 (Max)1815±12
  3. 3Claude Sonnet 5.5 (xHigh)1772±10
  4. 4Gemini 4 Carbon (High)1768±15
  5. 5GPT-6 Astra (Max)1764±9
  6. 6GPT-6.1 Sol (Max)1743±10
  7. 7Claude Fable 5.1 (Max)1742±8
  8. 8Muse Wonder Full (Max)1723±14
  9. 9Claude Sonnet 5.5 (High)1715±8
  10. 10Gemini 4 Hydrogen (High)1704±14
  11. 11Claude Opus 5 (Max)1691±6
  12. 12Kimi K3.1 Pro (Max)open1687±10
  13. 13Gemini 4 Argon (High)1681±10
  14. 14GPT-6.1 Terra (Max)1678±11
  15. 15Qwen3.8-Max (0902)1676±6
  16. 16GPT-6 Sol (Max)1673±8
  17. 17Qwen3.8-Max1670±12
  18. 18DeepSeek-V4.1-Flash-20261017 (Max)open1664±7
  19. 19Kimi K3 (Max)open1658±6
  20. 20Claude Opus 5 (High)1657±5

Preference data: Arena (arena.ai, formerly LMArena), leaderboard dataset, used under CC-BY 4.0. Datum chooses which leaderboards to show and draws the ranges; the votes, ratings and ranks are the Arena's. The dataset.

How to read these numbers

A benchmark is a fixed set of questions or tasks, put to every model the same way — graduate physics problems, real bugs from open-source projects, chess positions. The percentage beside a model is the share of that particular test it got right, and nothing else: 94% on one test and 48% on another are answers to two different questions, not a ranking. Every percentage below is on its own test's scale, so two of them never compare with each other, and this page never prints them in one column or adds them up.

There is no overall winner here, and that is the finding rather than a gap. Nobody has calibrated these tests against each other, so a model can top one and sit mid-table on the next — which is information about what it is good at, not a contradiction to be averaged away into a single rank.

Each test below is kept as its own list, and a test appears here only if whoever ran it published three things: the date it was run, an error bar, and a route to the run itself. Most published benchmark numbers have none of the three.

67 further tests are collected and deliberately not published here — what else we measure, and why it is not on the Board.

Two things travel with every score here. The date is the day the test was run, never the day the model came out — the gap between those two is where a model quietly acquires the answers. And the setting is named, because the same model on the same test scores differently depending on how long it was allowed to think and how many attempts it got: on every test here that swing is wider than the gap between the top two models, and on one of them it is eighty-eight times wider. A score without its setting is a measurement of the setting.

Chess puzzlesEpoch AI

A hundred chess positions with one best move, judged by a chess engine, with the obvious captures left out.

  • 179.0% (max)±4.1claude-fable-5-2Anthropictests disagreeevaluated 2026-11-06
  • 278.0% (high)±4.2gemini-4-carbonGoogle DeepMindtests agreeevaluated 2026-11-02
  • 376.0% (promax)±4.3gpt-6-astraOpenAI74.0%–76.0% across 2 settingstests agreeevaluated 2026-10-15
  • 468.0% (promax)±4.7gpt-5.6-solOpenAI7.0%–68.0% across 4 settingstests disagreeevaluated 2026-10-14
  • 568.0% (max)±4.7gpt-6.1-solOpenAItests agreeevaluated 2026-10-13
  • 667.0% (max)±4.7claude-opus-5-5Anthropictests disagreeevaluated 2026-10-13
  • 766.0% (high)±4.8gemini-3.7-flashGoogle DeepMindtests disagreeevaluated 2026-10-13
  • 863.0% (xhigh)±4.9gpt-5.5OpenAI9.0%–63.0% across 3 settingstests disagreeevaluated 2026-10-13
  • 962.0% (xhigh)±4.9gpt-5.5-proOpenAItests agreeevaluated 2026-10-14
  • 1060.0% (xhigh)±4.9gpt-5.2-pro-2025-12-11OpenAItests agreeevaluated 2026-10-14
  • 1159.0% (xhigh)±4.9gpt-5.4-pro-2026-03-05OpenAItests agreeevaluated 2026-10-14
  • 1259.0% (max)±4.9claude-sonnet-5-5Anthropictests disagreeevaluated 2026-10-13

EBR-BenchEpoch AI

An unfamiliar campaign board game played ten times over and scored on the last games, to see whether the model learns from practice.

  • 190.5% (max)±1.5claude-fable-5-2Anthropictests disagreeevaluated 2026-11-06
  • 287.6% (high)±6.8gemini-4-carbonGoogle DeepMindtests agreeevaluated 2026-11-05
  • 378.1% (max)±5.1gpt-6.1-terraOpenAItests agreeevaluated 2026-10-26
  • 476.2% (max)±8.4gpt-6-astraOpenAItests agreeevaluated 2026-08-30
  • 571.4% (max)±7.1claude-opus-5-5Anthropictests disagreeevaluated 2026-09-22
  • 664.8% (high)±4.9gemini-4-argonGoogle DeepMindtests agreeevaluated 2026-10-13
  • 757.1% (max)±6.0claude-fable-5-1Anthropictests disagreeevaluated 2026-09-08
  • 854.3% (max)±5.8gpt-6.1-solOpenAItests agreeevaluated 2026-09-29
  • 953.3% (max)±4.6gpt-6-solOpenAItests agreeevaluated 2026-09-22
  • 1045.7% (max)±3.6claude-opus-5Anthropictests agreeevaluated 2026-09-05
  • 1144.8% (max)±4.2gpt-5.6-solOpenAItests disagreeevaluated 2026-09-04
  • 1243.8% (max)±1.8claude-haiku-5-5Anthropictests agreeevaluated 2026-10-09

FrontierMathEpoch AI

Unpublished research-level maths problems, so no model can have been trained on them.

FrontierMath: Erdős problemsEpoch AI

Sixty-eight unsolved maths problems posed by Paul Erdős: settle one, with a proof a computer can check.

  • 14.4% (max)±2.5gpt-6.1-solOpenAItests agreeevaluated 2026-10-01
  • 22.9% (max)±2.1gpt-6-astraOpenAItests agreeevaluated 2026-08-28
  • 32.9% (max)±2.1claude-opus-5-5Anthropictests disagreeevaluated 2026-09-24
  • 42.9% (high)±2.1gemini-4-carbonGoogle DeepMindtests agreeevaluated 2026-11-07
  • 52.9% (max)±2.1claude-sonnet-5-5Anthropictests disagreeevaluated 2026-10-08
  • 61.5% (max)±1.5gpt-6-solOpenAItests agreeevaluated 2026-09-30
  • 71.5% (high)±1.5gemini-4-argonGoogle DeepMindtests agreeevaluated 2026-10-20
  • 80.0% (max)±0.0gpt-5.6-solOpenAItests disagreeevaluated 2026-08-28
  • 90.0% (xhigh)±0.0gpt-5.5OpenAItests disagreeevaluated 2026-08-28
  • 100.0% (max)±0.0kimi-k3Moonshottests agreeevaluated 2026-08-28
  • 110.0% (max)±0.0claude-opus-5Anthropictests agreeevaluated 2026-08-28
  • 120.0% (max)±0.0claude-fable-5Anthropictests agreeevaluated 2026-08-28

FrontierMath Tier 4Epoch AI

The hardest tier of FrontierMath's unpublished problems — set for research mathematicians.

FrontierMath Tiers 1–3 (v2)Epoch AI

FrontierMath's main set of unpublished research-level problems, in the corrected second edition.

  • 195.1% (max)±1.3claude-fable-5-2Anthropictests disagreeevaluated 2026-11-06
  • 293.7% (max)±1.4gpt-6-astraOpenAItests agreeevaluated 2026-08-30
  • 393.7% (max)±1.4gpt-6.1-solOpenAItests agreeevaluated 2026-09-29
  • 491.2% (max)±1.7claude-opus-5-5Anthropictests disagreeevaluated 2026-09-22
  • 590.9% (max)±1.7gpt-6.1-terraOpenAItests agreeevaluated 2026-10-26
  • 690.5% (high)±1.7gemini-4-carbonGoogle DeepMindtests agreeevaluated 2026-11-02
  • 790.2% (max)±1.8claude-fable-5-1Anthropictests disagreeevaluated 2026-09-01
  • 889.8% (max)±1.8gpt-6-solOpenAItests agreeevaluated 2026-09-22
  • 989.1% (max)±1.8gpt-5.6-solOpenAItests disagreeevaluated 2026-07-09
  • 1088.8% (max)±1.9claude-sonnet-5-5Anthropictests disagreeevaluated 2026-09-29
  • 1187.7% (xhigh)±1.9gpt-5.5-proOpenAItests agreeevaluated 2026-06-12
  • 1287.0% (max)±2.0claude-fable-5Anthropictests agreeevaluated 2026-06-09

Furniture assemblyEpoch AI

Photos of half-built flat-pack furniture beside its manual: say whether the build is right so far, and if not, which step went wrong.

  • 190.0% (max)±3.9claude-fable-5-2Anthropictests disagreeevaluated 2026-11-06
  • 283.3% (max)±4.7claude-opus-5-5Anthropictests disagreeevaluated 2026-09-22
  • 380.0% (max)±4.9gpt-6-astraOpenAItests agreeevaluated 2026-09-10
  • 480.0% (max)±4.9gpt-6.1-solOpenAItests agreeevaluated 2026-09-29
  • 579.2% (max)±5.1gpt-6.1-terraOpenAItests agreeevaluated 2026-10-26
  • 677.5% (high)±5.4gemini-4-carbonGoogle DeepMindtests agreeevaluated 2026-11-03
  • 775.0% (max)±5.5claude-sonnet-5-5Anthropictests disagreeevaluated 2026-09-29
  • 872.5% (high)±5.6gemini-4-argonGoogle DeepMindtests agreeevaluated 2026-10-13
  • 970.0% (max)±5.7claude-fable-5-1Anthropictests disagreeevaluated 2026-09-10
  • 1060.8% (max)±6.2claude-opus-5Anthropictests agreeevaluated 2026-09-10
  • 1158.3% (max)±6.4gpt-6-solOpenAItests agreeevaluated 2026-09-23
  • 1256.7% (max)±6.3gpt-5.6-solOpenAItests disagreeevaluated 2026-09-10

GPQA DiamondEpoch AI

Graduate-level science questions, written so the answer cannot simply be looked up.

MirrorCodeEpoch AI

Rebuild a whole program without seeing its source, matching its output exactly on tests the model never sees.

  • 197.8% (max)±1.5claude-sonnet-5-5Anthropictests disagreeevaluated 2026-10-05
  • 295.6% (max)±2.6claude-opus-5-5Anthropic78.9%–95.6% across 2 settingstests disagreeevaluated 2026-10-30
  • 393.3% (max)±3.9claude-fable-5-1Anthropic73.3%–93.3% across 2 settingstests disagreeevaluated 2026-09-10
  • 487.8% (max)±7.2claude-opus-5Anthropictests agreeevaluated 2026-10-20
  • 578.9% (max)±8.0claude-haiku-5-5Anthropictests agreeevaluated 2026-10-21
  • 676.7% (max)±9.3gpt-6.1-terraOpenAItests agreeevaluated 2026-10-30
  • 770.0% (max)±10.4claude-fable-5Anthropic63.9%–70.0% across 2 settingstests agreeevaluated 2026-10-30
  • 865.6% (max)±10.8claude-opus-4-8Anthropictests agreeevaluated 2026-10-21
  • 964.4% (max)±11.4deepseek-v4.1-flashDeepSeektests agreeevaluated 2026-10-21
  • 1063.3% (max)±10.7claude-sonnet-5Anthropictests agreeevaluated 2026-10-21
  • 1161.1% (max)±11.3gpt-6.1-solOpenAItests agreeevaluated 2026-10-05
  • 1260.0% (max)±11.8gpt-6-astraOpenAI46.7%–60.0% across 2 settingstests agreeevaluated 2026-10-29

Mystery game puzzlesEpoch AI

Find the one best move in a puzzle version of a well-known game that is kept secret, so it cannot have been studied.

  • 190.0% (max)±3.0claude-fable-5-2Anthropictests disagreeevaluated 2026-11-06
  • 284.0% (max)±3.7gpt-6-astraOpenAItests agreeevaluated 2026-08-30
  • 380.0% (max)±4.0gpt-6.1-solOpenAItests agreeevaluated 2026-09-29
  • 474.0% (max)±4.4gpt-6.1-terraOpenAItests agreeevaluated 2026-10-26
  • 573.0% (high)±4.5gemini-4-carbonGoogle DeepMindtests agreeevaluated 2026-11-02
  • 671.0% (max)±4.6claude-opus-5-5Anthropictests disagreeevaluated 2026-09-29
  • 765.0% (max)±4.8claude-sonnet-5-5Anthropictests disagreeevaluated 2026-09-29
  • 864.0% (high)±4.8gemini-4-argonGoogle DeepMindtests agreeevaluated 2026-10-13
  • 959.0% (max)±4.9claude-opus-5Anthropic37.0%–59.0% across 2 settingstests agreeevaluated 2026-08-06
  • 1058.0% (max)±5.0gpt-5.6-solOpenAI26.0%–58.0% across 3 settingstests disagreeevaluated 2026-08-27
  • 1158.0% (max)±5.0claude-fable-5-1Anthropictests disagreeevaluated 2026-09-01
  • 1256.0% (xhigh)±5.0gpt-5.5OpenAI18.0%–56.0% across 4 settingstests disagreeevaluated 2026-08-28

SimpleQA VerifiedEpoch AI

Short factual questions with one checkable answer — a test of whether a model knows or invents.

  • 194.0% (max)±0.8claude-fable-5-2Anthropictests disagreeevaluated 2026-11-06
  • 277.7% (high)±1.3gemini-4-carbonGoogle DeepMindtests agreeevaluated 2026-11-02
  • 376.0% (high)±1.4gemini-4-argonGoogle DeepMindtests agreeevaluated 2026-10-13
  • 475.6% (max)±1.4gpt-6-astraOpenAItests agreeevaluated 2026-08-30
  • 573.9% (max)±1.4gpt-6.1-solOpenAItests agreeevaluated 2026-09-29
  • 673.5% (high)±1.4gemini-3.1-pro-previewGoogle DeepMindtests disagreeevaluated 2026-08-10
  • 772.2% (max)±1.4claude-opus-5-5Anthropictests disagreeevaluated 2026-09-22
  • 870.8% (max)±1.4claude-fable-5-1Anthropictests disagreeevaluated 2026-09-01
  • 970.7% (xhigh)±1.4claude-fable-5Anthropictests agreeevaluated 2026-08-10
  • 1069.7% (max)±1.5gpt-5.6-solOpenAItests disagreeevaluated 2026-08-10
  • 1169.7% (high)±1.5gemini-3.8-flashGoogle DeepMindtests disagreeevaluated 2026-09-02
  • 1269.2% (high)±1.5gemini-3.7-flashGoogle DeepMindtests disagreeevaluated 2026-08-27

SpireclimbEpoch AI

  • 134.0% (max)±4.1gpt-6-astraOpenAItests agreeevaluated 2026-11-04
  • 223.8% (max)±3.7gpt-6.1-solOpenAItests agreeevaluated 2026-11-04
  • 322.3% (max)±3.4gpt-6.1-terraOpenAItests agreeevaluated 2026-11-04
  • 419.6% (max)±3.5claude-fable-5-2Anthropictests disagreeevaluated 2026-11-06
  • 59.2% (max)±2.4gpt-6-solOpenAItests agreeevaluated 2026-11-05
  • 65.8% (max)±2.0claude-opus-5-5Anthropictests disagreeevaluated 2026-11-04
  • 72.0% (max)±1.1claude-fable-5-1Anthropictests disagreeevaluated 2026-11-05
  • 81.3% (max)±0.9claude-sonnet-5-5Anthropictests disagreeevaluated 2026-11-05
  • 90.0% (max)±0.0gpt-6-lunaOpenAItests disagreeevaluated 2026-11-05

Spireclimb sts1 a20 heartEpoch AI

  • 133.3% (max)±6.1gpt-6-astraOpenAItests agreeevaluated 2026-11-04
  • 221.7% (max)±5.4claude-fable-5-2Anthropictests disagreeevaluated 2026-11-06
  • 318.3% (max)±5.0gpt-6.1-solOpenAItests agreeevaluated 2026-11-04
  • 410.0% (max)±3.9gpt-6.1-terraOpenAItests agreeevaluated 2026-11-04
  • 55.0% (max)±2.8claude-opus-5-5Anthropictests disagreeevaluated 2026-11-04
  • 65.0% (max)±2.8gpt-6-solOpenAItests agreeevaluated 2026-11-05
  • 70.0% (max)±0.0claude-fable-5-1Anthropictests disagreeevaluated 2026-11-05
  • 80.0% (max)±0.0gpt-6-lunaOpenAItests disagreeevaluated 2026-11-05
  • 90.0% (max)±0.0claude-sonnet-5-5Anthropictests disagreeevaluated 2026-11-05

Spireclimb sts2 a10Epoch AI

  • 134.7% (max)±5.5gpt-6-astraOpenAItests agreeevaluated 2026-09-22
  • 234.7% (max)±5.5gpt-6.1-terraOpenAItests agreeevaluated 2026-11-03
  • 329.3% (max)±5.3gpt-6.1-solOpenAItests agreeevaluated 2026-09-30
  • 417.6% (max)±4.5claude-fable-5-2Anthropictests disagreeevaluated 2026-11-06
  • 513.3% (max)±4.0gpt-6-solOpenAItests agreeevaluated 2026-09-22
  • 66.7% (max)±2.9claude-opus-5-5Anthropictests disagreeevaluated 2026-09-23
  • 74.0% (max)±2.3claude-fable-5-1Anthropictests disagreeevaluated 2026-09-25
  • 82.7% (max)±1.9claude-sonnet-5-5Anthropictests disagreeevaluated 2026-09-29
  • 90.0% (max)±0.0gpt-6-lunaOpenAItests disagreeevaluated 2026-09-28

SWE-bench VerifiedEpoch AI

Real bugs from open-source projects: the fix has to make the project's own tests pass.

MATH Level 5Epoch AIsaturating

The hardest tier of a competition maths set that has been public for years.

Saturating: 20% of all scores are above 90%. Treat small differences near the top as noise — the benchmark is running out of room before the models run out of ability.

FrontierMath Tier 4 (v2)Epoch AIretired

The hardest tier of FrontierMath's unpublished problems, in the corrected second edition.

Retired as an instrument: some model has scored 100.0% — a perfect result — so this no longer separates the models at the top. The series is kept because when a benchmark stopped working is itself a finding, but nothing new is added to it.

  • 1100.0% (max)±0.0gpt-6.1-solOpenAItests agreeevaluated 2026-09-29
  • 2100.0% (max)±0.0claude-fable-5-2Anthropictests disagreeevaluated 2026-11-06
  • 397.6% (high)±2.4gpt-6-astraOpenAI82.9%–97.6% across 6 settingstests agreeevaluated 2026-08-30
  • 495.1% (high)±3.4gemini-4-carbonGoogle DeepMindtests agreeevaluated 2026-11-02
  • 595.0% (max)±4.1claude-opus-5-5Anthropictests disagreeevaluated 2026-09-22
  • 692.7% (max)±4.1gpt-6.1-terraOpenAItests agreeevaluated 2026-10-26
  • 790.2% (max)±4.6claude-fable-5Anthropictests agreeevaluated 2026-06-09
  • 890.0% (max)±5.2gpt-6-solOpenAItests agreeevaluated 2026-09-22
  • 987.8% (max)±5.2claude-fable-5-1Anthropictests disagreeevaluated 2026-09-01
  • 1082.9% (max)±5.9gpt-5.6-solOpenAI80.5%–82.9% across 2 settingstests disagreeevaluated 2026-07-09
  • 1180.5% (max)±6.3claude-sonnet-5-5Anthropictests disagreeevaluated 2026-09-29
  • 1278.0% (xhigh)±6.5gpt-5.5-proOpenAItests agreeevaluated 2026-06-12

OTIS Mock AIME 2024–25Epoch AIretired

Mock papers for AIME, the exam that qualifies US high-school students for the maths olympiad.

Retired as an instrument: some model has scored 100.0% — a perfect result — so this no longer separates the models at the top. The series is kept because when a benchmark stopped working is itself a finding, but nothing new is added to it.

  • 1100.0% (max)±0.0gpt-5.6-solOpenAI68.9%–100.0% across 3 settingstests disagreeevaluated 2026-08-07
  • 2100.0% (high)±0.0claude-fable-5Anthropic97.8%–100.0% across 3 settingstests agreeevaluated 2026-08-06
  • 3100.0% (xhigh)±0.0gpt-5.5-pro-pre-releaseOpenAItests agreeevaluated 2026-04-24
  • 4100.0% (xhigh)±0.0gpt-5.5-pre-releaseOpenAItests agreeevaluated 2026-04-24
  • 5100.0% (max)±0.0claude-fable-5-1Anthropictests disagreeevaluated 2026-09-01
  • 6100.0% (max)±0.0gpt-6-astraOpenAItests agreeevaluated 2026-08-30
  • 7100.0% (xhigh)±0.0qwen3.8-max-0902Alibabatests agreeevaluated 2026-09-02
  • 8100.0% (max)±0.0claude-opus-5-5Anthropictests disagreeevaluated 2026-09-22
  • 9100.0% (max)±0.0gpt-6-solOpenAItests agreeevaluated 2026-09-22
  • 10100.0% (max)±0.0claude-sonnet-5-5Anthropictests disagreeevaluated 2026-09-29
  • 11100.0% (max)±0.0gpt-6.1-solOpenAItests agreeevaluated 2026-09-29
  • 12100.0% (max)±0.0gpt-6.1-terraOpenAItests agreeevaluated 2026-10-26

What else we measure

We collect 67 more tests and keep them off the lists above, because each is missing an evaluation date or an error bar — without the first, a score could have been run after the questions leaked into a model's training; without the second, nobody can say whether the gap between two models is real. They still count as corroboration: 21 of 67 are topped by a model that also leads a test above. That is a count of agreement, not a score.

Every test we hold back
dtbench180 models · 241 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
critpt164 models · 269 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
lmca142 models · 203 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
scicode139 models · 238 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
mmlu136 models · 136 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
weirdml136 models · 172 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
ale bench127 models · 141 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
simplebench108 models · 124 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
webdev arena105 models · 134 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
arc agi101 models · 287 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
arc agi 298 models · 275 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
gsm8k93 models · 93 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
forecastbench87 models · 95 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
wino grande80 models · 80 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
arc ai277 models · 77 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
bool q77 models · 77 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
hella swag76 models · 76 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
vending bench 267 models · 72 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
apex agents61 models · 65 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
piqa60 models · 60 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
aider polyglot58 models · 71 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
bbh50 models · 50 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
live bench50 models · 52 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
video mme50 models · 50 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
frontiercode46 models · 46 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
hle46 models · 51 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
geobench45 models · 51 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
terminalbench45 models · 146 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
lech mazur writing44 models · 49 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
open book qa42 models · 42 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
enigma eval41 models · 46 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
metr time horizons40 models · 46 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
gdp pdf39 models · 56 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
trivia qa39 models · 39 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
balrog37 models · 40 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
blueprint bench 237 models · 37 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
gso31 models · 31 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
vpct28 models · 38 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
deepswe26 models · 26 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
lambada26 models · 26 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
surface evolver bench26 models · 29 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
weirdml v326 models · 26 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
deepresearchbench25 models · 41 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
frontierswe23 models · 23 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
gbaeval23 models · 23 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
science qa23 models · 23 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
cl bench21 models · 23 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
cursorbench21 models · 90 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
cybench21 models · 22 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
posttrainbench19 models · 19 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
algotune18 models · 18 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
rli17 models · 19 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
cad eval15 models · 15 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
cl bench life14 models · 17 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
the agent company14 models · 14 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
adversarial nli11 models · 11 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
gdpval11 models · 11 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
btf310 models · 12 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
exploitbench9 models · 10 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
os world9 models · 20 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
osworld 29 models · 14 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
spatialviz bench8 models · 8 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
superglue8 models · 8 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
common sense qa 26 models · 6 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
mindcube5 models · 5 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
proofbenchretired84 models · 85 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.
fictionlivebenchretired41 models · 42 scoresPublished with no evaluation date and no error bar, so we cannot say when they were run or whether a gap between two models is real.

Benchmark data: Epoch AI, 'AI Benchmarking Hub'. https://epoch.ai/benchmarks. CC-BY-4.0. Datum computes the ranking, the scaffold range and the agreement test; the measurements are Epoch's.

Which will answer?

a different question from the tests above · SpeechMap

Every model below was asked the same questions on contested political and social subjects — write the persuasive essay, make the case, argue the side — each one put four different ways. Another model read every reply and sorted it: answered in full, deflected (changed the subject, watered it down, lectured instead) or refused.

Asked the same 2,120 questions, the 57 models tested in the last 90 days answered between 24% and 100% of them in full.

This measures what a model will do, not how good it is, so it is kept apart from the tests above and plays no part in which model leads them. Answering more is not better: whether a refusal was right depends on what was asked. Every question asks for an argument, never for instructions, so a model that answers all of them has shown nothing about how it handles a request to help cause harm. The sorting was done by a model and will sometimes be wrong. With 530 questions behind each figure, two models within about 5 points of each other cannot be told apart — that margin is our estimate, not SpeechMap's. The date is when the model's replies were published, so the test ran on or before it.

answereddeflectedrefused57 models tested in the last 90 days, most answered first
  1. 1sherlock-dash-alphaUnannounced100%0% deflect · 0% refusetested by 2026-09-17
  2. 2sherlock-think-alphaUnannounced99%0% deflect · 1% refusetested by 2026-09-17
  3. 3sonoma-dusk-alphaUnannounced98%0% deflect · 2% refusetested by 2026-09-17
  4. 4sonoma-sky-alphaUnannounced93%0% deflect · 7% refusetested by 2026-09-17
  5. 5gemini-4-argonGoogle87%2% deflect · 10% refusetested by 2026-10-14
  6. 6gemini-3.8-flashGoogle86%2% deflect · 12% refusetested by 2026-09-07
  7. 7claude-opus-5.5Anthropic83%2% deflect · 13% refusetested by 2026-09-26
  8. 8claude-sonnet-5.5Anthropic82%5% deflect · 13% refusetested by 2026-09-30
  9. 9claude-haiku-5.5Anthropic79%3% deflect · 18% refusetested by 2026-10-09
  10. 10pony-alphaUnannounced77%1% deflect · 21% refusetested by 2026-09-17
  11. 11deepseek-v4-flash-vision-expDeepSeek73%2% deflect · 25% refusetested by 2026-08-27
  12. 12deepseek-v4-flash-vision-expDeepSeek72%4% deflect · 25% refusetested by 2026-08-27
show the 33 in between
  1. 13glm-5.3Z.ai71%3% deflect · 25% refusetested by 2026-08-21
  2. 14gpt-6.1-solOpenAI69%9% deflect · 22% refusetested by 2026-10-09
  3. 15ox-alphaUnannounced69%3% deflect · 28% refusetested by 2026-09-17
  4. 16quasar-alphaUnannounced69%1% deflect · 31% refusetested by 2026-09-17
  5. 17gpt-6-astraOpenAI68%10% deflect · 22% refusetested by 2026-09-26
  6. 18deepseek-v4.1-flashDeepSeek67%1% deflect · 31% refusetested by 2026-09-12
  7. 19optimus-alphaUnannounced67%3% deflect · 30% refusetested by 2026-09-17
  8. 20mimo-v2.6-flashXiaomi67%3% deflect · 30% refusetested by 2026-09-30
  9. 21union-alphaUnannounced67%6% deflect · 28% refusetested by 2026-09-17
  10. 22mimo-v2.6-proXiaomi67%4% deflect · 30% refusetested by 2026-09-30
  11. 23qwen3.6-35b-a3bAlibaba66%3% deflect · 31% refusetested by 2026-08-30
  12. 24glm-5.3-flashZ.ai65%3% deflect · 29% refusetested by 2026-08-27
  13. 25qwen3.8-max-0902Alibaba65%2% deflect · 30% refusetested by 2026-09-17
  14. 26grok-4.7xAI64%1% deflect · 34% refusetested by 2026-09-30
  15. 27gpt-6-solOpenAI64%13% deflect · 23% refusetested by 2026-09-26
  16. 28gpt-6-lunaOpenAI63%9% deflect · 27% refusetested by 2026-09-26
  17. 29hy4-previewTencent61%2% deflect · 38% refusetested by 2026-09-03
  18. 30step-5-previewStepFun60%1% deflect · 35% refusetested by 2026-10-13
  19. 31ling-3.1-flashInclusionai57%7% deflect · 35% refusetested by 2026-10-13
  20. 32gpt-5-nanoOpenAI56%10% deflect · 34% refusetested by 2026-09-26
  21. 33qwen3.8-flashAlibaba56%5% deflect · 36% refusetested by 2026-08-30
  22. 34healer-alphaUnannounced56%7% deflect · 37% refusetested by 2026-09-17
  23. 35quasar-438bCompactifai53%17% deflect · 29% refusetested by 2026-09-03
  24. 36deepseek-v4.1-flashDeepSeek53%4% deflect · 43% refusetested by 2026-09-12
  25. 37elephant-alphaUnannounced51%3% deflect · 46% refusetested by 2026-09-17
  26. 38healer-alphaUnannounced49%7% deflect · 45% refusetested by 2026-09-17
  27. 39mistral-large-4-0Mistral49%3% deflect · 49% refusetested by 2026-10-09
  28. 40qwen3.8-27bAlibaba46%6% deflect · 48% refusetested by 2026-08-18
  29. 41polaris-alphaUnannounced46%10% deflect · 44% refusetested by 2026-09-17
  30. 42aurora-alphaUnannounced46%1% deflect · 53% refusetested by 2026-09-17
  31. 43granite-4.2-8bIBM43%10% deflect · 46% refusetested by 2026-09-17
  32. 44mercury-2.5Inception40%6% deflect · 53% refusetested by 2026-09-17
  33. 45qwen3.8-27bAlibaba39%5% deflect · 57% refusetested by 2026-08-18
  1. 46owl-alphaUnannounced35%12% deflect · 52% refusetested by 2026-09-17
  2. 47qwen3.8-flashAlibaba34%5% deflect · 60% refusetested by 2026-08-30
  3. 48mercury-2.5Inception32%3% deflect · 65% refusetested by 2026-09-17
  4. 49hunter-alphaUnannounced32%13% deflect · 55% refusetested by 2026-09-17
  5. 50horizon-betaUnannounced32%8% deflect · 60% refusetested by 2026-09-17
  6. 51hy4-previewTencent31%12% deflect · 57% refusetested by 2026-09-03
  7. 52granite-4.2-8bIBM29%4% deflect · 66% refusetested by 2026-09-17
  8. 53qwen3.6-35b-a3bAlibaba29%4% deflect · 67% refusetested by 2026-08-30
  9. 54solar-mini4Upstage29%4% deflect · 67% refusetested by 2026-09-30
  10. 55seed-2-1-turboByteDance27%32% deflect · 40% refusetested by 2026-09-17
  11. 56hunter-alphaUnannounced26%7% deflect · 67% refusetested by 2026-09-17
  12. 57seed-2-1-turboByteDance24%49% deflect · 25% refusetested by 2026-09-17

Not ranked, because more than 10% of their requests failed or were blocked by the provider before the model replied: muse-spark-1.3 (59% failed), muse-wonder (55% failed), qwen3.8-omni-flash (13% failed), claude-fable-5.1 (11% failed).

354 older or undated readings are held and not listed; SpeechMap keeps them all.

Refusal data: SpeechMap (speechmap.ai), by xlr8harder. Responses and judgments: github.com/xlr8harder/speechmap-data, Apache-2.0. Datum chooses the window, the order and the margin; the replies and the sorting are SpeechMap's. speechmap.ai