AI benchmarks explained: what the 2026 scores really mean

Every alarming or exciting AI headline traces back to a benchmark, and almost nobody explains what the benchmark is. Here are the AI benchmarks explained in plain English. A benchmark is a standardised exam for AI models: a fixed set of questions or tasks, scored the same way every time, so you can compare models and track progress. Below: the ones that matter in 2026, what each tests, who leads today, and where each quietly misleads.

AI benchmarks explained: who leads in September 2026

Leaderboards change every few weeks, so a score only means something with its date and its source. These are the latest published results from independent evaluators and the vendors themselves.

Benchmark	What it measures	Latest leading result	Date and source
Epoch Capabilities Index	A composite of many benchmarks	GPT-6 Astra, 167	28 Sep 2026, Epoch AI
Artificial Analysis Intelligence Index	Ten evaluations, from coding to knowledge work	Claude Opus 5.5 (max), 58	22 Sep 2026, Artificial Analysis
Humanity's Last Exam	Very hard expert questions	Claude Opus 5.5, 61.4%	22 Sep 2026, Artificial Analysis
Terminal-Bench 4.0	Agentic coding in a terminal	Claude Opus 5.5 at 59.6%, level with GPT-6 Astra	22 Sep 2026, Artificial Analysis
FrontierMath Erdős	68 unsolved maths problems	GPT-6 Astra solved 2 (3%)	3 Sep 2026, Epoch AI
METR time horizon	Length of task an agent completes half the time	Claude Mythos Preview, about 17 hours	8 May 2026, METR
LMArena text	Which answer people prefer, by vote	Claude Opus 5.5 (early votes)	September 2026, LMArena
ARC-AGI-2	New visual logic puzzles	Gemini 3.1 Pro, 77.1% verified	February 2026, Google
SWE-bench Verified	Fixing real GitHub issues	Best submitted agent, 79.2%	Official leaderboard

METR says its measurements above 16 hours are unreliable with its current task suite. Vendor-reported scores, such as Google's ARC-AGI-2 figure, are the vendor's own numbers.

SWE-bench Verified: can it fix real code?

SWE-bench Verified takes real issues from open-source Python projects on GitHub and asks an AI agent to fix them. The clever part is the marking: the fix is checked by running the project’s own automated tests. There is no partial credit for sounding right. Either the code works or it does not. The Verified set has 500 tasks that human developers checked for clear descriptions and fair tests.

On the official leaderboard, the best submitted agent resolves 79.2% of the tasks. Treat the number with care: the leaderboard only shows what teams submit, and Epoch AI says some samples may remain ambiguous, with an error rate it has estimated at 5 to 10%. For the newest agentic coding, evaluators such as Artificial Analysis also use newer tests like Terminal-Bench 4.0.

GPQA Diamond: hard science questions

GPQA Diamond is a set of graduate-level multiple-choice questions in biology, chemistry and physics, written by PhD-level experts and designed to resist a quick web search. With four options, guessing scores 25%. When OpenAI recruited PhD experts to sit it, they scored 69.7%, according to Epoch AI.

GPQA was the benchmark behind many “AI beats PhDs” headlines in 2024 and 2025. It is not among the ten evaluations in Artificial Analysis’s current Intelligence Index, which now leans on harder tests such as Humanity’s Last Exam. That is the normal life cycle of a benchmark: once the best models do well on it, evaluators move to something harder.

Humanity’s Last Exam and ARC-AGI: the harder exams

Humanity’s Last Exam has 2,500 challenging questions across more than a hundred subjects, written by nearly 1,000 subject experts, and it keeps a private set of questions to catch models that have memorised the public ones. On Artificial Analysis’s run, Claude Opus 5.5 scored 61.4% in September 2026, ahead of the previous best of 59.1% from Claude Fable 5.1.

ARC-AGI tests something different: solving entirely new logic patterns in visual puzzles, which rewards working out a new rule on the spot rather than recalling knowledge. When Google launched Gemini 3.1 Pro in February 2026, it reported a verified score of 77.1% on ARC-AGI-2, more than double Gemini 3 Pro’s.

Time horizons: how long can an agent work?

METR, an independent research group, measures how long a task an AI agent can complete. It times human experts on software tasks, then finds the task length at which an agent succeeds half the time. In its latest public update, on 8 May 2026, a preview of Anthropic’s Claude Mythos reached about 17 hours, and Claude Opus 4.6 about 12 hours. METR notes that measurements above 16 hours are unreliable with its current tasks, and it estimates the time horizon has doubled roughly every four months since 2023.

This is the benchmark that best explains why AI agents became practical in 2026: the tasks they can finish have grown from minutes to many hours.

Leaderboards and composite indexes

Composite indexes blend many tests into one number. Epoch AI’s Capabilities Index put GPT-6 Astra on top at 167 when it updated on 28 September 2026, and Epoch reports that Astra solved 2 of 68 unsolved Erdős maths problems. Artificial Analysis’s Intelligence Index put Claude Opus 5.5 on top at 58 on 22 September. The two indexes weigh different tests, so they can disagree.

LMArena works differently: people compare two anonymous answers and vote for the better one. It measures preference, not correctness. In September 2026 Claude Opus 5.5 led its text leaderboard, but on only a few thousand votes, so its position may move.

The Maths Olympiad: the reasoning milestone

The International Mathematical Olympiad is one of the hardest reasoning competitions in the world. In July 2025, Google DeepMind said an advanced Gemini Deep Think system scored 35 of 42 points, solving five of six problems, and the IMO’s president confirmed it as a gold-medal score. These problems demand long, creative, multi-step proofs.

It is a genuine milestone, and the benchmark most likely to be over-read. An Olympiad problem is clean, fully specified and has a correct answer. Your business is none of those.

How a benchmark misleads, and how to read one honestly

Benchmarks fail you in three predictable ways. First, a clean exam is not your messy workflow: the test is curated, your work is not. Second, models can be tuned for popular tests, and vendors choose which results to publish. Third, a high average hides the tail, and in business it is the confident mistakes that cost you.

The honest way to read any benchmark: find out what it measures, prefer independent evaluators and tests checked by running code or hidden questions, check the date, then try the tool on your own data before you trust a single number. That last step is the one most people skip. MIT’s NANDA initiative reported in August 2025 that only about 5% of enterprise generative AI pilots delivered rapid revenue gains, and it pointed to poor fit with real workflows rather than weak models. A strong benchmark score does not fix a badly aimed project. If the terms are new to you, start with what an LLM is, and for the bigger picture read AI in 2026.