// category · 17 tools · 1 open source
LLM Benchmarks
The standard tests used to compare models on knowledge, reasoning, coding and abstraction. Check what each one measures, how it is scored and whether results may be contaminated.
Benchmarks are fixed test sets that let you compare AI models on the same questions under the same rules. This category lists the benchmarks that appear most often in model launch posts and leaderboards, from older knowledge exams to recent agent and coding tests, with a short note on what each one measures.
A benchmark is only useful while it still separates strong models from weak ones. Older exams such as MMLU saturated once top models passed 90 percent, which is why harder sets followed: MMLU Pro, GPQA for graduate level science, and Humanity's Last Exam, a set of 2,500 expert written questions from the Center for AI Safety and Scale AI. In September 2026 the HLE authors also released HLE Diamond, a cleaned 1,000 question subset split into reasoning and knowledge halves; our HLE Diamond guide explains what it is and how to read its scores.
Other entries test different skills. SWE-bench and SWE-Bench Pro measure whether a model can fix real software issues. ARC-AGI tests learning a new rule from a few examples rather than recalling knowledge. GDPval grades work products from real occupations. RAID is different again: it benchmarks AI text detectors, not language models.
Before you trust a reported score, check five things:
- Which version of the set. Full, text only, verified or a cleaned subset can all carry the same name.
- Tools or no tools. Web search and code execution can add more than 20 points.
- Settings. Reasoning effort, token limits and the judge model all move results.
- Who ran it. A lab's own number and an independent run often differ.
- Noise. A one or two point gap on a few hundred questions may mean nothing.
Contamination is the other risk. When test questions leak into training data, a benchmark starts measuring memory instead of skill, which is why many datasets are gated and carry canary strings.
For a full walkthrough, read LLM benchmarks explained, MMLU vs MMLU Pro and MMLU vs SWE-bench. To see how benchmark scores are combined into rankings, browse the leaderboards category.
// guides
Guides to LLM benchmarks
SWE-Bench Pro Explained: V1, V2, Scores
SWE-Bench Pro explained: how Scale AI's coding benchmark differs from Verified, what V2 changed, and the November 2026 full, HARD-51 and private scores.
How to Use the MMLU Benchmark, Step by Step
How to use MMLU benchmark runs: pick the right variant, run it with lm-evaluation-harness or Inspect, smoke test the format, and read the score honestly.
MMLU vs SWE-bench: Which Should You Use?
MMLU vs SWE-bench explained: what each measures, why the scores are not comparable, and which benchmark to trust for knowledge vs coding agents in 2026.
LLM Benchmarks Explained: What the Scores Mean
LLM benchmarks explained: what MMLU, GPQA Diamond, Humanity's Last Exam, SWE-bench and ARC-AGI measure, why scores saturate, and how to spot contamination.
Gemini 4 Neon Benchmarks: Which Claims Hold Up
Gemini 4 Neon benchmarks checked against Artificial Analysis, Vals, Arena, Terminal-Bench, Zapier and Cognition boards: which launch claims match and which do not.
MMLU vs GPQA: Key Differences and Which to Use
MMLU vs GPQA explained: broad exam recall versus expert written science questions, why GPQA scores run lower, and which benchmark still means anything in 2026.
GDPval Benchmark: 1,320 Real Work Tasks, 44 Jobs
GDPval is OpenAI's AI test of 1,320 real work tasks from 44 occupations. How grading works, how to read the GDPval-AA board, and what it misses.
MMLU vs MMLU Pro: Key Differences and Which to Use
MMLU vs MMLU Pro explained: how the 4 option and 10 option benchmarks differ, why their scores aren't comparable, and which one to trust for LLMs in 2026.
Gemini 4 Hydrogen Benchmarks: What Holds Up
Gemini 4 Hydrogen benchmarks checked against Artificial Analysis, Vals, Terminal-Bench, Snorkel and Arena boards: which launch scores hold up and which do not.
ARC-AGI vs Humanity's Last Exam: Key Differences
ARC-AGI vs Humanity's Last Exam: one tests how fast a model learns a new puzzle, the other tests expert knowledge. Scores, grading, cost and which to trust.
Terminal-Bench Explained: 4.0, 4.1, Scores
Terminal-Bench explained: how the 66 task agent shell benchmark works, what changed in 4.0 and 4.1, and how to read official, Vals and Artificial Analysis scores.
Artificial Analysis Intelligence Index v5 Explained
Artificial Analysis Intelligence Index v5: the nine evaluations, new 25% weights, what was dropped, why old scores no longer compare, and how to read a v5 score.
FrontierCode Benchmark Explained: How It Scores
The FrontierCode benchmark explained: how Cognition grades whether maintainers would merge AI code, what Main and Diamond mean, and how to read the leaderboards.
HLE Diamond Leaderboard and Scores (November 2026)
HLE Diamond leaderboard for November 2026: Claude Fable 5.2 leads both official tables, 72.4% on Scale's 21 model board and 74.3% in the authors' table.
LLM Leaderboards Compared: Which Ones to Trust
Every major LLM leaderboard compared: how Arena, Artificial Analysis, LiveBench, SEAL, Epoch AI and Vellum score models, and where each one can mislead you.
Agent Arena Index: How Arena Ranks AI Agents
The Agent Arena Index explained: the four signals behind Arena's agent leaderboard, how the score is weighted and anchored, and how to read the November 2026 board.
SO-Arena Benchmark: How System One Models Rank
SO-Arena tests System One decision models on 2,919 tasks in 11 domains. Who leads, why the top scorer is only seventh on reliability, and how to read it.
Harness Leaderboard: How the Harness Moves Scores
The Harness Leaderboard runs 9 coding agent harnesses on the same open models. What v0.1.3 shows, who ties for first, what it costs, and the caveats.
VIALS-AA: What Scientific Image Scores Mean
VIALS-AA is Artificial Analysis's independent board for Handshake AI's 161 task scientific image test. How pass cubed works, what scores show, and the limits.
Coding Agent Benchmark Rankings (October 2026)
Coding agent benchmark rankings from SWE-Bench Pro, FrontierCode, Terminal-Bench and the Artificial Analysis index, with sourced scores, dates and why they differ.
Artificial Hivemind: Why AI Models Sound Alike
The Artificial Hivemind paper explained: why AI models give the same answers, what Infinity-Chat and NoveltyBench measure, and how to test output diversity yourself.
GPTZero vs QuillBot: Which Catches Paraphrased AI?
GPTZero vs QuillBot in 2026: free limits, prices, our read of both RAID result files, and which detector caught QuillBot paraphrased AI text in outside tests.
GPTZero vs ZeroGPT: Which AI Detector to Trust?
GPTZero vs ZeroGPT in 2026: who makes each, free limits, prices, and what RAID and four independent studies found about false flags on real human writing.
Pangram vs Copyleaks: Which AI Detector Wins?
Pangram vs Copyleaks in 2026: price per 1,000 words, free limits, plagiarism, LMS plugins, and what the 2023 and 2026 studies found when each detector was tested.