The Agent Arena Index is the headline score on Arena's agent leaderboard, launched on November 18, 2026. It ranks models by how they behave in real agent sessions on arena.ai, not on a fixed test set, by combining four signals: Task Completion, Praise vs Complaint, Steerability and Bash Reliability. On the November 18 board, Claude Fable 5.2 (Max) leads at 1,448, a clear 112 points ahead of Claude Opus 5.5 (Max) at 1,336 and Gemini 4 Neon (High) at 1,333, which are statistically tied. We rebuilt all 68 scores from Arena's published data and found one detail the launch post leaves out: Bash Reliability counts half as much as each of the other three signals.
This guide explains what each signal measures, how the score is scaled, what our recomputation showed, and how to use the index next to fixed benchmarks before you pick a model for agent work. For the wider picture of public boards, see our LLM leaderboard comparison.
What the Agent Arena Index measures
Arena's Agent Mode lets people hand real tasks to a randomly assigned model: write code, fix a project, research a topic, build a document. Arena then mines those sessions for outcomes. Its signal definitions page (last updated November 18) describes the four signals that feed the index:
| Signal | What counts | What it rewards |
|---|---|---|
| Task Completion | Answers to the popup "Did this complete your task?" Tasks with no answer are excluded | Tasks users confirm as done |
| Praise vs Complaint | Turns with explicit praise or complaint; a task succeeds when praise outnumbers complaints | Work users react well to |
| Steerability | Every assistant turn with a user reply, scored by steering effort | Models that need fewer corrections, and land them the first time |
| Bash Reliability | Every bash call, marked success, failure or exempt | Fewer errors the agent caused, such as a mistyped flag |
Task Completion is the renamed Confirmed Success signal, with no change to how it is measured. Bash Reliability replaced Bash Recovery on October 12, according to the notes on Arena's dataset card. Arena retired a fifth signal, Tool Hallucination, on November 18 because frontier models now rarely call tools that do not exist (launch post).
The key point: every signal comes from users doing their own work. Nobody grades a fixed answer key. That makes the index good at catching what people actually find frustrating, and weak at telling you how a model handles a specific hard task you care about.
How the score is calculated
Arena's original methodology post estimates, for each model, how much better or worse its sessions go than the average active model on each signal, after normalizing each signal to z scores. Because models are assigned at random, this works like a controlled experiment rather than a popularity contest.
The November 18 update kept that core and changed the scale. The new score is 1,000 times the old net improvement score plus an "anchor" offset. One index point is one thousandth of a standard deviation of the signal. The anchor is reset at each release so that well established models with many sessions keep the same average score, which means a model no longer drops just because a strong newcomer raised the field average. Arena set the scale so the average model in July 2026 scored about 1,000. The offset is the same for every model, so it never changes the order.
The launch post says signals are "combined into a single index through averaging." Our recomputation shows the averaging is weighted, which the next section covers.
What we found when we recomputed the board
Arena publishes its leaderboard snapshots as an open dataset, lmarena-ai/leaderboard-dataset, with one table for the index and one per signal. We downloaded the November 18 agent tables on November 20 and checked the math.
The weights are 2, 2, 2 and 1. A linear fit of the 68 index scores against the four signal scores is exact: Task Completion, Praise vs Complaint and Steerability each carry 2/7 of the weight, and Bash Reliability carries 1/7. With an offset of 1,098.75, the formula reproduces every published score within 0.003 points and every rank. A plain equal average does not.
The half weight matters for Gemini. All four Gemini 4 models score below average on Bash Reliability, ranking 46th (Hydrogen) to 55th (Neon) of 68 on that signal, while Claude models hold the top three bash spots. With equal weights, Gemini 4 Carbon would fall from fourth to sixth, and Claude Sonnet 5.5 (Max) and GPT-6.2 Astra (Max) would each move up one place. The top three stay the same either way.
Second to sixth place overlap. Fable 5.2's lower confidence bound (1,409.8) sits well above anyone else's upper bound, so its lead is real. The next five models do not separate cleanly:
| Rank | Model | Index | 95% interval | Median cost per task |
|---|---|---|---|---|
| 1 | Claude Fable 5.2 (Max) | 1,448 | 1,410 to 1,486 | $9.13 |
| 2 | Claude Opus 5.5 (Max) | 1,336 | 1,309 to 1,362 | $5.80 |
| 3 | Gemini 4 Neon (High), preliminary | 1,333 | 1,300 to 1,365 | $2.54 |
| 4 | Gemini 4 Carbon (High) | 1,303 | 1,295 to 1,312 | $1.36 |
| 5 | Claude Sonnet 5.5 (Max) | 1,296 | 1,259 to 1,333 | $3.68 |
| 6 | GPT-6.2 Astra (Max) | 1,291 | 1,256 to 1,326 | $3.33 |
Index and interval figures come from the dataset; costs are the board's medians over the last 14 days, read from the Agent Arena leaderboard on November 20. Every one of ranks 3 to 6 has an upper bound above Opus 5.5's lower bound of 1,309.5. Treat that group as one tier and let cost decide.
Session totals disagree across Arena's own pages. The board header says 3,093,488 sessions, and the per model session counts in the dataset add up to exactly that. The launch post says "over 2.9M user sessions," and the leaderboard changelog entry for the same day says "over 10.9M." The board figure is the one the data supports.
Google's Neon claim checks out on sessions. Google said Neon's debut score rested on more than 4,200 agent sessions. The index table lists 4,238 for Gemini 4 Neon (High). Its score has since moved from the 1,336 Google quoted to 1,333, as our Gemini 4 Neon benchmark check covers.
Dropping Tool Hallucination reshuffled the lower ranks. The dataset still carries an older aggregate table that also counts Tool Hallucination. Compared with it, the new index keeps the top 14 in the same order but moves 29 of 68 models, mostly by one place. Claude Haiku 5.5 (High) gains the most, from 41st to 36th. The launch post's line that rescaling "does not alter model rankings" is true of the rescaling alone, not of the signal change.
The narrower indexes and the Alignment Index
Alongside the headline number, Arena added six focused views:
- Agentic Work, Coding and Chat: the same four signals, limited to one task category. Fable 5.2 leads all three. On Coding, GPT-6.2 Astra (Max) is third at 1,338, above Neon, which ranks fifth.
- Task Outcome: Task Completion and Praise vs Complaint only, weighted equally. Neon is second here at 1,424, ahead of Opus 5.5.
- Steering: Steerability alone. Neon ranks second on this signal behind Fable 5.2.
- Tool Use: Bash Reliability alone for now, "with more signals to come."
The Alignment Index, introduced on October 8 with 27 models, stays separate and is scored 0 to 100. It flags sessions with an unauthorized action, a false attribution to the user, or a deceptive claim of completion. On November 17 Arena changed how it detects these: LLM judges from several labs now review every sampled session, and a session is flagged only when most of them agree. Arena says this catches more failures users never noticed, so most models now show higher failure rates. If you use the index to choose an agent that touches real files or accounts, read the Alignment Index too. Our guide to LLM as a judge covers why majority voting across judges helps.
Agent Arena Index vs fixed benchmarks
The index answers "how do real users fare with this model in Arena's agent harness?" Fixed benchmarks answer "can this model solve these specific tasks?" You usually want both.
| Question | Better source |
|---|---|
| Will users need to correct it often? | Agent Arena Index (Steerability) |
| Does it finish everyday mixed tasks? | Agent Arena Index (Task Completion) |
| Can it fix real repository issues? | SWE-bench family |
| Can it work in a terminal on hard tasks? | Terminal-Bench |
| How smart is it across many tests at once? | Artificial Analysis Intelligence Index |
Three limits to keep in mind. Every score reflects Arena's own agent harness and tool set, so a model can rank differently inside Claude Code, Codex or your own stack; our coding agent benchmark rankings show how much the harness can change results. The users are Arena's users, and the task mix leans toward coding, research and planning. And new models carry a "preliminary" label until sessions build up, so early ranks move. Gemini 4 Hydrogen was fifth on November 15 and tenth on November 18, after Neon and GPT-6.2 Astra entered above it and the scores were rebuilt.
How to read the board before choosing a model
- Start with the interval, not the rank. Models whose ranges overlap are a tie for practical purposes.
- Check the signal breakdown. A model that leads on Praise vs Complaint but trails on Bash Reliability may suit research and writing more than shell heavy work.
- Weigh cost per task. On November 20, Gemini 4 Carbon sat in the tied group at about a quarter of Opus 5.5's median cost per task.
- Look for a "preliminary" tag and come back in a week before relying on a new model's rank.
- Confirm with one fixed benchmark that matches your work, then test on a handful of your own tasks.
FAQ
What is the Agent Arena Index?
It is the headline score on Arena's agent leaderboard, launched November 18, 2026. It combines four signals mined from real Agent Mode sessions (Task Completion, Praise vs Complaint, Steerability and Bash Reliability) into one number, scaled so the average model in July 2026 scored about 1,000.
Who is first on the Agent Arena leaderboard?
On the November 18, 2026 board, Claude Fable 5.2 (Max) is first at 1,448, followed by Claude Opus 5.5 (Max) at 1,336 and Gemini 4 Neon (High) at 1,333. Fable's lead is outside the confidence intervals; second through sixth place overlap.
Is the Agent Arena Index a simple average of its signals?
No. Arena describes it as averaging, but the published data fits a weighted average exactly: 2/7 each for Task Completion, Praise vs Complaint and Steerability, and 1/7 for Bash Reliability, plus a fixed offset.
How is the Agent Arena Index different from the old Agent Arena leaderboard?
Scores are now 1,000 times the old net improvement score plus an anchor that keeps established models stable between releases. Tool Hallucination was retired, Confirmed Success was renamed Task Completion, and six narrower indexes were added.
Can I download Agent Arena data?
Yes. Arena publishes every leaderboard release as the lmarena-ai/leaderboard-dataset on Hugging Face, with an agent table, per signal tables, and per category index tables. Agent data starts on June 4, 2026.
