Gemini 4 Hydrogen benchmarks hold up best where Google borrowed the number from an independent board. The Artificial Analysis Intelligence Index (54 at xhigh, 51 at the default high setting), HLE-Diamond at 58.8%, AutomationBench-AA at 76.1%, first place on Vals Tax Agent Bench at 52.30% and fifth on Arena's Agent Arena all check out on the public boards as of November 13, 2026. The headline agent scores do not: Terminal-Bench 4.1 at 57%, MKA at 53.4% and Agents' Last Exam at 71.6% are Google's own runs with no matching row on the official boards yet, and the "tied for first" Legal Research Bench claim does not match the Vals page.

Google released Hydrogen on November 12 as the fast, cheaper member of the Gemini 4 family (launch post). This guide does not review the model. It takes each benchmark in the launch post and model card, finds out who ran it, and checks it against the board that owns the test.

Gemini 4 Hydrogen benchmarks at a glance

BenchmarkGoogle's figureWho ran HydrogenPublic board on Nov 13
AA Intelligence Index v5.0.151 (high), 54 (xhigh)Artificial AnalysisMatches
HLE-Diamond58.8%Artificial AnalysisMatches
AutomationBench-AA76.1%Artificial AnalysisMatches
Vals Tax Agent Bench52.3%, firstVals AIMatches: 52.30%, first
Vals Legal Research Bench55.8%, tied firstVals AIDoes not match: 53.85%, sixth
Agent ArenafifthArenaMatches (preliminary)
Terminal-Bench 4.157.0%GoogleNo Hydrogen row
MKA53.4%GoogleNo Hydrogen row on AA
Agents' Last Exam v1.171.6%GoogleNo Hydrogen row
OSWorld-2.1, SWE-sweep, Chartography, Terminal-Bench-SciencevariousGoogleSelf computed

Where the launch numbers came from

The answer is in Google's evaluation methodology page, which most coverage skipped. Three rules shape every number in the model card table:

  • Hydrogen scores use the Gemini API at the highest thinking setting unless noted, and they are pass@1 with no majority voting.
  • Scores for the two comparison models, Claude Sonnet 5.5 and GPT-6.1 Terra, are "sourced from providers' self reported numbers unless otherwise mentioned."
  • Several rows say where the data came from. HLE-Diamond and AutomationBench come from the Artificial Analysis leaderboard. MKA competitor scores come from Artificial Analysis, but Hydrogen's MKA score is self computed. Terminal-Bench 4.1, Agents' Last Exam, OSWorld, SWE-sweep and Chartography are self computed for Hydrogen.

That split matters more than any single score. A borrowed number was produced by someone with no stake in the launch. A self computed number used Google's harness, Google's settings and Google's choice of which run to report.

The scores you can check on independent boards

Artificial Analysis Intelligence Index. On the Hydrogen model page the xhigh setting scores 53.9 (shown as 54) and high scores 50.8, against 50.4 for Gemini 4 Argon at high and 30.8 for Gemini 3.8 Flash. So "Hydrogen beats Argon" is true by less than half a point at the default level. Our Intelligence Index v5 explainer covers what the nine evaluations inside it measure.

HLE-Diamond and AutomationBench-AA. Both figures in the model card are Artificial Analysis runs, and the board data matches to the decimal: 58.8% on HLE-Diamond and 76.1% on AutomationBench for Hydrogen at xhigh, with Sonnet 5.5 at 53.3% and 71.8%, and Terra at 47.3% and 60.2%. Note what "leads its class" hides: Gemini 4 Carbon scores 69.1% on the same HLE-Diamond run. For what Humanity's Last Exam and its Diamond subset test, see our HLE guide.

Fabricated claims. Google says Hydrogen cuts fabricated claims by more than five times compared with 3.8 Flash. Artificial Analysis has an independent proxy: on AA-Omniscience its hallucination rate is 8.5% for Hydrogen at xhigh against 55.1% for 3.8 Flash at high, with accuracy almost unchanged (50.5% for both). That supports the direction of the claim, though Google did not say which test it used.

Vals Tax Agent Bench. The Vals leaderboard, updated November 11, puts Hydrogen first at 52.30%, ahead of Gemini 4 Carbon at 51.78% and Claude Fable 5.1 at 50.75%, at $3.87 per test. The lead is about half a point on 193 private questions, so treat it as a tie at the top rather than a clear win. Vals grades with an LLM judge and only counts a question if every rubric check passes.

Agent Arena. Arena's agent leaderboard, whose header reads November 11 and 64 models, lists Gemini 4 Hydrogen (High) fifth, with a net improvement of 14.80% over 6,465 sessions and a rank range of 3 to 10. It is marked preliminary, and Arena's changelog says Hydrogen was added on November 12. Expect the rank to move as sessions build up, and note that Arena rebuilt the board on November 18 as the Agent Arena Index. Arena measures behavior in real agent sessions, not a fixed test set, which our leaderboard comparison explains.

The scores only Google has run

Terminal-Bench 4.1: 57.0%. Google ran Hydrogen in a mini-swe-agent vision harness, averaged over five attempts per task, at the high effort level, with "cheating adjusted" accuracy. The official 4.1 leaderboard has no Hydrogen row. Its top rows are Claude Code with Opus 5.5 and Fable 5.2 at 65.76%, then Sonnet 5.5 at 63.94%. Gemini 3.8 Flash sits at 18.18% in mini-SWE-agent, which is where Google's "more than tripling" comparison comes from. The board also lists GPT-6.1 Terra in Codex at 53.33%, while Google's table shows a dash for Terra. Since Terminal-Bench scores swing with the agent harness (see our harness leaderboard guide), a mini-swe-agent run and a Claude Code run are not the same contest. Artificial Analysis's independent Terminal-Bench 4.0 run, a different version, puts Hydrogen xhigh at 53.5% against 19.7% for 3.8 Flash, so the jump over Flash is real even if the exact 4.1 figure is unconfirmed. Our Terminal-Bench explainer covers what changed in 4.1.

MKA: 53.4%. This is the most interesting gap. Google's MKA comparison numbers for Sonnet 5.5 (52.0%) and Terra (43.8%) match the Artificial Analysis MKA board exactly. Hydrogen's own 53.4% does not appear there: the board lists 21 models and Hydrogen is not one of them. If the runs were comparable, 53.4% would sit between Claude Opus 5.5 (53.0%) and Gemini 4 Carbon (54.3%), behind Claude Fable 5.2 (56.6%). Mixing a self computed score with a third party's competitor scores is the pattern to watch for in any launch table.

Agents' Last Exam v1.1: 71.6%. Google used its own Antigravity CLI 1.4.8 harness with a five hour window, and the methodology says it used the task version from October 17 "without applying the Oct 26, 2026 update." The official v1.1 board, released October 26, has no Hydrogen row. It ranks by pass rate, where Gemini 4 Carbon leads at 45.9%. The 71.6% is the partial credit score, the column where Carbon reaches 77.4% and Terra 69.8%. Sonnet 5.5's 60.4% in Google's table was also run by Google, in the ALE-Claw harness; Sonnet is not on the official board.

The rest. OSWorld-2.1 uses an 82 task offline subset, and Hydrogen and Sonnet scores are the best of three runs. SWE-sweep and Chartography are self computed. On Terminal-Bench-Science Google reports 51.4% for Hydrogen, while Artificial Analysis measured 49.5% at xhigh. None of these is wrong by default, but none has been repeated by anyone else yet.

The launch post says Hydrogen "ties for #1 on the Vals Legal Research Bench (55.8%)." The Vals Legal Research Bench page, updated November 11, shows Gemini 4 Carbon first at 55.77% and Gemini 4 Hydrogen sixth at 53.85%, behind Muse Spark 1.3 Max, Claude Opus 5, Claude Fable 5.1 (all 55.29%) and Gemini 4 Argon (54.81%). The 55.8% figure is Carbon's score. We could not find a Hydrogen row at 55.8% on the page. It may come from an earlier snapshot or a different setting, but neither the post nor the methodology page says so. Until Google or Vals explains it, quote the Vals page, not the launch post.

What MKA actually measures

MKA (Multimodal Knowledge Agent) is a new Google DeepMind benchmark that Artificial Analysis runs on 200 tasks across 10 industry verticals. Each task gives an agent a workspace of files where the key facts sit in images: CAD drawings, schematics, map rasters, lab plates, charts. The agent must produce a real deliverable such as a slide deck, spreadsheet or report. Artificial Analysis runs it in its Stirrup harness with up to 250 turns, and GPT-6 Luna (medium) judges each rubric item as met or not. The headline Climb WSR is the weighted share of the harder "Climb" criteria met per task.

We downloaded the ten public showcase tasks from Google DeepMind's Hugging Face dataset to see what a task looks like. Across the ten there are 835 rubric criteria, from 48 to 134 per task, weighted 1 to 5 (236 at weight 1, 51 at weight 5). Slide decks dominate: 452 of the 835 criteria grade a presentation file. Inputs range from 3 to 12 files per task and from under 1 MB to about 947 MB for the GIS task. One limit for anyone hoping to reproduce scores: the public rubrics carry no Climb or Regression labels, so you cannot compute the headline metric from them. MKA is a hard test of visual knowledge work, and right now only the board's own runs can rank models on it.

Thinking level changes the answer

Every Hydrogen score depends on the setting. On Artificial Analysis the index falls from 54 at xhigh to 51 at high, 41 at medium and 27 at low. The xhigh run cost $3.16 per index task and produced 304 million output tokens across the index, which ranks it 76th of 77 models for verbosity. So the API price of $1.50 per million input tokens and $7.50 per million output tokens understates the cost of the top scores. Arena tested the high setting; Google's Terminal-Bench 4.1 run used high; most other model card rows used the highest setting. When you compare Hydrogen with another model, match the effort level first, as our guide to evaluating model performance recommends.

How to read any launch table

  1. Open the methodology page before the chart. Look for "self computed" and "self reported."
  2. Find the board that owns the test and check for a row with the same model and setting.
  3. Check the harness. Agent scores from different harnesses are not comparable.
  4. Check the version and date. Agents' Last Exam and Terminal-Bench both changed tasks in late October and early November.
  5. Look at the gap to second place and the sample size. Half a point on 193 questions is noise.
  6. Match the thinking level and note the token cost.

The same checks applied to Google's next release are in our Gemini 4 Neon benchmarks check.

FAQ

Is Gemini 4 Hydrogen better than Gemini 4 Argon?

On the Artificial Analysis Intelligence Index, slightly: 50.8 at high against 50.4 for Argon at high, and 53.9 at xhigh. On Vals Legal Research Bench Argon scores higher (54.81% against 53.85%). Treat them as close, with Hydrogen cheaper per token.

Which Gemini 4 Hydrogen benchmarks are independent?

As of November 13, 2026: the Artificial Analysis index, HLE-Diamond, AutomationBench-AA and AA-Omniscience, Vals Tax Agent Bench and Legal Research Bench, and Arena's Agent Arena. Terminal-Bench 4.1, MKA, Agents' Last Exam, OSWorld-2.1, SWE-sweep and Chartography figures were run by Google.

What is the MKA benchmark?

MKA, or Multimodal Knowledge Agent, is a Google DeepMind benchmark of 200 professional tasks where key information is in visual files and the agent must produce a deliverable. Artificial Analysis runs it and grades with an LLM judge against weighted rubrics. Claude Fable 5.2 leads its board at 56.6% Climb WSR.

Where does Gemini 4 Hydrogen rank on Terminal-Bench?

It has no row on the official Terminal-Bench 4.1 board yet. Google reports 57.0% from its own runs. On the Artificial Analysis Terminal-Bench 4.0 run, Hydrogen at xhigh scores 53.5%.

Google's post says 55.8% and tied first, but the Vals page updated November 11 shows Hydrogen sixth at 53.85%, with 55.77% belonging to Gemini 4 Carbon. Neither Google nor Vals has explained the difference.