// benchmark explainer
GPQA Diamond, explained
By the TrueIQ editorial team at Ahzy Capital · Last reviewed Oct 15, 2026 · How we work
GPQA is a set of very hard biology, physics and chemistry questions written by PhD level experts. Diamond is the highest quality subset and the version most labs report.
| Made by | Rein et al., NYU and others (2023) |
|---|---|
| Format | Multiple choice, 4 options |
| Size | 448 questions in the full set; 198 in Diamond |
| Paper | arXiv 2311.12022 |
What it measures
Questions were written so that skilled nonexperts with web access still do badly, which is why the paper calls them 'Google-proof'. In the paper, domain experts did far better than nonexperts with unlimited web search. That makes GPQA a test of real scientific reasoning rather than lookup.
Where it falls short
- The Diamond set is small (198 questions), so one or two questions can move the score noticeably.
- It covers only three sciences.
- Leading models now score above the expert baseline reported in the paper, so it separates top models less than it used to.
How to read a score
Treat differences of a few percentage points as roughly equal. Check whether a reported score is a single run or an average of several runs.
Where to see current scores
We don't republish scores. Leaderboards change weekly, and numbers depend on how the test was run. Check these instead:
- Artificial Analysis: GPQA Diamond results (independently run)
- GPQA paper