GDPval Benchmark Explained: Tasks, Scores, Limits

The GDPval benchmark is OpenAI's test of whether AI models can do real professional work: 1,320 tasks from 44 occupations in the nine sectors that contribute most to US GDP, each built from an actual work product by an expert with about 14 years of experience. A model has to hand back a finished deliverable (a spreadsheet, a slide deck, a legal memo, a nursing care plan), and the score is how often expert graders rate it as good as or better than the human expert's version. OpenAI no longer hosts a GDPval leaderboard, so the numbers you see in launch posts today almost always come from GDPval-AA, a separate agentic run of the 220 public tasks by Artificial Analysis, graded by AI judges.

That distinction is the most common source of confusion, and this guide covers it in detail: what GDPval tests, how each version is graded, how to read the current leaderboard, and what the benchmark leaves out. The timing is apt. On October 16, 2026, Mercor's Benchmark Map reached the Hacker News front page with its estimate that 90 percent of US working hours go to tasks existing benchmarks barely cover. GDPval is one of the few serious attempts to close that gap.

What the GDPval benchmark measures

OpenAI introduced GDPval on September 25, 2025 (OpenAI; paper, arXiv:2510.04374). The design starts from the economy, not from an exam:

1. Pick every US sector that contributes more than 5 percent of GDP. That gave nine sectors, from real estate and manufacturing to government, health care and information.
2. In each sector, pick the five occupations with the largest total wages that are mostly digital work. OpenAI counted an occupation as digital if at least 60 percent of its O*NET task statements (the US Labor Department's job taxonomy) could be done on a computer.
3. Have experienced professionals in each occupation write tasks based on their own past work, with the reference files a colleague would get.

The result spans software developers, lawyers, registered nurses, mechanical engineers, financial analysts, customer service representatives, concierges, film editors and journalists. Each task went through at least three and on average five rounds of expert review. Tasks come with reference files (up to 38 in the full set) and expect real file outputs, including CAD drawings, audio, video and spreadsheets.

In plain words: the public part is 220 tasks, five per occupation, and each one takes a professional most of a working day. The other 1,100 tasks stay private, which limits contamination but means outsiders can only run the gold subset.

How GDPval grading works

GDPval does not check answers against a key. Graders who work in the task's occupation see the request, the reference files and two or more unlabeled deliverables, one of them by the human expert. They rank them, explain why, and mark the model's work as better than, as good as, or worse than the expert's. A single comparison took graders over an hour on average.

Because the metric is a win rate against a baseline rather than a percent correct, OpenAI argues GDPval has no fixed ceiling: once models beat today's experts, the baseline can be swapped for stronger work.

Two caveats from the paper itself are worth knowing:

• Blinding is imperfect. Graders were not told which output was which, but style could give models away. The authors note OpenAI outputs often used em dashes, Claude outputs often wrote in the first person, and Grok sometimes named itself.
• The automated grader is a proxy. OpenAI released an experimental grading model at evals.openai.com. It agreed with expert graders 65.7 percent of the time, against 70.8 percent agreement between two experts. It is less reliable on outputs from capable OpenAI models, consistent with known self preference in model judges, and 12 of the 220 public tasks can't be graded by it at all.

If you plan to use model judges on tasks like these, read our guide to LLM as a judge first. It also helps to know why models and judges tend to agree with each other, covered in our Artificial Hivemind guide.

What the first GDPval results showed

OpenAI ran GPT-4o, o4-mini, o3, GPT-5, Claude Opus 4.1, Gemini 2.5 Pro and Grok 4 on the gold subset in 2025. Claude Opus 4.1 did best, with 47.6 percent of its deliverables rated as good as or better than the expert's, and stood out on formatting and slide layout. GPT-5 was strongest on accuracy and instruction following. OpenAI's own models improved roughly linearly over time.

The headline that models were "100x faster and cheaper" needs its fine print. That ratio compares raw model run time and API cost with expert time. The paper also models a realistic workflow: an expert asks the model once, spends time reviewing the output, and redoes the task if it falls short. Reviews averaged 109 minutes. Under that setup, GPT-5 was only about 1.12 times faster and 1.18 times cheaper than the expert working alone. Letting the expert retry the model several times before taking over raised that to about 1.39 times faster and 1.63 times cheaper.

Other findings that hold up well:

• The most common reason experts preferred the human deliverable was that the model did not fully follow instructions. GPT-5 lost mostly on formatting.
• Win rates were highest on tasks an expert finishes in under two hours and fell as tasks got longer.
• Among GPT-5's losing deliverables, experts rated roughly 3 percent as catastrophic, meaning harmful or dangerously wrong if used.
• More reasoning effort, more task context and better prompting all raised scores.

GDPval vs GDPval-AA: which leaderboard is which

OpenAI's hosted GDPval leaderboard is no longer active, and its page points readers to Artificial Analysis instead (OpenAI Evals). So when someone quotes a "GDPval score" in October 2026, they almost always mean Artificial Analysis's GDPval-AA v2.1, which works differently (methodology):

The judge panel currently consists of GPT-5.6 Sol, Gemini 3.8 Flash and Claude Opus 5, so no single lab's model grades every match. Version 2.1 changed only how the Elo scale is fixed, but that is enough to make v2 and v2.1 numbers incomparable. GDPval-AA also feeds 10 percent of the Artificial Analysis Intelligence Index v4.3.2, so it shapes the composite scores covered in our LLM leaderboards guide.

The practical rule: an OpenAI win rate and a GDPval-AA Elo are different units from different setups. Never put them in the same column.

How to read the GDPval leaderboard today

On the GDPval-AA v2.1 leaderboard as of October 17, 2026, the top three entries are Claude Opus 5.5 at max effort (1866), Meta's Muse Wonder Full at max effort (1857, released in October), and Claude Sonnet 5.5 at max effort (1838). The board lists 296 models.

Three habits make those numbers useful:

1. Check the confidence interval before calling a winner. The top entry's interval is plus or minus 26 points and the second's is plus or minus 25. A 9 point gap is far inside that, so treat the top two as statistically tied.
2. Note the effort setting. The same model can sit in very different places. Claude Opus 5.5 scores 1866 at max effort, 1704 at high and 1235 at low. A score without its effort level is half a fact.
3. Look at cost per task. Artificial Analysis plots Elo against average cost per task, because higher effort usually means many more tokens. The best value pick is often not the top row.

What the board cannot tell you is how those models do on your files, your formats and your reviewers. That is a job for your own test set (more on that below).

What GDPval does not cover

OpenAI is open about the limits (paper):

• One shot and fully specified. Every task arrives with complete context. Real work involves figuring out what the client wants, then revising after feedback. In a test with deliberately vague prompts (42 percent of the original length), GPT-5 did worse because it struggled to find the context it needed.
• Digital knowledge work only. Manual and physical tasks are excluded, as are tasks that need tacit knowledge, private data, proprietary software or talking to other people.
• Narrow within each job. Five public tasks per occupation can't represent a whole profession. The gold subset touches about 14 percent of the O*NET task statements for its 44 occupations.

Mercor's Benchmark Map shows the wider version of the same problem. It mapped 29 agentic, economically focused benchmarks onto all 923 O*NET occupations and 18,797 tasks, and found that only 208 occupations have even one task covered by at least five benchmark prompts. Tasks rated as exposed to AI were about six times more likely to be covered than unexposed ones, and occupations with bigger total wages were overrepresented: lawyers' tasks are heavily covered, while similarly exposed legal secretaries and court clerks barely appear. Mercor calls these results provisional, since a longer benchmark list would raise coverage. The direction is still clear: benchmarks like GDPval sample a small, well paid slice of work.

How to use GDPval in your own evaluation

GDPval is most useful as a shortlist signal when your work produces documents, spreadsheets and slides. To turn it into a decision:

1. Start from the public tasks. Download the 220 gold tasks and pick the ones closest to your occupation and file types.
2. Add your own. Write 20 or more tasks from your real backlog, with the reference files and a gold deliverable from your best person. Our guide on how to evaluate LLM model performance covers fixed settings, paired comparisons and error bars.
3. Grade blind. Have your own experts compare outputs with labels removed. Use a model judge only after you've checked it agrees with your people.
4. Pair it with the right specialist tests. For code, SWE-Bench Pro (our explainer) is the better signal. For academic reasoning, see Humanity's Last Exam and HLE Diamond. Our benchmarks explainer and MMLU vs SWE-bench show where each one fits.

The GDPval tool page collects the official links in one place.

FAQ

What is GDPval?

GDPval is an OpenAI benchmark that measures how well AI models perform economically valuable, real world tasks. It has 1,320 expert written tasks across 44 occupations in the nine largest US sectors by GDP, and models are judged on whether their deliverables match or beat a human expert's.

Is the GDPval dataset public?

Partly. OpenAI open sourced a gold subset of 220 tasks, five per occupation, with prompts and reference files on Hugging Face. The other 1,100 tasks of the full set are not public.

What is the difference between GDPval and GDPval-AA?

GDPval is OpenAI's dataset and its original evaluation, graded by human experts against expert deliverables. GDPval-AA is Artificial Analysis's independent run of the 220 public tasks, with models acting as agents in its Stirrup harness and a panel of AI judges comparing models head to head. GDPval reports win rates; GDPval-AA reports Elo ratings.

Which model leads the GDPval leaderboard?

On the GDPval-AA v2.1 leaderboard as of October 17, 2026, Claude Opus 5.5 at max effort leads with 1866 Elo, just ahead of Muse Wonder Full at 1857. Their confidence intervals overlap heavily, so the top two are effectively tied. Rankings change as new models are added.

How long does a GDPval task take a human?

On the public gold subset, the average expert completion time is about 404 minutes, close to a full working day, with an estimated value of about $361 at median wages. Some tasks in the full set take far longer.

Does a high GDPval score mean AI can do a whole job?

No. GDPval covers a few precisely specified, one shot tasks per occupation and leaves out physical work, ambiguity and collaboration. OpenAI itself notes that most jobs are more than a list of tasks that can be written down.