ARC-AGI

arcprize.org · Benchmarks

Free

#2 in Benchmarks

Abstraction and Reasoning Corpus series from Francois Chollet and the ARC Prize Foundation that tests skill acquisition on novel tasks; ARC-AGI-3 adds interactive games.

#benchmark#reasoning#generalization#prize

Visit ARC-AGI ↗GitHub ↗

About the Benchmarks category

The standard tests used to compare models on knowledge, reasoning, coding and abstraction. Check what each one measures, how it is scored and whether results may be contaminated.

ARC-AGI is one of 5 benchmarks tools indexed on TrueIQ. Facts on this page come from the tool's official site and public APIs; prices and features change, so confirm details on the official website before you commit.

// related

Related tools

All Benchmarks →

BIG-benchgithub.comBeyond the Imitation Game: a collaborative benchmark of 200+ diverse tasks from 450 authors at 132 institutions, built to probe and extrapolate language model capabilities.BenchmarksFree★ 3.2k

GPQAgithub.comGraduate-level, Google-proof science Q&A: 448 expert-written biology, physics and chemistry questions, with a 198-question Diamond subset widely reported in model launches.BenchmarksFree★ 540

SWE-benchswebench.comBenchmark of real GitHub issues from popular Python repositories where a fix counts only if the repo's tests pass; includes the Verified, Lite and other subsets.BenchmarksFree★ 6k

MMLUgithub.comMassive Multitask Language Understanding: a multiple-choice benchmark spanning 57 subjects from math to law, introduced in 2020 and now largely saturated by frontier models.BenchmarksFree★ 1.6k

LiveBenchlivebench.aiBenchmark refreshed with new questions regularly and scored against objective answers to limit test-set contamination.LeaderboardsOpen source★ 1.3k

Aider Leaderboardsaider.chatAider's code-editing benchmarks that test how well models edit real source files across several programming languages.LeaderboardsFree★ 49.4k