Funded benchmarks

Sort: Newest
Terminal-Bench 4.1

Evaluates terminal agents on continuously updated software-engineering tasks.

With Harbor, Laude Institute
Link for Terminal-Bench 4.1
Joules-to-GDP

A framework that measures how well AI turns energy into economic value and shows which tasks, models, and hardware give the best return.

With Stanford, UC Berkeley, UT Austin
Link for Joules-to-GDP
Agents’ Last Exam v1.1

Evaluating AI agents on long-horizon, economically valuable professional workflows with verifiable outcomes.

With UC Berkeley RDI, RDI Foundation
Link for Agents’ Last Exam v1.1
OSWorld 2.1

Evaluates computer-use agents on 108 long-horizon workflows.

With XLANG Lab (University of Hong Kong)
Link for OSWorld 2.1
MBABench 2.0

Evaluating AI agents on building financial spreadsheets from scratch

With Columbia Business School
Link for MBABench 2.0
LibraryDesignBench

Measures how well AI agents design libraries that other agents can use.

With DARPA, National Science Foundation, Laude Institute
Link for LibraryDesignBench
Terminal-Bench-Science

Evaluates agents on scientific workflows derived from researchers’ own work.

With Stanford, Laude Institute, Harbor
Link for Terminal-Bench-Science
Terminal-Bench 4.0

Evaluates terminal agents on continuously updated software-engineering tasks.

With Harbor, Laude Institute
Link for Terminal-Bench 4.0
Senior SWE-bench

Evaluates coding agents on senior-level software engineering tasks.

With Princeton University, UW-Madison
Link for Senior SWE-bench
Terminal-Bench 3.0

Evaluates terminal agents on containerized tasks across seven domains.

With Harbor, Laude Institute
Link for Terminal-Bench 3.0
Agents’ Last Exam

Evaluating AI agents on long-horizon, economically valuable professional workflows with verifiable outcomes.

With UC Berkeley RDI, RDI Foundation
Link for Agents’ Last Exam
OSWorld 2.0

Evaluates computer-use agents on 108 long-horizon workflows.

With XLANG Lab (University of Hong Kong)
Link for OSWorld 2.0
Terminal-Bench 2.1

28 of 89 tasks fixed; continuous validation added to track benchmark reliability.

With Stanford, Laude Institute, Harbor
Link for Terminal-Bench 2.1
Continual Learning Bench

Measures improvement across sequential, stateful tasks.

With UC Berkeley SkyLab, UW-Madison
Link for Continual Learning Bench
SlopCodeBench

Tracks code quality as agents repeatedly extend their own code under changing specs.

With UW-Madison, DARPA, National Science Foundation
Link for SlopCodeBench
Terminal-Bench 2.0

Challenges terminal agents with 89 hard, human-verified tasks in containerized environments.

With Stanford, Laude Institute, Harbor
Link for Terminal-Bench 2.0
JudgmentBench

Assesses models on 30 real-world legal tasks using 1,539 rubric scores and 1,530 pairwise judgments from practicing attorneys.

With Stanford Law (LIFT Lab)
Link for JudgmentBench
ARC-AGI-3

Tests models’ ability to explore and acquire skills in novel environments through hundreds of hand-crafted interactive games.

With ARC Prize Foundation
Link for ARC-AGI-3

In development

FrontierCS 2

Evaluates agents on 48 open-ended computer science R&D tasks.

With UC Berkeley, Princeton
Link for FrontierCS 2
LegalBench 2

Evaluates AI autograders on their ability to judge the quality of other AI systems’ legal work.

With Stanford University
Link for LegalBench 2
KoraBench

Evaluates how safely AI models and apps behave with children and teens, across 26 child-specific risks.

With KORA
Link for KoraBench
PhilosophyBench

Assesses AI systems’ capabilities and limitations in English-language philosophy writing.

With Stanford
Link for PhilosophyBench
CollusionBench

Tests whether internal activations can reveal covert coordination among AI agents.

With Carnegie Mellon University
MedPAIR

Compares physician trainees and LLMs on clinical relevance across 2,000+ questions, using sentence-level expert labels.

With MIT
Link for MedPAIR
STELLA-Bench

Tests LLM safety in 30-turn conversations with vulnerable users, guided by an adaptive director model.

With Atella AI
Image

Call for proposals

We are seeking applications from researchers, labs, and engineers building benchmarks for the next wave of AI capabilities. We're looking for benchmarks that drive the fundamental axes for AI agency (read more on our blog) and welcome independent directions from the research community as well.

Apply for a grant