Open Benchmarks Grants // Rolling acceptances open

$30M for the next wave of open benchmarks

We’re expanding Open Benchmarks Grants by 10x, to a $30M commitment, to support researchers, domain experts, and open-source teams pushing the frontier of evaluation.

Propose a benchmark How to get involved ↓

25+

open benchmarks funded or in development since launch, across a dozen institutions

Cited in frontier model cards

Terminal-Bench, Terminal-Bench-Science, OSWorld, and Agents’ Last Exam.

In partnership with

Three axes of frontier evaluation: autonomy horizon, environment complexity, and output complexity

Benchmarks that define and advance the frontier

Benchmarks are guideposts for—and drivers of—AI progress. But when benchmark development falls behind frontier model development, their value for guiding, evaluating, and aligning models falls with it, and agents are taking on more autonomous, higher-stakes work.

Keeping pace with the frontier requires more diverse, robust, and continuously evolving benchmarks, built and maintained by a broader ecosystem of researchers and domain leaders, and kept open so the field can inspect methods and reproduce findings. Through Open Benchmarks Grants, we support the people and research needed to build and sustain them.

Announcement · October 7, 2026

Frontier AI is Accelerating.
Open Benchmarks Need to Keep Up.

Read →

Steering committee

Karthik Narasimhan

Princeton University

Professor of Computer Science

Chris Ré

Stanford University

Associate Professor ·
Co-founder, Snorkel AI

Ludwig Schmidt

Stanford University

Assistant Professor

Yu Su

Ohio State University

Professor

Lewis Tunstall

Hugging Face

Machine Learning Engineer

Fred Sala

Univ. of Wisconsin–Madison

Assistant Professor

Get involved

Work with us on the frontier of evaluation

If you have a capability worth measuring, a domain to measure it in, or a benchmark that needs hardening, there is a way in.

Propose a benchmark

Expert data, research and engineering support, and compute for teams building open benchmarks and new measurement methods.

Propose a benchmark Details ↓

Join as a Research Fellow

Researcher-led evaluation with a Snorkel research co-advisor and funded access to experts, compute, and engineering support. Fellows keep their roles and affiliations.

Apply as a Fellow Details ↓

Red-team your benchmark

We help builders find and fix weaknesses in their benchmarks: reward-hacking exploits, contamination, instruction ambiguity, broken verifiers, insufficient distributional diversity, stale task sets. Findings go to the team first.

Submit your benchmark Details ↓

What we’re funding

Propose a benchmark

We invite proposals where the stakes are highest and measurement is hardest, building on the priorities from launch: more realistic environments, longer autonomy horizons, more open-ended outputs. Independent directions welcome.

What we’re looking for

Safety & alignment

Cybersecurity

Physical AI

Long-running work

Open-ended outputs

Dynamic settings

Human uplift

Scientific agents

What a grant includes

Expert data

Clinicians, lawyers, scientists, and engineers to label, generate, and verify data through Snorkel’s expert network.

Research and engineering support

Pipeline design, schema validation, and quality control from Snorkel’s research team.

Compute and hosting

Compute credits, hosting, and distribution through program partners.

Propose a benchmark

Snorkel Research Fellowship

A flexible program for active academics and builders working on frontier evaluation. Fellows set their own research questions and lead the work from design through release, with a Snorkel research co-advisor and funded access to experts, compute, and research and engineering support.

What fellows get

A co-advisor from Snorkel’s research team

Funded access to domain experts, compute, and research and engineering support

Fellows keep their roles and academic affiliations; open release with academic attribution

Who it’s for

Postdocs and PhD students working on evaluation

Domain experts entering AI evaluation: scientists, clinicians, lawyers, security researchers

How it works

Submit your application and cover letter

Scope the project with your co-advisor; build with experts, compute, and the Snorkel platform

Release openly, with attribution

Apply to be a Research Fellow

Open Benchmarks Red Team

We work with builders to identify and fix weaknesses in their evaluations: reward hacking, contamination, instruction ambiguity, broken verifiers, insufficient distributional diversity, stale task sets. Expert review plus specialized QC systems reproduce failures and test fixes. Findings go to the team first.

What we test for

Reward hacking and grader exploits

Verifier or grader reliability

Task specification and solvability

Contamination and leakage

Environment robustness

Difficulty and saturation

How it works

Tell us about your benchmark

What it measures, how outputs are graded, what stage it’s in, and what you’d like examined.

We identify failure cases

Domain-expert review combined with specialized QC systems to identify and reproduce failure cases.