The benchmark data could not be loaded.
Serve this directory over HTTP with benchmark.json beside this page.
Loading the benchmark…
01 / Model evaluations
AI leaderboard
One row per model version and evaluation run. Rankings compare the same assigned questions, access track and budget. Subset runs are welcome; results from separate runs are not pooled.
| Rank in cohort | Model / version | Evaluation run | Verified points | Verification |
|---|
Each eligible ordered pair is worth one point. Implications count once within a run; cohort ranks compare matching assignments, tracks and budgets.
02 / Evaluation protocol
One model. One declared run.
Choose the questions.
Assign a subset or the full frozen suite. The class definitions and cited baseline are public. The release audit records which questions were open at the cutoff.
Run the model.
Choose an explicit model version and token and time limits. The OpenAI and Anthropic adapters preserve prompts, answers, proof files and provider usage.
Review the evidence.
Check the run record and each proof. New ordinary claims use the Lean proof checker; independence metatheorems require explicit expert review. Existing cited results are trusted.
Count the resolutions.
Combine accepted proofs with the frozen implications. Count each resolved pair admitted by the cutoff audit once. A reviewed run with no solutions can score zero.
Read the full evaluation protocol ↗ · Get the frozen candidate suite ↗
03 / Public task suite
The questions to put to an AI.
Rows are A. Columns are B.
Each cell asks whether A ⊆ B.
No classes match these filters.
The release records historical decisions for the open questions. Those decisions can be revised if a pre-cutoff result was missed; known results earn no point. Circuit uniformity and language-versus-promise conventions are part of the task.
04 / How credit propagates
Scoring examples
Assuming the claims are proved
and the affected pairs are eligible.
These examples explain the scorer. They are not AI evaluations, submissions or forecasts.
Follow the reasoning
A proof trail for every point.
Submit a proof
Submit a multi-file Lean project using trusted textbook results and uniform integer-array interfaces with explicit complexity and size guarantees. For additional results published by the cutoff, review checks the cited statement and its match to the benchmark’s model. You do not need to reprove those results.
05 / Scope & provenance
What the score measures.
The benchmark measures an AI run's ability to resolve open questions about total-language complexity classes. It is intended for mathematical reasoning beyond the current public baseline. It does not attempt to cover every kind of progress in complexity theory: promise-only separations, individual algorithmic speedups, search problems and fixed-exponent lower bounds can fall outside this suite.
Design audit
The design audit documents what this class roster can and cannot measure.
Source registry
The cited baseline and implication rules are trusted premises for this release. Existing literature proofs do not need to be re-formalized to run the benchmark. Each positive point still requires a historical eligibility review.
—