Benchmarks
Every source-backed benchmark, grouped by cognitive dimension — open any chart for its detail and cost view
Abstract Reasoning
IQ BenchmarksChess Puzzles ScoresEpoch’s 100 novel chess positions, presented as text (FEN). Exact best-next-move accuracy; no vision input. Chess knowledge and planning both contribute. Contributes to Abstract Reasoning using a provisional calibrated scale.Data: Epoch AIOpen chartIQ BenchmarksARC-AGI-2 ScoresEach model's ARC-AGI-2 score. Color = provider.Data: ARC PrizeOpen chartIQ BenchmarksARC-AGI-1 Benchmark ScoresEach model's ARC-AGI-1 score. Color = provider.Data: ARC PrizeOpen chartIQ BenchmarksMystery Game Puzzles ScoresEpoch’s 100 text-state puzzles in an undisclosed game. Best-next-move accuracy in a minimal agent scaffold; invalid or missing moves receive no credit. Prompts and transcripts are private. Contributes to Abstract Reasoning using a provisional calibrated scale.Data: Epoch AIOpen chartIQ BenchmarksARC-AGI-3 Benchmark ScoresEach model's ARC-AGI-3 score. Color = provider.Data: ARC PrizeOpen chart
Mathematical Reasoning
IQ BenchmarksFrontierMath Tier 1-3 Benchmark ScoresEach model's FrontierMath Tier 1-3 score. Color = provider.Data: Epoch AI FrontierMathOpen chartIQ BenchmarksFrontierMath Tier 4 Benchmark ScoresEach model's FrontierMath Tier 4 score. Color = provider.Data: Epoch AI FrontierMathOpen chartIQ BenchmarksAIME Benchmark ScoresEach model's AIME score. Color = provider.Data: Vals.ai, Artificial AnalysisOpen chartIQ BenchmarksMathArena Benchmark ScoresEach model's MathArena expected performance. Color = provider.Data: MathArenaOpen chartIQ BenchmarksProofBench 1.1 ScoresNative Lean 4.25.2 verification including native_decide. Version 1.1 only; not comparable to the unversioned historical cohort. Contributes to Mathematical Reasoning using a provisional calibrated scale.Data: ProofBench 1.1Open chart
Academic Reasoning
IQ BenchmarksHumanity's Last Exam Benchmark ScoresEach model's Humanity's Last Exam score. Color = provider.Data: Artificial Analysis, Artificial Analysis Intelligence Index v4.3.2Open chartIQ BenchmarksSciCode Benchmark ScoresEach model's SciCode score. Color = provider.Data: Artificial Analysis, Artificial Analysis Intelligence Index v4.3.2Open chartIQ BenchmarksCritPt Benchmark ScoresEach model's CritPt percentage score. Color = provider.Data: Artificial AnalysisOpen chartIQ BenchmarksGPQA Diamond Benchmark ScoresEach model's GPQA Diamond score. Color = provider.Data: Artificial Analysis, Artificial Analysis Intelligence Index v4.3.2Open chartIQ BenchmarksMMMU-Pro Benchmark ScoresEach model's MMMU-Pro score. Color = provider.Data: Artificial Analysis, Artificial Analysis Intelligence Index v4.3.2Open chartIQ BenchmarksMMLU-Pro Benchmark ScoresEach model's MMLU-Pro score. Color = provider.Data: Manual source capture, TIGER-Lab MMLU-Pro leaderboard (missing rows only)Open chart
Programmatic Reasoning
IQ BenchmarksTerminal-Bench 4.0 ScoresExact AA v4.3.2 model/configuration; never substitute older benchmark versions. Contributes to Programmatic Reasoning using a provisional calibrated scale.Data: Artificial AnalysisOpen chartIQ BenchmarksLiveCodeBench Benchmark ScoresEach model's LiveCodeBench score. Color = provider.Data: Vals.ai, Vals.ai LiveCodeBenchOpen chartIQ BenchmarksProgramBench ScoresAlmost Resolved score: share of ProgramBench tasks passing at least 95% of hidden tests. Color = provider.Data: Vals.ai ProgramBenchOpen chartIQ BenchmarksIOI V2 Scores2024–2026 olympiad tasks, no interactive grading feedback. Version 2 only; separately calibrated from archived V1. Contributes to Programmatic Reasoning using a provisional calibrated scale.Data: Vals.ai IOI V2Open chartIQ BenchmarksSWE-rebench Benchmark ScoresEach model's SWE-rebench resolved rate. Color = provider.Data: SWE-rebenchOpen chartIQ BenchmarksMirrorCode ScoresEpoch ML,+Private,2L: 30 tasks, three repeats, seven days and 10B tokens per attempt. Average rate of completely solved programs; distinct from the paper cohort. Contributes to Programmatic Reasoning using a provisional calibrated scale.Data: Epoch AIOpen chart
Computer Use
IQ BenchmarksArena.ai Agent Arena ScoresAggregate net improvement across real-world Agent Mode sessions. Color = provider.Data: Arena.ai Agent ArenaOpen chartIQ BenchmarksBrowseComp Benchmark ScoresEach model's BrowseComp score. Color = provider.Data: LLM Stats, LLM Stats BrowseCompOpen chartIQ BenchmarksMCP Atlas Benchmark ScoresReal-world MCP tool-use pass rates. Color = provider.Data: Scale Labs, Scale Labs MCP AtlasOpen chartIQ BenchmarksAgents' Last Exam Benchmark ScoresFull / Overall pass rates from Agents' Last Exam. Color = provider.Data: Agents' Last ExamOpen chartIQ BenchmarksOSWorld-Verified Benchmark ScoresEach model's OSWorld-Verified score. Color = provider.Data: LLM Stats, OSWorld, LLM Stats OSWorld-VerifiedOpen chart
Reliability
IQ BenchmarksAA Omniscience ScoresArtificial Analysis Omniscience scores for factual reliability. Color = provider.Data: Artificial AnalysisOpen chartIQ BenchmarksAA Long Context Reasoning v1.1 ScoresExact AA v4.3.2 model/configuration; never substitute older benchmark versions. Contributes to Reliability using a provisional calibrated scale.Data: Artificial AnalysisOpen chartIQ BenchmarksIFBench Benchmark ScoresInstruction-following and constraint-adherence scores. Color = provider.Data: Artificial Analysis, Artificial Analysis Intelligence Index v4.3.2Open chartIQ BenchmarksBullshitBench v2 ScoresClear Pushback rate: share of attempts where a model clearly challenges a false premise instead of accepting nonsense. Color = provider.Data: BullshitBench v2Open chartIQ BenchmarksSimpleQA Verified ScoresCorrect-answer rate on Epoch AI's verified factuality set. Color = provider.Data: Epoch AI SimpleQA VerifiedOpen chartIQ BenchmarksFACTS Grounding ScoresGoogle FACTS Grounding scores for long-form responses fully grounded in provided documents. Color = provider.Data: Kaggle FACTS Grounding leaderboardOpen chart
Knowledge Work
IQ BenchmarksGDPval-AA v2.1 normalized score ScoresExact AA v4.3.2 model/configuration; never substitute older benchmark versions. Contributes to Knowledge Work using a provisional calibrated scale.Data: Artificial AnalysisOpen chartIQ BenchmarksAA-Briefcase v1.1 Elo ScoresExact AA v4.3.2 model/configuration; never substitute older benchmark versions. Contributes to Knowledge Work using a provisional calibrated scale.Data: Artificial AnalysisOpen chartIQ BenchmarksAutomationBench-AA ScoresExact AA v4.3.2 model/configuration; never substitute older benchmark versions. Contributes to Knowledge Work using a provisional calibrated scale.Data: Artificial AnalysisOpen chartIQ BenchmarksGDP.pdf ScoresExact AA v4.3.2 model/configuration; never substitute older benchmark versions. Contributes to Knowledge Work using a provisional calibrated scale.Data: Artificial AnalysisOpen chartIQ BenchmarksAA-AnalystAgent ScoresQuantitative analysis from spreadsheets and documents. Headline pass^5: correct on every one of five attempts, not pass@5. Contributes to Knowledge Work using a provisional calibrated scale.Data: Artificial AnalysisOpen chartIQ BenchmarksEnterpriseOps-Gym-AA ScoresStrict success on 1,117 oracle-mode enterprise workflows, graded against final database state using AA’s Stirrup harness. Contributes to Knowledge Work using a provisional calibrated scale.Data: Artificial AnalysisOpen chart
Applied Domain Benchmarks
These benchmarks contribute to their applied domain scores.
Applied Domain BenchmarksCode Migration ScoresFull standalone benchmark: 120 CLI migrations across 30 repositories plus 10 COBOL-to-Java repositories, weighted 75/25. Hidden behavioral test pass rate; not the Vals Index subset. Contributes to Software Engineering using a provisional calibrated scale.Data: Code MigrationOpen chartEmotional & InteractionEQ-Bench 4 Elo ScoresPublished multi-judge Elo on 120 personas and 16 turns. No inherited EQ-Bench 3 Anthropic penalty. Subjective roleplay evaluation; not a validated human EI measure. Contributes to Emotional Reasoning using a provisional calibrated scale.Data: EQ-Bench 4 EloOpen chart
Diagnostic Benchmarks
Standalone versioned results outside composite scoring. Cost views use the site’s effective-cost estimate.
Diagnostic BenchmarksVals Index v2 ScoresVersion 2 cohort only. Accuracy, cost and latency must come from the same source configuration. Highest published effort; Max and default retained separately in source snapshot. Standalone raw result; does not contribute to IQ/EQ.Data: Vals Index v2Open chartDiagnostic BenchmarksTerminal-Bench Science 0.1 (Vals) ScoresAll 70 tasks from v0.1.0; fixed Terminus 2 harness, pass@1, eight-hour budget. Infrastructure retries permitted. Not interchangeable with native-agent official scores. Standalone raw result; does not contribute to IQ/EQ.Data: Terminal-Bench Science 0.1 (Vals)Open chartDiagnostic BenchmarksTax Agent Bench ScoresWeighted partial credit over the private 193-question test set, with must-pass checks and citation adjustment. Not the secondary All-Pass metric. Standalone raw result; does not contribute to IQ/EQ.Data: Tax Agent BenchOpen chartDiagnostic BenchmarksOSWorld 2.0 ScoresOfficial OSWorld 2.0 leaderboard only: 108 long-horizon desktop tasks, 500-step budget, full task set, binary completion accuracy, best reasoning/tool configuration per model. Partial-credit scores and provider-reported launch figures are excluded. Standalone raw result; does not contribute to IQ/EQ.Data: OSWorld 2.0 official leaderboardOpen chartDiagnostic BenchmarksArXivMath August 2026 ScoresAugust 2026 release under the September 15 grading revision. Agentic offline harness. Models released after the problems carry source contamination warnings; interpret as diagnostic evidence. Standalone raw result; does not contribute to IQ/EQ.Data: ArXivMath August 2026Open chartDiagnostic BenchmarksBrokenArXiv August 2026 ScoresAugust 2026 release under the September 15 grading revision. Agentic offline harness. Models released after the problems carry source contamination warnings; interpret as diagnostic evidence. Standalone raw result; does not contribute to IQ/EQ.Data: BrokenArXiv August 2026Open chart