SWE-Bench Pro Benchmark Scores
Each model's SWE-Bench Pro percentage score. Color = provider.
SWE-Bench Pro Benchmark Scores
More / ExperimentalSWE-Bench Pro vs Effective CostX = effective cost (log). Y = SWE-Bench Pro %. Color = provider.Data: Artificial Analysis, ARC Prize, Vals Index v2 Cost per Test +3 moreOpen chartCoreIQ Bell CurveEach model's estimated IQ, placed on a standard normal IQ distribution.Data: ARC Prize, Epoch AI, Epoch AI +26 moreOpen chartIQ BenchmarksARC-AGI-3 Benchmark ScoresEach model's ARC-AGI-3 score. Color = provider.Data: ARC PrizeOpen chartIQ BenchmarksARC-AGI-2 ScoresEach model's ARC-AGI-2 score. Color = provider.Data: ARC PrizeOpen chartIQ BenchmarksARC-AGI-1 Benchmark ScoresEach model's ARC-AGI-1 score. Color = provider.Data: ARC PrizeOpen chartIQ BenchmarksFrontierMath Tier 4 Benchmark ScoresEach model's FrontierMath Tier 4 score. Color = provider.Data: Epoch AI FrontierMathOpen chartIQ BenchmarksFrontierMath Tier 1-3 Benchmark ScoresEach model's FrontierMath Tier 1-3 score. Color = provider.Data: Epoch AI FrontierMathOpen chart