SWE-Bench Verified vs Effective Cost
X = effective cost (log). Y = SWE-Bench Verified %. Color = provider.
SWE-Bench Verified vs Effective Cost
SWE-Bench Verified vs Effective Cost
X = effective cost (log). Y = SWE-Bench Verified %. Color = provider.
Controls:
How to read this chart
Each point is a public model. The chart compares SWE-Bench Verified % against Effective Cost (per 1M I/O Tokens), with color showing the model provider.
Data sources
More / ExperimentalTerminal-Bench 2.0 Benchmark ScoresLegacy Terminal-Bench 2.0 scores retained for historical comparison. Color = provider.Data: Terminal-BenchOpen chartMore / ExperimentalARC-AGI-2 Cost EfficiencyX = ARC Prize reported cost/task (log). Y = ARC-AGI-2 %. Each line connects one model's published reasoning-effort levels, so the score-vs-cost tradeoff is visible per model. Shows each model line's current and previous model. Color = provider.Data: ARC PrizeOpen chartMore / ExperimentalBenchmark Correlation HeatmapA pairwise matrix of source-backed benchmark score correlations across public models. Color shows Pearson r; each cell uses models with both benchmark scores.Data: ARC Prize, Epoch AI FrontierMath, Vals.ai +28 moreOpen chartMore / ExperimentalFlagship Lab Benchmark Correlation HeatmapA pairwise benchmark correlation matrix using only the latest flagship model from each major US and Chinese lab.Data: ARC Prize, Epoch AI FrontierMath, Vals.ai +28 moreOpen chartMore / ExperimentalARC-AGI-3 vs CostX = ARC Prize reported Cost (V3) (log). Y = ARC-AGI-3 %. Color = provider.Data: ARC PrizeOpen chartMore / ExperimentalARC-AGI-2 vs Cost/TaskX = ARC Prize reported cost/task (log). Y = ARC-AGI-2 %. Color = provider.Data: ARC PrizeOpen chart