ProgramBench vs Effective Cost
X = effective cost (log). Y = ProgramBench Almost Resolved. Color = provider.
ProgramBench vs Effective Cost
ProgramBench vs Effective Cost
X = effective cost (log). Y = ProgramBench Almost Resolved. Color = provider.
Controls:
How to read this chart
Each point is a public model. The chart compares Almost Resolved against Effective Cost (per 1M I/O Tokens), with color showing the model provider.
More / ExperimentalTerminal-Bench 2.0 Benchmark ScoresLegacy Terminal-Bench 2.0 scores retained for historical comparison. Color = provider.Data: Terminal-BenchOpen chartMore / ExperimentalARC-AGI-3 Cost EfficiencyX = ARC Prize reported Cost (V3) (log). Y = ARC-AGI-3 %. Each line connects one model's published reasoning-effort levels, so the score-vs-cost tradeoff is visible per model. Defaults to the current model generation. Color = provider.Data: ARC PrizeOpen chartMore / ExperimentalARC-AGI-2 Cost EfficiencyX = ARC Prize reported cost/task (log). Y = ARC-AGI-2 %. Each line connects one model's published reasoning-effort levels, so the score-vs-cost tradeoff is visible per model. Defaults to the current model generation. Color = provider.Data: ARC PrizeOpen chartMore / ExperimentalARC-AGI-1 Cost EfficiencyX = ARC Prize reported cost/task (log). Y = ARC-AGI-1 %. Each line connects one model's published reasoning-effort levels, so the score-vs-cost tradeoff is visible per model. Defaults to the current model generation. Color = provider.Data: ARC PrizeOpen chartMore / ExperimentalChess Puzzles Cost EfficiencyX = estimated inference cost for Epoch AI's 100-puzzle run (log), derived from source-reported token usage and canonical model pricing. Y = Chess Puzzles exact-match accuracy. Single configurations appear as points; Claude Opus 4.6's published thinking-budget runs form a connected effort curve. Color = provider.Data: Epoch AI Chess PuzzlesOpen chartMore / ExperimentalMMLU-Pro Cost EfficiencyMMLU-Pro score versus runtime effective cost. Explicit reasoning and nonreasoning variants are connected; other published configurations remain standalone points. Color = provider.Data: Artificial Analysis, ARC Prize, Vals.ai +2 moreOpen chart