AI IQ
Consultation Subscribe

SWE-Bench Verified Benchmark Scores

Each model's SWE-Bench Verified score. Color = provider.

SWE-Bench Verified Benchmark Scores
SWE-Bench Verified Benchmark Scores
Each model's SWE-Bench Verified score. Color = provider.

How to read this chart

Bars rank models by the source-backed benchmark value used for this chart. Longer bars indicate higher published scores.

Data sources
Vals.aiLLM Stats
SWE-Bench Verified vs Effective Cost
SWE-Bench Verified vs Effective Cost
X = effective cost (log). Y = SWE-Bench Verified %. Color = provider.
Controls:
SWE-Bench1:1Cost

How to read this chart

Each point is a public model. The chart compares SWE-Bench Verified % against Effective Cost (per 1M I/O Tokens), with color showing the model provider.

Data sources
Artificial AnalysisARC PrizeVals.aiLLM Stats
CoreAI Models on the IQ Bell CurveEach model's estimated IQ plotted on a standard normal IQ distributionData: ARC Prize, Epoch AI FrontierMath, Vals.ai +24 moreOpen chartIQ BenchmarksARC-AGI-3 Benchmark ScoresEach model's ARC-AGI-3 score. Color = provider.Data: ARC PrizeOpen chartIQ BenchmarksARC-AGI-2 ScoresEach model's ARC-AGI-2 score. Color = provider.Data: ARC PrizeOpen chartIQ BenchmarksARC-AGI-1 Benchmark ScoresEach model's ARC-AGI-1 score. Color = provider.Data: ARC PrizeOpen chartIQ BenchmarksFrontierMath Tier 4 Benchmark ScoresEach model's FrontierMath Tier 4 score. Color = provider.Data: Epoch AI FrontierMathOpen chartIQ BenchmarksFrontierMath Tier 1-3 Benchmark ScoresEach model's FrontierMath Tier 1-3 score. Color = provider.Data: Epoch AI FrontierMathOpen chart