SWE-rebench
A bird's-eye view of the benchmark: what it measures and every AI IQ chart built on it.
| # | Model | Effort | Score | Cost per problem | Completion time (est.) | Weights |
|---|
About SWE-rebench
What SWE-rebench measures
SWE-rebench has source-backed multi-effort configurations. The Cost Efficiency chart and the model table's expandable rows show those configurations; score bars remain one canonical row per model.
Reading the cost view
Each line uses the shared model-configuration dataset to compare source-matched reasoning-effort levels on SWE-rebench. Every point shows the score and cost per problem for that exact configuration. Canonical score bars and bell curves remain one point per model.
Reading the completion-time view
Each point is a public model's SWE-rebench score against its estimated completion time (time to last token) for the input and output lengths set by the sliders. Timing comes from the site's per-model speed measurements, not from the benchmark run, so every point is an estimate and labels carry a ~ prefix. The newest generation with a published result is shown by default. Up and to the left is better.
Reading the score bars
Bars rank models by the source-backed benchmark value used for this chart. Longer bars indicate higher published scores.
Reading the model table
Each row is one canonical model: the score shown is the same best published configuration the score bars use. Models with several published reasoning-effort levels expand to one sub-row per level, with the score and cost per problem the source reports for that level. Completion time is the site's estimated time to last token for a 10K-token input and 1K-token output (marked ~), not a benchmark run time. The top 20 rows are shown by default; click a column heading to sort, and missing values always sort last.