Machine Learning Benchmarks
Machine learning engineering benchmarks and AI model leaderboards for post-training, kernel optimization, and software optimization work
Machine Learning Engineering: Scores and Sources
Explore 5 benchmarks and their source-backed results below. Domain IQ uses the same shared scorer as the interactive charts and public API and remains outside Composite IQ. Coverage shows how many domain benchmarks contribute to each estimate; incomplete coverage and different evaluation harnesses limit direct comparisons. Read the scoring methodology.
Highest domain scores among tracked public models
| Model | Domain IQ | Coverage |
|---|---|---|
| GPT 6 Astra | 151 | 1/5 |
| Fable 5 | 147 | 4/5 |
| Opus 4.8 | 145 | 5/5 |
| Opus 5 | 145 | 1/5 |
| GPT 5.6 Terra | 144 | 1/5 |
| Glm 5.2 | 143 | 4/5 |
| GPT 5.3 Codex | 143 | 1/5 |
| Kimi K3 | 143 | 1/5 |
| Opus 4.6 | 142 | 2/5 |
| GPT 5.5 | 141 | 5/5 |
| Opus 4.7 | 141 | 3/5 |
| GPT 5.6 Sol | 140 | 4/5 |
| GPT 5.4 | 136 | 3/5 |
| GPT 5.2 | 135 | 2/5 |
| Sonnet 4.6 | 135 | 1/5 |
WeirdML — 123 models
Average max accuracy across 17 unusual machine-learning tasks. Higher scores are better. Source: WeirdML v2.
| Model or system | Evaluation harness | Source-backed score |
|---|---|---|
| GPT 6 Astra | OpenAI responses | 92.87% |
| Fable 5 | OpenRouter | 91.94% |
| GPT 5.6 Sol | OpenRouter | 88.76% |
| Fable 5 | Anthropic messages | 87.85% |
| GPT 5.5 | OpenRouter | 84.91% |
| GPT 5.5 | OpenAI responses | 83.9% |
| Opus 4.8 | OpenRouter | 82.89% |
| GPT 5.6 Terra | OpenRouter | 78.27% |
| Opus 4.6 | Anthropic messages | 77.95% |
| GPT 5.3 Codex | OpenRouter | 77.9% |
| GPT 5.4 | OpenAI responses | 77.7% |
| Opus 4.7 | Anthropic messages | 76.44% |
| Opus 4.7 Nonreasoning | Anthropic messages | 76.4% |
| Opus 4.8 | OpenRouter | 76.04% |
| Opus 4.7 | OpenRouter | 75.45% |
| GPT 5.2 | OpenAI responses | 72.19% |
| Gemini 3.1 Pro | OpenRouter | 72.07% |
| Opus 4.8 Nonreasoning | OpenRouter | 70.45% |
| Glm 5.2 | OpenRouter | 70.12% |
| Gemini 3 Pro | OpenRouter | 69.93% |
| Sonnet 5 | OpenRouter | 68.78% |
| Glm 5.2 | OpenRouter | 67.31% |
| GPT 5.5:no_thinking | OpenAI responses | 67.15% |
| Sonnet 4.6 | OpenRouter | 66.07% |
| Opus 4.6 Nonreasoning | OpenRouter | 65.87% |
| Opus 4.5 | Anthropic messages | 63.74% |
| GPT 5.2:medium | OpenAI responses | 63.44% |
| Gemini 3.5 Flash | OpenRouter | 62.64% |
| Gemini 3 Flash | OpenRouter | 61.6% |
| GPT 5.6 Luna | OpenRouter | 60.86% |
| GPT 5.1 | OpenRouter | 60.77% |
| GPT 5 | OpenAI responses | 60.7% |
| GPT 5.2 Pro | OpenAI responses | 60.39% |
| GPT 5.4 Mini | OpenAI responses | 60.3% |
| O3 Pro | OpenAI responses | 58.21% |
| GPT 5.4 | OpenRouter | 57.44% |
| Glm 5.1 | OpenRouter | 57.1% |
| Kimi K2.6 | OpenRouter | 55.86% |
| GPT 5.2 Codex | OpenRouter | 54.53% |
| Kimi K2.7 Code | OpenRouter | 54.12% |
| Gemini 2.5 Pro | OpenRouter | 54.03% |
| GPT 5 Mini | OpenAI responses | 52.67% |
| O4 Mini | OpenRouter | 52.56% |
| O3 | OpenAI responses | 52.42% |
| Gemma 4 31b | Ollama | 52.26% |
| Grok 4.20 | OpenRouter | 52.26% |
| Gemini 3.1 Flash Lite | OpenRouter | 52.19% |
| Grok 4.3 | OpenRouter | 49.89% |
| GPT 5.2:none | OpenAI responses | 49.6% |
| GPT 5.4 Nano | OpenAI responses | 49.23% |
| Deepseek V4 Pro | fireworks | 48.9% |
| Glm 5 | OpenRouter | 48.17% |
| GPT Oss 120b | Ollama | 48.17% |
| Sonnet 4.5 | OpenRouter | 47.71% |
| O1 | OpenRouter | 47.56% |
| Deepseek V3.2 | OpenRouter | 46.73% |
| Sonnet 4.5 | OpenRouter | 46.7% |
| Deepseek V4 Pro | fireworks | 46.53% |
| Claude 4 Sonnet 20250522:16k | OpenRouter | 46.11% |
| O1 | OpenRouter | 46.08% |
| Opus 4.1 | OpenRouter | 45.86% |
| Grok 4 | OpenRouter | 45.73% |
| Deepseek V4 Flash | OpenRouter | 45.63% |
| Kimi K2.5 | OpenRouter | 45.6% |
| Haiku 4.5 | OpenRouter | 45.4% |
| Haiku 4.5 | OpenRouter | 44.15% |
| Claude 4 Sonnet 20250522 | OpenRouter | 43.88% |
| Deepseek V4 Flash | OpenRouter | 43.76% |
| Claude 4 Opus 20250522:16k | OpenRouter | 43.72% |
| Mistral Medium 3.5 | Ollama | 43.71% |
| O3 Mini High 2025 01 31 | OpenRouter | 43.7% |
| Nemotron 3 Ultra | OpenRouter | 43.45% |
| Mercury 2 | OpenRouter | 43.2% |
| Grok 4.1 Fast | OpenRouter | 42.86% |
| Kimi K2 | OpenRouter | 42.79% |
| Grok 3 Mini | OpenRouter | 42.58% |
| Glm 5.2 | OpenRouter | 42.1% |
| Gemini 2.5 Flash 09 2025:16k | OpenRouter | 41.91% |
| GPT Oss 120b | OpenRouter | 41.88% |
| Deepseek R1 | OpenRouter | 41.63% |
| Qwen3 Coder | OpenRouter | 41.17% |
| Qwen3 235b A22b Thinking 2507 | OpenRouter | 41.04% |
| Gemini 2.5 Flash:16k | OpenRouter | 40.95% |
| GPT Oss 20b | Ollama | 40.93% |
| Glm 4.5 | OpenRouter | 40.58% |
| Claude 3.5 Sonnet | OpenRouter | 39.97% |
| GPT 5 Chat | OpenRouter | 39.77% |
| Qwen3.5 27b | Ollama | 39.53% |
| Deepseek V3.2 Exp | OpenRouter | 39.48% |
| GPT 4.5 | OpenRouter | 39.37% |
| Kimi K2 | OpenRouter | 39.36% |
| GPT 4.1 2025 04 14 | OpenRouter | 39.04% |
| Qwen3 235b A22b 07 25 | OpenRouter | 38.7% |
| Deepseek Chat V3.1:thinking | OpenRouter | 38.36% |
| GPT 5 Nano | OpenAI responses | 38.06% |
| Nemotron 3 Super:120b A12b Q4_K_M | Ollama | 38.01% |
| GPT 5.4 Nano:none | OpenAI responses | 37.96% |
| GPT 5.4 Mini:none | OpenAI responses | 37.9% |
| GPT 4.1 Mini 2025 04 14 | OpenRouter | 37.61% |
| Deepseek Chat V3.1 | OpenRouter | 37.5% |
| Qwen3 235b A22b | OpenRouter | 37.28% |
| Grok 3 | OpenRouter | 37.24% |
| Minimax M2.7 | OpenRouter | 36.95% |
| GPT Oss 20b | OpenRouter | 36.83% |
| Kimi K2 | OpenRouter | 36.68% |
| Deepseek R1 | OpenRouter | 36.49% |
| O1 Mini | OpenRouter | 36.32% |
| Deepseek Chat V3 0324 | OpenRouter | 36.08% |
| Gemini 2.5 Flash Lite 06 17:16k | OpenRouter | 35.22% |
| Gemma4:26b A4b It Q4_K_M | Ollama | 35.17% |
| Grok Code Fast 1 | OpenRouter | 35.06% |
| Qwen3.6 35b A3b | Ollama | 34.49% |
| Qwen3 Coder Next | OpenRouter | 34.4% |
| Mistral Medium 3.1 | OpenRouter | 33.13% |
| Claude 3.5 Sonnet 20240620 | OpenRouter | 30.97% |
| Claude 3.5 Haiku | OpenRouter | 30.73% |
| Qwen3 30b A3b | OpenRouter | 29.75% |
| GPT 5 Nano | OpenAI responses | 25.89% |
| Gemini 2.0 Flash 001 | OpenRouter | 25.77% |
| GPT 4o | OpenRouter | 25.12% |
| Gemini Flash 1.5 | OpenRouter | 24.87% |
| Llama 4 Maverick | OpenRouter | 24.47% |
| Grok 2 | OpenRouter | 22.24% |
| Gemini Pro 1.5 | OpenRouter | 22.2% |
| Llama 3.1 405b Instruct | OpenRouter | 21.38% |
| Claude 3 Opus | Anthropic messages | 19.22% |
| GPT 4.1 Nano 2025 04 14 | OpenRouter | 18.98% |
| GPT 4 Turbo | OpenRouter | 18.01% |
| Qwen 2.5 72b Instruct | OpenRouter | 15.97% |
| Llama 3.3 70b Instruct | OpenRouter | 14.44% |
| Deepseek R1 0528 Qwen3 8b | OpenRouter | 12.99% |
| GPT 4 | OpenRouter | 12.36% |
| GPT 4o Mini | OpenRouter | 11.76% |
| Qwen2:72b Instruct Q8_0 | Ollama | 11.3% |
| Claude 3 Sonnet | OpenRouter | 10.16% |
| Claude 3 Haiku | OpenRouter | 9.84% |
| Llama 3.1 70b Instruct | OpenRouter | 8.97% |
| Claude 2.1 | OpenRouter | 7.06% |
| GPT 3.5 Turbo | OpenRouter | 3.48% |
| Mixtral 8x22b Instruct | OpenRouter | 3.17% |
| Llama 3.1 8b Instruct | OpenRouter | 1.73% |
GSO-Bench — 20 models
GSO Opt@1 scores for agentic software optimization against expert commits. Higher scores are better. Source: GSO leaderboard.
| Model or system | Evaluation harness | Source-backed score |
|---|---|---|
| Opus 4.8 | OpenHands | 47.06% |
| Opus 4.7 | OpenHands | 44.12% |
| Opus 4.6 | OpenHands | 41.18% |
| GPT 5.5 | OpenHands | 40.2% |
| Sonnet 5 | OpenHands | 37.25% |
| GPT 5.4 | OpenHands | 31.37% |
| GPT 5.2 | OpenHands | 27.45% |
| Opus 4.5 | OpenHands | 26.47% |
| Gemini 3.1 Pro | OpenHands | 22.55% |
| Gemini 3 Pro | OpenHands | 18.63% |
| Sonnet 4.5 | OpenHands | 14.71% |
| GPT 5.1 | OpenHands | 13.73% |
| Gemini 3 Flash | OpenHands | 9.8% |
| O3 | OpenHands | 8.82% |
| GPT 5 | OpenHands | 6.86% |
| Opus 4 | OpenHands | 6.86% |
| Kimi K2 | OpenHands | 4.9% |
| Gemini 2.5 Pro | OpenHands | 3.92% |
| O4 Mini | OpenHands | 3.6% |
| GPT 4o | OpenHands | 0% |
KernelBench Hard — 13 models
Mean scored peak-fraction across KernelBench Hard problems. Higher scores are better. Source: kernelbench.com Hard.
| Model or system | Evaluation harness | Source-backed score |
|---|---|---|
| GPT 5.6 Sol | codex, 4/4 scored | 30.02% |
| Glm 5.2 | zai-claude, 6/6 scored | 26.14% |
| Opus 4.8 | claude, 6/6 scored | 24.44% |
| Fable 5 | claude, 5/5 scored | 23.41% |
| Sonnet 5 | claude, 6/6 scored | 22.79% |
| Minimax M3 | minimax-claude, 5/6 scored | 22.44% |
| GPT 5.5 | codex, 6/6 scored | 21.74% |
| Grok 4.5 | grok, 6/6 scored | 21.42% |
| Deepseek V4 Pro | deepseek-claude, 5/6 scored | 19.08% |
| Kimi K2.7 Code | kimi-claude, 5/6 scored | 17.22% |
| Longcat 2.0 | longcat-claude, 5/6 scored | 16.41% |
| Composer 2.5 Fast | cursor, 6/6 scored | 15.39% |
| Gemini 3.5 Flash | gemini, 6/6 scored | 12.23% |
KernelBench Mega — 12 models
Best decode speedup over optimized PyTorch on KernelBench Mega. Higher scores are better. Source: kernelbench.com Mega.
| Model or system | Evaluation harness | Source-backed score |
|---|---|---|
| Opus 4.8 | claude / B200 | 19.351× |
| Fable 5 | claude / RTX PRO 6000 | 18.715× |
| Glm 5.2 | zai-claude / RTX PRO 6000 | 11.142× |
| GPT 5.5 | codex / B200 | 9.37× |
| Minimax M3 | minimax-claude / B200 | 4.151× |
| Sonnet 5 | claude / RTX PRO 6000 | 4.03× |
| Gemini 3.5 Flash | gemini / H100 | 2.737× |
| GPT 5.6 Sol | codex / RTX PRO 6000 | 2.637× |
| Kimi K2.7 Code | kimi-claude / RTX PRO 6000 | 2.586× |
| Composer 2.5 Fast | cursor / RTX PRO 6000 | 2.482× |
| Deepseek V4 Pro | deepseek-claude / RTX PRO 6000 | 2.106× |
| Grok 4.5 | grok / RTX PRO 6000 | 0.816× |
PostTrainBench v1.1 — 11 models
Recomputed agent scores under v1.1 integrity checks for post-training small language models with a 10-hour H100 budget. Higher scores are better. Source: PostTrainBench v1.1 leaderboard.
| Model or system | Evaluation harness | Source-backed score |
|---|---|---|
| Fable 5 | Claude Code · Max | 41.79% |
| GPT 5.6 Sol | Codex CLI · Max | 36.23% |
| Opus 5 | Claude Code | 34.06% |
| Opus 4.8 | Claude Code · High | 33.84% |
| Opus 4.8 | Claude Code · Max | 32.9% |
| Kimi K3 | Claude Code · 1M context | 31.96% |
| Glm 5.2 | Claude Code · Max | 31.7% |
| Opus 4.7 | Claude Code · xHigh | 28.56% |
| GPT 5.5 | Codex CLI · xHigh | 27.23% |
| Grok 4.5 | Cursor CLI · High | 23.45% |
| Gemini 3.1 Pro | OpenCode | 21.99% |
| GPT 5.4 | Codex CLI · High | 19% |