Mystery Game Puzzles Cost Efficiency
Source-backed Mystery Game Puzzles scores plotted against estimated effective cost per 1M I/O tokens (a runtime estimate, not a published task cost). Explicit reasoning/nonreasoning siblings are connected where both are public. Color = provider.
Mystery Game Puzzles Cost Efficiency
Source-backed Mystery Game Puzzles scores plotted against estimated effective cost per 1M I/O tokens (a runtime estimate, not a published task cost). Explicit reasoning/nonreasoning siblings are connected where both are public. Color = provider.
This chart is part of the Mystery Game Puzzles benchmark page, which adds the model table, sources and how to read each view.
More / ExperimentalMystery Game Puzzles vs Effective CostX = effective cost (log). Y = Mystery Game Puzzles. Epoch’s 100 text-state puzzles in an undisclosed game. Best-next-move accuracy in a minimal agent scaffold; invalid or missing moves receive no credit. Prompts and transcripts are private.Data: Artificial Analysis, ARC Prize, Vals Index v2 Cost per Test +2 moreOpen chartIQ BenchmarksMystery Game Puzzles ScoresEpoch’s 100 text-state puzzles in an undisclosed game. Best-next-move accuracy in a minimal agent scaffold; invalid or missing moves receive no credit. Prompts and transcripts are private. Contributes to Abstract Reasoning using a provisional calibrated scale.Data: Epoch AIOpen chartMore / ExperimentalSWE-Bench Verified Benchmark ScoresHistorical data. Vals archived cohort; retired from default Software Engineering domain. Each model's SWE-Bench Verified score. Color = provider.Data: Vals.ai, LLM StatsOpen chartMore / ExperimentalTerminal-Bench 2.0 Benchmark ScoresHistorical data. Superseded by 2.1 and 4.0. Legacy Terminal-Bench 2.0 scores retained for historical comparison. Color = provider.Data: Terminal-BenchOpen chartMore / ExperimentalTerminal-Bench 2.1 Benchmark ScoresHistorical data. Superseded in Programmatic Reasoning by Terminal-Bench 4.0. Each model's Terminal-Bench 2.1 pass@1 score, using Artificial Analysis as canonical and Vals.ai as fallback. Color = provider.Data: Artificial Analysis Terminal-Bench v2.1, Vals.ai Terminal-Bench 2.1, Artificial Analysis Intelligence Index v4.3.1Open chartMore / ExperimentalTerminal-Bench Hard Benchmark ScoresHistorical data. Historical hard subset; not a Terminal-Bench 4.0 result. Each model's Terminal-Bench Hard score. Color = provider.Data: Artificial AnalysisOpen chart