MultiChallenge Scores
Diagnostic multi-turn instruction retention, inference memory, editing, and self-coherence. Does not contribute to IQ. Color = provider.
MultiChallenge Scores
Diagnostic multi-turn instruction retention, inference memory, editing, and self-coherence. Does not contribute to IQ. Color = provider.
MultiChallenge Cost Efficiency
MultiChallenge scores plotted against estimated effective cost per 1M I/O tokens (a runtime estimate, not a published task cost). Explicit reasoning/nonreasoning siblings are connected where both are public. Color = provider.
This chart is part of the MultiChallenge benchmark page, which adds the model table, sources and how to read each view.
More / ExperimentalMultiChallenge vs Effective CostX = effective cost (log). Y = MultiChallenge score. Color = provider.Data: Artificial Analysis, ARC Prize, Vals Index v2 Cost per Test +2 moreOpen chartMore / ExperimentalMultiChallenge Cost EfficiencyMultiChallenge scores plotted against estimated effective cost per 1M I/O tokens (a runtime estimate, not a published task cost). Explicit reasoning/nonreasoning siblings are connected where both are public. Color = provider.Data: Artificial Analysis, ARC Prize, Vals Index v2 Cost per Test +2 moreOpen chartDiagnostic BenchmarksTerminal-Bench Science 0.1 (Vals) ScoresAll 70 tasks from v0.1.0; fixed Terminus 2 harness, pass@1, eight-hour budget. Infrastructure retries permitted. Not interchangeable with native-agent official scores. Standalone raw result; does not contribute to IQ/EQ.Data: Terminal-Bench Science 0.1 (Vals)Open chartDiagnostic BenchmarksTax Agent Bench ScoresWeighted partial credit over the private 193-question test set, with must-pass checks and citation adjustment. Not the secondary All-Pass metric. Standalone raw result; does not contribute to IQ/EQ.Data: Tax Agent BenchOpen chartDiagnostic BenchmarksVals Index v2 ScoresVersion 2 cohort only. Accuracy, cost and latency must come from the same source configuration. Highest published effort; Max and default retained separately in source snapshot. Standalone raw result; does not contribute to IQ/EQ.Data: Vals Index v2Open chartDiagnostic BenchmarksArXivMath August 2026 ScoresAugust 2026 release under the September 15 grading revision. Agentic offline harness. Models released after the problems carry source contamination warnings; interpret as diagnostic evidence. Standalone raw result; does not contribute to IQ/EQ.Data: ArXivMath August 2026Open chart