BrokenArXiv August 2026 Scores
August 2026 release under the September 15 grading revision. Agentic offline harness. Models released after the problems carry source contamination warnings; interpret as diagnostic evidence. Standalone raw result; does not contribute to IQ/EQ.
BrokenArXiv August 2026 Scores
More / ExperimentalBrokenArXiv August 2026 vs Effective CostX = effective cost (log). Y = BrokenArXiv August 2026. August 2026 release under the September 15 grading revision. Agentic offline harness. Models released after the problems carry source contamination warnings; interpret as diagnostic evidence.Data: Artificial Analysis, ARC Prize, Vals Index v2 Cost per Test +2 moreOpen chartDiagnostic BenchmarksMultiChallenge ScoresDiagnostic multi-turn instruction retention, inference memory, editing, and self-coherence. Does not contribute to IQ. Color = provider.Data: Scale Labs MultiChallengeOpen chartDiagnostic BenchmarksTerminal-Bench Science 0.1 (Vals) ScoresAll 70 tasks from v0.1.0; fixed Terminus 2 harness, pass@1, eight-hour budget. Infrastructure retries permitted. Not interchangeable with native-agent official scores. Standalone raw result; does not contribute to IQ/EQ.Data: Terminal-Bench Science 0.1 (Vals)Open chartDiagnostic BenchmarksTax Agent Bench ScoresWeighted partial credit over the private 193-question test set, with must-pass checks and citation adjustment. Not the secondary All-Pass metric. Standalone raw result; does not contribute to IQ/EQ.Data: Tax Agent BenchOpen chartDiagnostic BenchmarksVals Index v2 ScoresVersion 2 cohort only. Accuracy, cost and latency must come from the same source configuration. Highest published effort; Max and default retained separately in source snapshot. Standalone raw result; does not contribute to IQ/EQ.Data: Vals Index v2Open chartDiagnostic BenchmarksArXivMath August 2026 ScoresAugust 2026 release under the September 15 grading revision. Agentic offline harness. Models released after the problems carry source contamination warnings; interpret as diagnostic evidence. Standalone raw result; does not contribute to IQ/EQ.Data: ArXivMath August 2026Open chartDiagnostic BenchmarksOSWorld 2.0 ScoresOfficial OSWorld 2.0 leaderboard only: 108 long-horizon desktop tasks, 500-step budget, full task set, binary completion accuracy, best reasoning/tool configuration per model. Partial-credit scores and provider-reported launch figures are excluded. Standalone raw result; does not contribute to IQ/EQ.Data: OSWorld 2.0 official leaderboardOpen chart