Methodology

September 30, 2026 scoring update. Composite IQ now averages seven equally weighted dimensions and uses 39 benchmark inputs. Chess Puzzles and Mystery Game Puzzles join Abstract Reasoning. Knowledge Work adds AA-Briefcase v1.1, GDPval-AA v2.1, AutomationBench-AA, GDP.pdf, AA-AnalystAgent, and EnterpriseOps-Gym-AA. MirrorCode joins Programmatic Reasoning and the separate Software Engineering domain. CL-bench and CL-bench Life remain outside scoring. Six dimensions use Bayesian estimates; Abstract Reasoning retains its direct-evidence estimator with ARC-based fallbacks.

Benchmark mappings remain provisional and site-relative, rather than a new human IQ study. Models lacking successor results receive no raw backfill from older versions. Posterior ranges describe uncertainty under the model's assumptions and are not empirically calibrated confidence intervals. Historical results remain distinct; benchmark-specific limitations are described below.

AI IQ assigns each model an estimated IQ score across 7 equally weighted capability dimensions, each measured by source-backed benchmarks where coverage exists. Every benchmark uses a calibrated ladder: what score would correspond to IQ 70, 85, 100, 115, 130, 145, and 160 on that task? One source-backed benchmark is sufficient to estimate a dimension; additional coverage increases confidence. Mathematical, Academic, Programmatic, Computer Use, Reliability, and Knowledge Work IQ use Bayesian dimension estimates from direct results and uncertain predecessor priors. Abstract Reasoning keeps its direct-evidence method with ARC-based fallbacks. Every derived composite averages all seven dimensions.

This page documents the full scoring system: how source data is captured, how the seven dimensions are defined, how raw scores map to IQ, and how missing values are imputed without changing source-backed benchmark data. Every dimension value remains an estimate even at complete benchmark coverage. Emotional Reasoning (EQ) is a separate applied-capability domain and is excluded from Composite IQ.

The 7-Dimension Framework

AI IQ organizes Composite IQ evaluation into seven equally weighted dimensions. The 39 active benchmark inputs inform seven dimensions. Abstract Reasoning averages direct ARC and puzzle projections; the other six dimensions estimate a latent ability from the observed benchmark matrix:

• Frontier benchmarks are hard, low-gameability tests where a large score gain at the frontier usually represents a large capability gain.
• Saturating benchmarks are easier, more gameable, or already clustered near the top. Their ladders still reach IQ 160 where appropriate, but the score expectations rise quickly near the high end so a merely high score does not overstate ability.

The composite IQ requires at least 2 of 7 scored dimensions to have direct measurements or compatible ancestral measurement evidence. Peer estimates and assumed zeros do not establish eligibility. The displayed coverage count includes direct measurements only; all seven final dimension estimates enter the composite.

Formulas

Each benchmark raw score \(s\) is converted to an IQ value by comparing it with that benchmark's expected score at fixed IQ levels. Let \(S(q_i)\) be the expected raw score for IQ level \(q_i\), where \(q_i \in \{70,85,100,115,130,145,160\}\). For a score between two adjacent expected scores, AI IQ uses piecewise-linear interpolation:

Direct scores first pass through the fixed benchmark ladders. For six dimensions, a Bayesian model then accounts for benchmark-specific offsets and estimates model ability without manufacturing missing benchmark observations.

For each Bayesian dimension D2–D7, model ability and benchmark offsets are estimated jointly:

\(f\) is piecewise-linear interpolation through each benchmark's expected-score ladder.

The composite IQ is the mean of all seven scored dimension scores. At least 2 dimensions must contain a direct measurement or compatible ancestral measurement evidence before a model receives composite IQ. Peer estimates, momentum estimates, and assumed zeros do not qualify. The coverage count reports directly measured dimensions; inherited dimension estimates do not count as direct measurements. Any missing whole dimensions are then filled:

D1: Abstract Reasoning

Abstract Reasoning covers novel pattern solving and planning over structured states. ARC measures visual abstraction and adaptation; Chess Puzzles and Mystery Game Puzzles add best-next-move reasoning. Chess performance also depends on learned game knowledge.

Chess Puzzles and Mystery Game Puzzles

Epoch evaluates 100 positions in each benchmark. Chess is presented as text in FEN notation, not images. Mystery Game uses text states and a move-submission tool; its game identity, prompts and transcripts are withheld to reduce targeted preparation. Both score best-next-move accuracy. Furniture Assembly and Earthborne Rangers are not included.

Equal benchmark weights, separate difficulty curves. Each of the five D1 benchmarks contributes equally after conversion through its own IQ 70/85/100/115/130/145/160 expected-score ladder. The two games receive no extra group weight or reliability discount. With all five observed, each contributes 20% of D1. With partial coverage, the available direct projections are averaged; missing results are not zero and inherited cells are not counted as extra direct observations. A measured zero is still direct evidence. Direct puzzle results bypass the no-observation ARC predictor, including when ARC coverage is absent.

The puzzle ladders are selected to predict ARC-only Abstract IQ: the equal mean of each reference model’s available measured ARC-AGI-1/2/3 projections. Puzzle scores, imputed ARC values and composite IQ never enter this target. The frozen reference sets contain 46 models across eight provider groups for Chess and 35 across eight for Mystery Game.

We compare eight monotone curve candidates, including the earlier percentile transfer, linear, square-root, log, quadratic and three isotonic blends. Training weights balance provider groups. Leave-one-provider-out error is measured on the actual seven-anchor ladders used in scoring. A one-standard-error rule selects the simplest curve statistically close to the lowest provider-fold mean squared error. Both selected curves are linear; nonlinear difficulty curves are not imposed when they fail to improve prediction reliably.

Nested provider holdouts repeat curve selection using only the remaining providers before predicting the held-out group. Provider-balanced RMSE falls from 11.20 to 10.59 Abstract IQ points for Chess and from 11.50 to 9.96 for Mystery Game; MAE falls from 8.14 to 7.84 and 9.61 to 8.79 respectively. Linear is selected in every outer fold. This improves aggregate prediction, not every provider. Chess’s best raw candidate has RMSE 10.57 versus linear’s 10.59, a negligible advantage that does not justify the more complex curve.

These remain provisional site-relative fits, not measured human IQ percentiles. Validation uses the current reference snapshot, not an untouched prospective test. ARC coverage varies, game specialization can differ from ARC, and tail anchors extrapolate. Negative raw anchors are interpolation guards allowing a fitted nonzero IQ at 0% accuracy; they are not possible benchmark scores. Benchmark weights stay equal after calibration.

External-target checks comparing ARC alone with ARC plus both puzzles show mixed results: three of six separate reasoning targets improve and three worsen, with all primary provider-bootstrap improvement intervals crossing zero. These exploratory checks do not establish that the additions improve measurement of abstract reasoning. We retain both puzzles as provisional complementary inputs; Furniture Assembly and Earthborne Rangers remain excluded.

Benchmark / IQ	70	85	100	115	130	145	160
Chess Puzzles	-0.546	15.902	32.35	48.798	65.246	81.694	98.142
Mystery Game Puzzles	-18.398	2.534	23.465	44.397	65.328	86.26	107.191

Sources: Epoch Chess Puzzles and Epoch Mystery Game Puzzles.

D2: Mathematical Reasoning

Mathematical reasoning and quantitative problem-solving — the ability to work with mathematical structures, proofs, and analytical frameworks. The hard benchmarks test novel quantitative reasoning that cannot be memorized from training data. Human references differ by benchmark: FrontierMath uses specialist and expert-team math references, ProofBench uses formalization ability, and MathArena/AIME use competition-math populations rather than the general public.

D3: Academic Reasoning

Academic Reasoning measures expert knowledge, research-style analysis, and difficult problem solving across academic fields. Its scored set uses Humanity's Last Exam, GPQA Diamond, CritPt, SciCode, MMLU-Pro, and MMMU-Pro. CritPt and SciCode remain here because their bottleneck is high-level scientific reasoning, even when mathematics or code is the medium.

Finance Domain

Finance is an applied domain outside Composite IQ. It measures financial analysis, long-document interpretation, spreadsheet modeling, banking operations, multimodal mortgage-tax extraction, and tax reasoning through Finance Agent v2, CorpFin v2, the Excel Modeling Benchmark, τ³-Banking, MortgageTax, and TaxEval v2.

Finance IQ balances two scored capability groups: Financial Analysis (Finance Agent v2) and Documents and Modeling (Excel Modeling). One source-backed scored benchmark is sufficient for a provisional estimate. Missing scores use the existing conservative matched-peer policy. Tax Agent Bench and τ³-Banking remain diagnostics and do not increase the scored coverage count. Retired CorpFin, Mortgage Tax and TaxEval cohorts are excluded.

These are model-plus-system evaluations. Finance Agent and Excel Modeling depend materially on tools and agent scaffolds; CorpFin and TaxEval use LLM judging. The July 24 calibration uses conservative occupational reference points rather than fitting the current model distribution: ordinary non-specialists sit in the low anchors, careful generalists or trainees around IQ 100, capable practitioners around IQ 115–130, and near-reliable expert work around IQ 145. Because matched professional-human score distributions are not published, every Finance projection remains provisional, sparse, or high-risk.

Benchmark / IQ	70	85	100	115	130	145	160
Finance Agent v2	0	5	20	45	70	90	105
CorpFin v2	5	20	45	72	90	98	108
Excel Modeling	0	5	20	70	90	98	108
τ³-Banking (diagnostic)	0	5	12	25	50	75	95
MortgageTax	10	35	70	90	97	100	110
TaxEval v2	5	20	45	75	90	98	110

WebDev & Design Domain

WebDev & Design is an applied domain outside Composite IQ. It combines coding with product reasoning, user experience, interaction design, visual-spatial layout, and full-stack implementation.

Methodology caveat: the current set is intentionally focused on greenfield app and web-artifact quality, and its inputs are not four fully independent sub-skills. Arena.ai WebDev, DesignArena Frontend, and DesignArena Full Stack all contain overlapping product, UI, UX, prompt-following, and implementation-quality signals. Vibe Code Bench adds a more task-suite-like app-building signal, but this domain should still be read as a practical product-building cluster rather than a rigorously factor-separated measurement. Broader coverage across product architecture, user-intent inference, accessibility, maintainability, and long-horizon iteration would make the domain stronger.

Legal Domain

Legal is an applied domain outside Composite IQ. Legal IQ equally weights three capabilities after projecting each benchmark onto the AI IQ scale: broad legal reasoning, multi-source legal research, and long-horizon production of professional legal work. A model needs source-backed results on at least two of the three benchmarks; missing values are not imputed.

How to read the calibration tables: the column headings are IQ anchors, and the second row gives the raw benchmark score expected at each anchor. For example, Harvey LAB raw scores of 0%, 5%, 15%, 35%, and 70% map to IQ 100, 115, 130, 145, and 160 respectively.

Engineering Design

Engineering Design tracks five source-backed benchmarks outside Composite IQ. Engineering Design IQ is a provisional, equally weighted mean of available benchmark scores after each is separately projected onto the AI IQ scale. One observed benchmark is sufficient; tooltips show coverage out of five. We do not impute missing scores. Raw charts retain source metrics on a 0–100 display scale, which are not interchangeable.

• Parametric CAD Bench v3: mean reward across 100 creation, editing, and drawing-reconstruction tasks. We select the highest published agent configuration per tracked model; earlier benchmark versions are excluded.
• BenchCAD: version 1.0 Vision2Code, using executed voxel intersection-over-union including failed executions. We use re-graded results, prefer the published thinking configuration for reasoning models, and exclude vendor-reported results on other subsets.
• HWE-bench: the CAE release, equally averaging four simulation-track scores across 77 tasks under Terminus 2. Published rounded scores are retained; confidence intervals and run details are available at the source.
• CADGenBench: the highest validated completed submission per mapped model on one dataset revision, across 81 generation and editing tasks. Submissions use different agents and budgets; this is a comparison of published systems, not an equal-compute experiment.
• AutoCAD-Bench: completion at a score of at least 75 on the published 50-task corpus, including development tasks. Judge-assisted assessment of drawings and 3D output does not establish correct construction history or parametric editability. Exact evaluator provenance remains a limitation.

Missing scores remain absent. Agent configurations are retained with results. Effective-cost comparisons use our estimated token-cost measure, not the cost of performing these CAD tasks. Each mapping uses provider-balanced linear least squares against frozen site Composite IQ references from September 29, 2026, with equal total weight per provider within a benchmark. Reference cohorts contain 6–11 models. This is site-relative calibration, not human CAD norming or independent predictive validation. Leave-provider-out errors are especially large for AutoCAD-Bench (up to 29.8 IQ points), so the composite should be read alongside its coverage and raw results; it does not establish precise rankings across different benchmark subsets.

Healthcare Domain

Healthcare is an applied domain outside Composite IQ. Healthcare IQ equally weights seven capabilities after projecting each benchmark onto the AI IQ scale: patient safety, clinical reasoning under uncertainty, complex diagnosis, clinical workflow execution, clinician communication, clinical documentation, and medical coding. A model needs source-backed results on at least two of the seven benchmarks; missing values are not imputed. Four benchmarks are sourced from the MAST clinical leaderboard, which is currently in preview, two from Vals.ai's private-data healthcare evaluations, and one from Medical Sphere's independent re-runs; snapshots are dated and scores move with their sources.

The projections are human-equivalent task-performance references, not psychometric IQ measurements of clinicians or models. IQ 100 represents an educated nonexpert or early trainee, IQ 115 a capable trainee or junior professional, IQ 130 an independent experienced professional, IQ 145 an exceptional specialist, and IQ 160 near-best human or expert-team performance. Exact same-protocol human results take priority; same-metric transfers and cohort comparisons come next. When no compatible human result exists, the ladder is an explicit lower-confidence estimate from task prerequisites, occupational competence, and the scoring floor and ceiling. Current model standings are inspected only after these anchors are fixed and are never used as the target.

Partial coverage changes confidence, not eligibility. The site first maps every result through its human-calibrated benchmark ladder, then fits a partially pooled two-way model across the sparse model-by-benchmark matrix. Each observation is represented as the model's latent Healthcare IQ plus a benchmark-specific AI task-profile effect. Those effects are learned only from models with the required two-result overlap and are constrained to average zero across the seven benchmarks, preserving the absolute human-anchored IQ scale. This corrects the tested-task mix without changing a benchmark's human calibration or treating leaderboard ranks as independent head-to-head wins.

The model estimate uses a weak IQ 120 prior with one-half benchmark of weight and shrinks benchmark effects with one benchmark of prior weight. These settings were selected by masking models with six or seven results down to every two- and three-benchmark subset: recovery RMSE fell from 8.48 IQ points for the available-only mean and 7.09 for the earlier fixed coverage adjustment to 2.48 and 2.05 respectively. Models with 2/7 or 3/7 coverage remain eligible and can rank first; they simply carry wider approximate 95% ranges. Tooltips disclose the raw available-benchmark mean, the net task-mix and pooling adjustment, coverage, and uncertainty range.

This estimator is currently a Healthcare-only pilot. Other applied domains retain their existing scoring methods. Composite IQ D2–D7 now use the separately reviewed Bayesian method described below; they do not inherit the Healthcare pilot settings. Expansion requires prospective evidence that newly arriving benchmark results fall reasonably within the pilot's predicted ranges, sparse estimates converge without systematic bias as coverage grows, learned task-profile effects remain stable, and uncertainty ranges are not materially overconfident. Any later domain must tune and validate its own prior, pooling strength, and residual variance rather than inheriting Healthcare's settings.

Retrieval-augmented clinical products and medically fine-tuned model variants that appear on the source leaderboards are excluded: Healthcare IQ compares general-purpose foundation models under a common protocol.

D4: Programmatic Reasoning & the Software Engineering Domain

MirrorCode

MirrorCode calibration. Programmatic Reasoning also includes Epoch's ML,+Private,2L leaderboard: complete reimplementation of 15 programs in two languages, averaged over three runs, scored by fully solved programs. Its ladder is fitted against pre-addition Programmatic Reasoning. Only nine models from three providers are available; provider-held-out error ranges from 6.3 to 35.4 IQ points. This is a particularly uncertain calibration and should be revisited as coverage expands.

Benchmark	IQ 70	IQ 85	IQ 100	IQ 115	IQ 130	IQ 145	IQ 160
MirrorCode	0	0	0	0	32.161	65.825	99.489

Programmatic Reasoning measures transferable problem solving through code: algorithmic reasoning, decomposition, execution, debugging, and task completion. It uses LiveCodeBench (historical coverage), IOI V2, Terminal-Bench 4.0, ProgramBench, and SWE-rebench. ProgramBench is scored using its Almost Resolved metric: the share of tasks passing at least 95% of hidden tests.

The broader Software Engineering domain also includes SWE-bench Verified, APEX-SWE, DeepSWE v1.1, and SWE Marathon. Those benchmarks are valuable measures of production repository work but are not separately counted in Composite IQ; ProgramBench and SWE-rebench intentionally overlap the dimension and domain.

Academic Reasoning Benchmark Details

Academic Reasoning captures expert knowledge, research-style reasoning, and difficult problem analysis under uncertainty. The frontier benchmarks test whether a model can reason through questions that push the boundaries of human expertise itself, while the broader benchmarks add cross-field academic and graduate-level knowledge signals.

Programmatic benchmark detail: Terminal-Bench 4.0

Terminal-Bench 4.0 is part of Programmatic Reasoning because its tasks are typically coding-heavy and require deep reasoning about code and complex scripts. Versions 2.1 and Hard are historical raw benchmarks and do not contribute to the current composite.

D5: Computer Use

Computer Use measures practical task execution across browsers, desktops, tools, and multi-step workflows. Its scored benchmarks are BrowseComp, OSWorld-Verified, MCP Atlas, Arena.ai Agent Arena, and Agents' Last Exam.

D6: Reliability

Reliability captures factuality, instruction following, long-chain reasoning, metacognition and calibration, false-premise resistance, and source-grounded document work. Its scored set is SimpleQA Verified, AA Omniscience, BullshitBench v2, IFBench, AA Long Context Reasoning v1.1, and FACTS Grounding.

This unified dimension is a deliberate interim choice. As benchmark coverage improves, Reliability may split into two narrower dimensions—Metacognition and Executive Control. Working Memory and Visual Reasoning are also candidates for future dimensions, but are not included yet.

D7: Knowledge Work

Knowledge Work measures completion of professional tasks: document and artifact creation, office automation, financial analysis, and enterprise operations. It has six inputs: AA-Briefcase v1.1, GDPval-AA v2.1, AutomationBench-AA, GDP.pdf, AA-AnalystAgent, and EnterpriseOps-Gym-AA. AA-Briefcase uses Elo; the other inputs use their published percentage or normalized scores. AA-AnalystAgent uses pass^5 (success on all five runs), not pass@5 (at least one success). EnterpriseOps-Gym-AA uses strict success across its 1,117 tasks.

The September 29 ladders are provisional, provider-balanced fits against the site's pre-addition IQ estimates for models with at least four directly measured dimensions. They provide continuity with the existing scale, not independent human validation. All six evaluations are administered by Artificial Analysis and may share correlated errors. This dimension receives one seventh of Composite IQ; missing observations are not invented.

Knowledge Work calibration and coverage

These are the raw-score expectations at each IQ anchor. AA-Briefcase is in Elo; the remaining rows are percentage or normalized-score units. Repeated zero anchors are extrapolated floor guards, and some high anchors exceed the attainable score range. Neither endpoint is a measured human reference.

Benchmark	IQ 70	IQ 85	IQ 100	IQ 115	IQ 130	IQ 145	IQ 160
AA-Briefcase v1.1 Elo	0	0	395.036	1023.855	1652.674	2281.493	2910.311
GDPval-AA v2.1 normalized score	0	0	5.096	34.025	62.954	91.883	120.813
AutomationBench-AA	0	0	0	27.311	63.448	99.585	135.721
GDP.pdf	0	0	1.839	12.256	22.672	33.089	43.506
AA-AnalystAgent	0	0	0	20.938	50.0	79.063	108.125
EnterpriseOps-Gym-AA	0	9.191	23.515	37.839	52.164	66.488	80.812

One direct benchmark is enough to estimate D7; its coverage is reported as 1/6. With no direct results, a compatible predecessor can supply an uncertain prior; otherwise the existing whole-dimension fallback applies. Such a D7 estimate has 0/6 direct coverage and must not be interpreted as measured poor performance. The composite's direct-dimension count remains X/7. Estimates still influence the equal-weight mean, so models with sparse Knowledge Work coverage can move even without new measurements.

Conditional posterior ranges omit uncertainty in the fitted ladders, correlated evaluations, and source or harness changes. Current-snapshot masking checks found lower D7 benchmark prediction error with partial measurements, but worse whole-dimension cold-start error than predecessor carry-forward (7.41 versus 6.68 IQ points). These checks are not historical forecasts or independent validation of a human IQ scale. CL-bench and CL-bench Life remain outside D7 and Composite IQ.

Emotional Reasoning (EQ)

Emotional Reasoning (EQ) measures social, emotional, and conversational judgment as a domain, separate from Composite IQ. EQ-Bench 4 supplies multi-judge roleplay evidence, AttuneBench supplies participant-grounded evidence, and the frozen EQ-Bench 3 cohort extends coverage to older and long-tail models. All three remain limited proxies for human emotional intelligence.

Expected-Score Interpolation

Each benchmark defines expected raw scores at seven fixed IQ levels: 70, 85, 100, 115, 130, 145, and 160. For scores that fall between two adjacent expected scores, AI IQ uses piecewise-linear interpolation:

If the score is at or below the lowest expected score, the model receives the lowest supported endpoint. Repeated zero anchors use the highest tied endpoint: 100 for Terminal-Bench 4.0 and 85 for Code Migration. If it is at or above the highest expected score, it receives IQ 160 for that benchmark. There is no extrapolation beyond the defined range.

This approach makes the calibration question explicit. Instead of first choosing a curve family, each benchmark asks: what would an IQ 70, 85, 100, 115, 130, 145, or 160 model-equivalent score on this source? Each segment can then have a different slope, allowing threshold-like benchmarks such as CritPt or FrontierMath T4 to behave differently from smoother bounded tasks such as ARC or Terminal-Bench.

Benchmark Calibration & Averaging

The seven dimensions retain equal weight in Composite IQ. Inside D2–D7, directly measured benchmark projections enter a Bayesian model with benchmark offsets. Missing observations contribute no likelihood; compatible predecessors supply one uncertain dimension prior. D1 retains its separate ARC policy.

Why Use Expected Scores?

The calibration is framed as a human-readable judgment rather than as an abstract curve fit. For each benchmark, the table answers seven concrete questions: what score would a 70, 85, 100, 115, 130, 145, or 160 IQ-equivalent system be expected to achieve?

That means an extremely hard benchmark can assign a high implied IQ to a low-looking score. CritPt, for example, treats 0% as compatible with IQ 100 because a normal human would not be expected to score on it. AIME does the opposite at the high end: because frontier systems can approach saturation and the public problem set is contamination-sensitive, its calibration ladder puts IQ 145 and 160 outside the source's natural 0–100% range.

Bayesian Dimension Estimates

For D2–D7, the observed projected score is modeled as latent dimension ability plus a benchmark offset and residual variation. Shared-benchmark comparisons therefore inform relative ability using score margins. Pairwise wins are not added as extra independent observations.

A model without a compatible predecessor uses a reference prior with mean 110 and standard deviation 30. Benchmark offsets have a zero-mean prior with standard deviation 10; aggregate residual variation has standard deviation 10. One compatible primary predecessor, or a secondary fallback, supplies a zero-change prior with standard deviation 12. An unmeasured successor inherits uncertainty as well as a mean; additional lineage hops widen that uncertainty. These are provisional site-scale assumptions, not validated human IQ norms.

Direct measurements replace guesswork. No inherited benchmark values enter this likelihood. Models with no direct or valid ancestral evidence in a dimension use the conservative whole-dimension fallback below; generic priors alone cannot establish ranking eligibility. Staging models and incompatible or future predecessors are excluded.

Rankings use the posterior mean. Expanded IQ tooltips and the API expose a model-based 95% range separately. This conditional range does not include uncertainty in the calibration ladders, task correlations, source selection or hyperparameters; it is not an empirically calibrated confidence interval. Benchmark offsets are learned jointly, so newly measured models may revise other estimates.

Validation and limits. On the September 22 snapshot, held-out benchmark-cell and provider-block prediction errors improved in all five changed dimensions. When an entire dimension was hidden for a new model, predictions of individual benchmark scores were less accurate than direct predecessor carry-forward. These cold-start estimates retain zero direct coverage and wider uncertainty. The checks use current captures and frozen projection ladders, not untouched historical forecasts; nominal 95% predictive coverage was not achieved consistently.

Abstract Reasoning remains separate: direct ARC and puzzle observations are equally averaged after benchmark-specific calibration. Its zero-observation estimator and conservative legacy lineage/peer fallbacks are retained.

FrontierSWE V1 and IOI V1 retain archived charts. IOI V2 replaces V1 only through a new field and independent calibration. MultiChallenge and OSWorld 2.0 remain diagnostics outside Composite IQ. AIME and Vals LiveCodeBench retain useful historical coverage.

Composite IQ Calculation

Step 1: Score Each Dimension

D2–D7 use Bayesian posterior means supported by direct or compatible ancestral observations; no extra confidence penalty is subtracted. Abstract Reasoning with direct ARC or puzzle evidence uses the equal mean of its observed calibrated projections.

Abstract Reasoning without measured ARC or puzzle results uses an equal blend of two estimates. We average the complete legacy Abstract Reasoning estimate with a newer predecessor-plus-trend estimate, giving each 50% weight. The newer estimator uses regularized regression on measured academic, programming, and reliability performance. It requires comparable ARC-AGI-1/2 observations from at least three other model families released in the preceding six months. Commercial tier is not a regression feature; it is used only for the explicit same-day source eligibility rule below. The legacy half retains its existing peer-selection rules.

Expand Abstract Reasoning in a chart’s IQ tooltip to inspect the evidence behind an estimate: a measured predecessor, eligible family history, or other-lab comparisons without eligible family history. These estimates do not count as measured ARC coverage. For eligible blended Abstract estimates, the expanded view also shows an exploratory error range of approximately ±18 Abstract IQ points, with endpoints rounded outward. The width uses the 18th-smallest absolute error among 21 earlier masked-ARC cases (an 80% target); it covered 13 of 14 later cases. These repeatedly inspected current-snapshot cases are not an untouched historical holdout, so this is not a validated confidence interval or a guarantee of future coverage. The range does not change the estimate, does not measure the model’s performance variability, and does not quantify uncertainty in the IQ scale itself. It applies only to the estimated Abstract dimension, not to overall IQ, measured ARC scores, or legacy-only fallbacks.

When a measured same-family predecessor is available, its Abstract Reasoning score is blended with the regression estimate according to capability similarity, release age, model line and reasoning mode. Sources must be released earlier, except that an explicitly selected same-day source may be used when its provider tier is strictly lower than the target model’s tier and it has measured ARC-AGI-1 and ARC-AGI-2 results. Reasoning-mode restrictions still apply. Same-day sources do not enter generic peer or trend-history pools. A restrained adjustment estimates change since an earlier predecessor’s release; same-day borrowing receives no elapsed-time uplift. Without a suitable predecessor, the regression uses a partially pooled lab adjustment and, when enough history exists, a regularized trend. Labs without measured ARC history receive no lab adjustment.

The blend is applied after the legacy calculation finishes, keeping the other six dimension scores fixed. No raw ARC benchmark result is invented, measured coverage remains zero, and the final blend receives no additional confidence penalty. At least two other scored dimensions are still required for composite IQ. These are central estimates, not guaranteed lower bounds. The exploratory error range described above is not a validated confidence interval. When evidence is insufficient for this predictor, the legacy ARC and dimension-imputation fallbacks remain available.

Step 2: Dimension-Level Imputation

If a model has at least 2 scored dimensions but is missing some of the others, every unsupported dimension receives a fallback before the composite is averaged. A Bayesian dimension supported by ancestral observations already has a posterior estimate and does not use this generic fallback. The legacy fallback cap is matched to models with real data for that dimension and similar capability across the other dimensions:

where \(\overline{\mathrm{IQ}}_{\text{scored dims}}\) is the model's average IQ across the dimensions it does have. For the cap, we look at models released on or before the scored model that have real data on the missing dimension, compare their average IQ across the other dimensions, and use the lower-quartile missing-dimension IQ among the closest comparable models. If the comparable set is too thin, we fall back to the same-era lower quartile for that dimension. These peer estimates are computed by the retained legacy scorer; after D2–D7 posterior estimates are inserted, the fallback ceiling is checked again against the final supported profile.

Some dimensions did not exist as measured skills when older models shipped — computer-use benchmarks, for example, postdate every model released before 2025, so those models have no same-era comparables at all. We still refuse a neutral default there. Instead, the same lower-quartile convention runs over comparably capable models from any era, with two extra safeguards. First, the model's own scored average remains a hard ceiling, so a dimension using this fallback cannot exceed that average — and because only strong modern models tend to get measured on late-emerging skills, that ceiling is usually what binds. Second, when fewer than four dimensions have supported estimates, the ceiling tightens to the average that excludes the model's best measured dimension: with most of the profile unmeasured, one strong area — typically knowledge-heavy academic benchmarks that older models did comparatively well on — must not set the level for everything unmeasured. This bounds unsupported fallbacks; it does not guarantee that a later measurement will raise the estimate.

In practice, comparable models are those whose average across the non-missing dimensions is within a small IQ band of the model being scored and, outside the any-era fallback, whose release date is not later than the scored model. If fewer than three comparable models are available, we use the nearest measured models instead.

All seven scored dimensions are always used for derived IQ. Missing dimensions are estimated rather than omitted from the average. Abstract Reasoning uses the predictor described above when supported; other missing dimensions retain conservative imputation.

Dimension Imputation Waterfall

Step	When it applies	What happens	Result
1. Scored or estimated dimension	The model has direct or compatible ancestral dimension evidence.	Use the Bayesian posterior mean for D2–D7, with conditional uncertainty shown separately; use the direct-evidence method with ARC-based fallbacks for D1.	Scored or estimated dimension IQ
2. Matched lower-quartile cap	A whole dimension is still missing and the model has at least two scored dimensions.	Find models with real data for the missing dimension and similar average across the other dimensions.	\(\min(\text{model scored-dimension average}, \text{matched lower-quartile }D_k)\)
3. Nearest-neighbor lower quartile	Too few models fall within the similarity radius.	Use the nearest comparable models by other-dimension average.	\(\min(\text{model scored-dimension average}, \text{nearest-neighbor lower quartile }D_k)\)
4. Global dimension lower quartile	The comparable set is still too thin.	Use the lower-quartile observed IQ for that dimension across models with real data.	\(\min(\text{model scored-dimension average}, \text{global lower-quartile }D_k)\)
5. Any-era lower-quartile cap	No real data for that dimension exists from the model's era or earlier (the dimension's benchmarks postdate the model).	Use the lower-quartile convention over comparably capable models from any era. With fewer than four measured dimensions, the scored-average ceiling excludes the model's best measured dimension.	\(\min(\text{trimmed scored-dimension average}, \text{any-era lower quartile }D_k)\)
6. No derived IQ	No real data exists for that dimension at all.	Do not invent a neutral default.	No derived all-dimension IQ.

Step 3: Compute the Composite

where all seven scored dimensions are used once missing dimensions are imputed.

Key rules:

• Minimum 2 dimensions required. Models with fewer than 2 dimensions supported by direct or compatible ancestral measurement evidence do not receive a derived composite IQ.
• No omitted dimensions. Models with enough coverage for derived IQ always use a 7-dimension composite; missing dimensions are estimated, with conservative fallbacks where the Abstract Reasoning predictor does not apply.
• Transparent count. The display shows X/7 so readers can see how many scored dimensions had source-backed data before dimension-level imputation.
• Equal weighting. All seven dimensions contribute equally. Within D2–D7, projected observations use equal residual variance, while benchmark offsets and uncertain lineage priors affect the posterior. Benchmark selection and calibration remain separate reviewed choices.

Rank Status

Each model receives a rank status reflecting the completeness of its evaluation:

• Full — All 7 dimensions have at least one direct measurement. This describes coverage, not validated certainty.
• Partial — At least 2 dimensions have direct or compatible ancestral evidence, but fewer than 7 are directly measured. Composite IQ is derived with incomplete direct coverage.
• Provisional — Only 1 dimension has direct or compatible ancestral measurement evidence. Not enough for a derived composite.
• Unranked — No dimension data available.

Tracked Benchmarks & Exclusions

Some benchmarks are tracked internally or shown in standalone charts but are not part of the composite IQ calculation:

• DesignArena Code Categories — A broad code-category Elo leaderboard from DesignArena. It is tracked as a source-backed raw benchmark, but is not yet part of Programmatic Reasoning or the WebDev & Design and Software Engineering domains because it needs its own calibration ladder and overlap review.
• GSO-Bench — Agentic software optimization: improve a codebase's runtime performance while preserving correctness. Optimization and kernel-style performance work fits the Machine Learning Engineering domain better, so it lives on the Machine Learning Benchmarks page alongside KernelBench and PostTrainBench v1.1 and is excluded from Composite IQ.

These benchmarks remain in the source-backed dataset for review and future charting — they are simply not included in the composite IQ computation.

Core Math Benchmarks

FrontierMath Tier 4, FrontierMath Tier 1–3, and MathArena are core inputs to the D2 (Mathematical Reasoning) dimension and are surfaced on the IQ page as standalone benchmark charts. Current benchmark lists sort by source-backed model coverage. Historical ladders are retained below for auditability.

• FrontierMath Tier 4 — the hardest math chart by current top-model percent correct; novel research-level problems with very low gameability.
• FrontierMath Tier 1–3 — harder than AIME, easier than T4, with Epoch's expert-human baseline supporting the current IQ 130 region.
• ProofBench — formally-verified proof writing. A different cognitive task than the problem-solving benchmarks because the model has to construct a verified proof, not just give an answer.
• MathArena — IRT-derived expected performance across non-deprecated math competitions. It adds broad competition-math coverage beyond AIME and FrontierMath while using a stricter midrange ladder so midrange scores do not overstate Mathematical Reasoning IQ.
• AIME — a useful competition-math signal, but treated as saturating because frontier systems are near the source ceiling and historical problem contamination is plausible.

Benchmark	IQ 70	85	100	115	130	145	160
FrontierMath T4 expected score	0	2	7	20	45	90	120
FrontierMath T1–3 expected score	0	8	18	36	62	95	125
ProofBench expected score	0	2	6	14	30	60	120
MathArena expected score	0	5	15	32	55	75	95
AIME expected score	-15	5	30	65	92	112	132

Emotional Reasoning Scoring

Emotional Reasoning (EQ) is a diagnostic domain, not a Composite IQ dimension. The three component signals below are mapped onto the shared 70–160 scale so users can inspect them together. These are not direct human IQ percentiles: EQ-Bench is AI-judged, and AttuneBench is participant-grounded but currently sparse.

The variable is written EQ above for historical continuity; it is a diagnostic Emotional Reasoning score averaged from the available source-backed component signals below.

One source-backed component is sufficient to produce a provisional Emotional Reasoning score, under this domain’s coverage rule. Available components are averaged; missing components are not imputed. Coverage is a confidence signal, so a one-source estimate should be read as substantially less certain than a score supported by multiple sources.

EQ-Bench 4 Elo → Emotional Reasoning

EQ-Bench 4 contributes using a stated Elo convention rather than a fitted relationship. A fit against AttuneBench for the eight overlapping models was rejected: the two benchmarks barely correlate in that sample, and the fitted slope compressed a 400-point Elo spread into eight EQ points with negative Elo pegs. Instead the frozen 27-model cohort median at capture, 1222 Elo, maps to EQ 115, and 100 Elo spans 15 EQ points, the same spacing the site uses for arena Elo ladders. This is a site-scale convention, not a human emotional-intelligence norm. The separate EQ-Bench 3 mapping and its provider adjustment are not applied to EQ-Bench 4.

EQ	70	85	100	115	130	145	160
Expected score	922	1022	1122	1222	1322	1422	1522

EQ-Bench 3 Elo → Emotional Reasoning

EQ-Bench 3 is a frozen legacy cohort: its publisher now leads with EQ-Bench 4 and no new models are added. It is retained as its own component, the way ARC-AGI-1 remains beside its successors, because it covers older and long-tail models that have no EQ-Bench 4 or AttuneBench result, and because the domain has few benchmarks. It was judged by a single Claude judge, so Anthropic rows have 300 Elo subtracted before mapping; on the eleven models that have both versions, that adjusted ladder lands within about one and a half EQ points of the EQ-Bench 4 convention on average. Its ladder is unchanged from before the EQ-Bench 4 migration.

Elo	200	600	900	1100	1300	1500	1700	2000	2300
EQ	75	85	92	96	100	113	123	132	140

AttuneBench Composite → Emotional Reasoning

AttuneBench evaluates models against participant annotations from 200 real multi-turn conversations in its Default mode. The Composite is a normalized aggregate across the benchmark's primary human-annotated metrics. This is the strongest participant-grounded Emotional Reasoning source, and its small human-baseline pilot suggests human annotators can bracket or exceed current model ranges on some metrics. AI IQ therefore maps 50 near the average-human reference, treats 52.5 as strong, and reserves 55+ for high-end emotional attunement. Current public coverage is only 11 rows in a narrow frontier range, so AttuneBench remains sparse and is not imputed.

AttuneBench Composite	Emotional Reasoning
45	85
50	100
52.5	112
55	125
60	140

Judge and style sensitivity

EQ-Bench 4 remains sensitive to roleplay style and automated judging. The same EQ-Bench 4 mapping applies to every provider, with no family-specific Elo penalty. EQ-Bench 3 results are never converted into EQ-Bench 4 scores; they count as their own component on their own ladder. A model with any one of the three components can receive a provisional EQ score.

Cost & Speed Metrics

Sticker Price — published price for a typical workload

AI IQ's effective-cost views are anchored to 1M I/O Tokens: 1M input tokens plus 1M output tokens, priced at the model's published per-million-token rates. Sticker Price is the dollar amount to process that standard workload:

where \(p_{\text{in}}\) and \(p_{\text{out}}\) are the published per-million-token prices in dollars.

Task Efficiency — how much work does the model use?

Sticker price alone hides large per-task differences in how much work a model uses to solve a benchmark. We estimate this with a measured-or-imputed usage multiplier. For each benchmark, AI IQ first estimates the task cost expected from a model's published input and output prices. The usage signal is the residual: actual task cost divided by expected task cost. Validated AA v4.3.2 output tokens per task are included as an additional workload-normalized signal where available. We compare only the same weighted evaluation suite and require paired source configuration, cost and token records. Historical suite-total tokens and costs are preserved as source data but excluded from this comparison; output usage does not measure input-token efficiency.

When validated direct token usage is available, AI IQ can use it directly. Benchmark-cost residuals are blended in when available, but a single benchmark-only residual is treated as provisional; benchmark-only effective cost requires at least two benchmark-cost signals. If a model has positive input/output pricing but no measured multiplier, AI IQ uses a conservative waterfall: one-generation-back same-family same-lineage multiplier, then two-generations-back same-family same-lineage multiplier, then the geometric average of the three closest measured peers, then a final assumed 1× multiplier. The Task Efficiency chart shows the inverse of the usage multiplier, so 2× means the model uses about half the task effort of the median model, and 0.5× means it uses about twice as much.

Effective Cost — what it actually costs to do the same task

The product of the two:

Reads as: what this model spends on a task after adjusting its 1M I/O Tokens sticker price by validated token usage and price-adjusted benchmark usage. Models below the diagonal (Effective Cost < Sticker Price) are task-efficient and cheaper than their sticker suggests; models above are task-hungry. This is the cost axis on every effective-cost-vs-quality chart.

Effective-cost charts require positive input and output token pricing. Free, zero-dollar, or rate-limited rows are not plotted as $0 effective cost. Open-weight models can enter these charts only when AI IQ records a specific nonzero hosted-provider price; temporarily subsidized or promotional rates should be avoided in favor of stable published pricing.

Token price and per-task cost

Alongside effective cost, the Cost page plots raw published token prices and two representative per-task cost blends. Token cost is a model's published input or output price per 1M tokens. Task cost weights those prices (plus cache-read pricing) by a realistic token mix for a kind of work over one million tokens:

• Coding task (cache-heavy): 800K cache-read + 100K fresh input + 100K output — 0.8·cache + 0.1·input + 0.1·output. Cache reads fall back to the input price when a provider publishes no cache pricing.
• Copywriting task (output-heavy): 150K input + 850K output — 0.15·input + 0.85·output.

These re-rank models by workload: the coding blend rewards cheap cache reads, while the copywriting blend rewards cheap output. Token and cache prices are published by each provider; the blends are AI IQ constructions documented here.

Speed metrics

AI IQ uses three user-facing speed terms. Latency is time to first token (TTFT): how long a model takes to begin responding. Throughput is tokens per second (TPS): how quickly the answer streams after generation begins. Completion time is time to last token (TTLT): how long the full answer takes to arrive.

The Speed page composes TTLT from a model's own measured parts rather than a single fixed figure: time to first answer token (which grows with input length and includes any hidden reasoning) plus output tokens ÷ TPS. Input- and output-length sliders set the workload. Time axes are reversed so faster models appear to the right.

Reading Chart Tooltips

Chart tooltips use the same structure across public chart surfaces. Click a model point, bar, or timeline label to open the tooltip; click outside it or scroll the page to dismiss it. Hover alone does not open a tooltip.

The first tab is a stable model summary rather than a chart-specific readout. Its order is always: IQ, Emotional Reasoning (EQ), Effective Cost/1M I/O, Task Efficiency, and Release Date. This makes models comparable across charts even when the chart axes are ordered differently.

The IQ and Cost tabs expose the supporting details. IQ shows the seven scored dimension scores, with benchmark-level evidence nested inside each dimension. Emotional Reasoning is shown separately as a diagnostic domain derived from EQ-Bench 4, historical EQ-Bench 3, and AttuneBench. Expanded Bayesian dimensions show direct benchmark coverage, a conditional 95% range, and the predecessor prior when present. Cost shows input cost per 1M tokens, output cost per 1M tokens, sticker cost per 1M I/O tokens, task efficiency, and effective cost per 1M I/O tokens.

Dimension bell-curve charts distinguish measured-enough points from lower-coverage estimates. For Abstract Reasoning, models with at least two source-backed D1/Abstract benchmarks use the standard solid provider-colored dots and are eligible for labels. Lower-coverage D1 estimates remain visible as same-size dots with slightly lighter provider-colored fills and ordinary solid outlines, but are not labeled by default. This keeps sparse but useful estimates inspectable without making them look as certain as source-backed D1 coverage.

Limitations & Transparency

• Dimension coverage varies. Some models have direct data for all 7 scored dimensions. Others qualify with direct or compatible ancestral evidence in as few as 2; their direct coverage can be lower when predecessor evidence supplies eligibility. Every score is an estimate, with confidence increasing as coverage rises. Always check the X/7 count and rank status.
• Benchmark mix matters. Two models with the same composite IQ may have very different underlying data quality. One might have all frontier benchmarks while another relies mostly on saturating or contamination-sensitive benchmarks. The rank status and dimension count help distinguish these cases.
• Imputation is uncertain. The Abstract Reasoning estimate equally blends the legacy calculation and a predictor based on observed capabilities, lineage and historical trends. D2–D7 jointly combine direct benchmark observations and one uncertain predecessor relationship. They do not treat inherited benchmark values as new measurements. Unsupported dimensions retain conservative legacy fallbacks; cold-start benchmark predictions remain a known weakness. These are reasonable estimates, not ground truth — a model's true ability on an unevaluated benchmark could be significantly higher or lower.
• Anchor calibration is subjective. The mapping from raw scores to IQ involves judgment calls about what different performance levels mean relative to human cognitive ability. We document our rationale for each benchmark, but reasonable people can disagree.
• IQ is a metaphor. Human IQ tests measure a specific construct via standardized instruments under controlled conditions. AI benchmark performance is a different thing. The IQ scale provides an intuitive frame of reference, not a claim of equivalence.
• Calibration ladders are a design choice. The expected score at each IQ level directly affects which models benefit and which are penalized. Models that excel on saturating benchmarks will need very high raw scores to receive very high implied IQ, which may feel unfair if those benchmarks genuinely reflect high ability. We believe the trade-off — rewarding harder evaluation — is correct, but the specific ladder values are judgment calls.
• Benchmarks become stale. As models improve and training data evolves, benchmark ladders, gameability ratings, and source selection may need revision. This methodology is a living document.

High-End Calibration

The expected-score ladders intentionally get more demanding above IQ 140 on many benchmarks. Each additional raw point often contributes less to implied IQ in the superhuman range than in the human range. This reflects three realities:

1. Human IQ distributions thin out at the tails. The difference between IQ 100 and IQ 120 is much more common than the difference between IQ 140 and IQ 160.
2. Superhuman benchmark scores are driven by breadth, not depth. A model scoring 50% on FrontierMath T4 isn't twice as smart as one scoring 25% — it covers more mathematical branches rather than being fundamentally more capable in any single branch.
3. Practical discrimination. Without demanding high-end ladders, reasoning vs. non-reasoning configurations of the same model can produce unrealistically large IQ gaps. The ladder shape keeps those differences meaningful without letting one saturated benchmark dominate the composite.

A perfect or near-perfect score can still imply IQ 160 when that is a reasonable interpretation of the source. In the current calibration, several mature or contamination-sensitive benchmarks instead require off-scale performance for IQ 145 or 160. The point is not to cap hard benchmarks forever, but to make the high-end requirements explicit enough that future calibration passes can decide when a sudden 100% score on a very hard benchmark should map to an appropriately high implied IQ.

Data Process

AI IQ keeps source-backed data, extracted updates, and derived scoring separate. That separation matters because a raw benchmark chart should show what a public source actually reported, while the composite IQ can use conservative scoring-only imputations to avoid rewarding missing coverage.

Source-backed benchmark data can exist for models that are not yet shown on public chart surfaces. A model must have launch metadata, must not be hidden, and must have a derived IQ before it appears in the main IQ charts. This keeps placeholder rows inspectable in the data table without letting a single benchmark import promote them into public trend charts.

Stage	Purpose	What is preserved
1. Capture	Save the raw leaderboard or source text used for an update.	The original pasted or scraped source capture.
2. Extract	Map source rows to canonical model entries and fields.	A small reviewable update listing exactly which model fields changed.
3. Apply	Write source-backed values into the model dataset.	Unknown values stay blank; unrelated fields are not guessed.
4. Score	Derive IQ on a temporary scoring copy.	Raw benchmark values remain source-backed; imputed values are used only for derived IQ.

Manual source captures that may be hard to reproduce exactly are archived so the same raw data can be re-parsed later if the extraction rules improve. Larger generated scrapes can be refreshed from the original source and do not need to be treated as permanent public evidence in the same way.

Model Tiers

The tier filter on charts groups models into six levels: Flagship, Secondary, Tertiary, Compact, Ultralight, and Coding. Tiers describe where a model sits within its own lab's lineup, not how it scores against other labs' models — a lab's best current offering is always Flagship, and the levels below follow that lab's own ladder down to its smallest models. Coding-specialist releases sit in their own filter bucket. Open-weight releases count as part of the same lab's lineup. This keeps tier and measured intelligence independent: filtering to Flagship compares each lab's best model head-to-head, and the scores — not the tier — show how those bets actually perform. Because tiers are lab-relative, a Tertiary model from one lab can outscore a Flagship model from another; that gap is a finding, not an error. Not every lab fills every level, and tier assignments change only when a lab restructures its lineup, never when new benchmark results arrive.

Chart Inclusion

AI IQ separates model data from chart display policy. A model can exist in the dataset, have source-backed benchmark rows, and still be absent from a public chart if it does not meet that chart's policy or required fields.

Most public chart surfaces use a default policy based on publication status, derived IQ availability, model type, provider tier, and generation recency. Each major lab's main model line keeps several previous generations on charts; other top-tier models keep one previous generation; lower-tier variants such as mini, nano, Sonnet, Haiku, Flash, and smaller open-weight sizes are included only for the current generation. Archive and hidden rows are excluded unless explicitly overridden.

The IQ Over Time chart uses a stricter frontier-timeline policy. It requires a release date, derived IQ, a public model row, a general-purpose model type, membership in a top-lab provider grouping, and either top-tier status or membership in one of the lab's recognized main model lines — the latter keeps a lab's historical flagship line (for example Opus) on the timeline after the lab restructures its lineup above it. Provider lines then connect non-decreasing IQ checkpoints rather than every model from that provider.

The policy fields are maintained in the admin dashboard: publication status, model type, provider tier, model line, generation offset, and display override. Provider tier follows each lab's own ladder as described under Model Tiers; lineup names such as OpenAI mini/nano, Anthropic Sonnet/Haiku, Google Flash, and Qwen size tiers map onto that ladder per lab rather than acting as generic cross-provider role names.

Sources

Benchmark scores, prices, and token usage come from publicly published leaderboards. Each source is sampled periodically and reconciled against published numbers before being applied.

• Artificial Analysis Intelligence Index — the primary aggregator. Provides scores for AIME, GPQA Diamond, SWE-Bench Verified, HLE, SciCode, Terminal-Bench 4.0, Terminal-Bench Hard, CritPt, LiveCodeBench, IFBench, MMMU-Pro, and the AA composite indices (Omniscience, GDPval, τ₂-Bench Telecom, Long Context Reasoning), plus per-model pricing, response time, median throughput, total evaluation cost for the AA suite, and token-usage data used for task efficiency.
• Arena.ai Overall — Arena.ai's head-to-head text Elo ratings and ranks
• Arena.ai WebDev — pairwise web-app development evaluations using Bradley-Terry ratings
• DesignArena Full Stack — human-preference Elo for end-to-end frontend + backend builds
• DesignArena Code Categories — broad code-category Elo tracked as a raw benchmark pending calibration
• ARC Prize leaderboard — ARC-AGI-1, ARC-AGI-2, ARC-AGI-3 scores, and ARC-AGI per-task cost where published
• Vals.ai — the Vals Index, AIME, ProofBench, SWE-Bench, and LiveCodeBench source views where available
• Vals.ai LegalBench, Legal Research Bench, and Harvey Legal Agent Benchmark — the three equally weighted Legal-domain source views
• Scale Labs SWE-Bench Pro Public Dataset — public SWE-Bench Pro Resolve Rate source
• SWE Marathon — long-horizon software-engineering Resolution Rate (Pass@1), using best published agent/harness rows for canonical model-level scoring
• Cognition FrontierCode — FrontierCode Diamond score for high-quality production-code tasks, using canonical public model rows where the source model identity is clear
• Mercor APEX-SWE — professional software-engineering overall Pass@1 scores
• SWE-rebench — continuously evolving and decontaminated software-engineering leaderboard rows
• MathArena — model-level expected performance across non-deprecated math competitions
• DeepSWE v1.1 — Datacurve's live leaderboard for original long-horizon software-engineering tasks
• GSO-Bench — software optimization Opt@1 leaderboard using OpenHands scaffold rows for canonical model-level scoring
• SWE-Bench — SWE-Bench Verified leaderboard rows, using clear single-model agent/model pairs for model-level scoring
• LiveCodeBench — continuously refreshed coding-problem benchmark sourced from live contests
• Terminal-Bench — Terminal-Bench 4.0 and Terminal-Bench Hard task accuracy
• BrowseComp — hard browsing benchmark with published human-trainer performance used for calibration
• Humanity's Last Exam — benchmark semantics for the HLE calibration caveat
• CritPt paper — research-physics benchmark semantics for the CritPt calibration caveat
• SciCode — scientific coding benchmark results
• Agents' Last Exam — agent/harness leaderboard for real-world professional workflows, using Full / Overall pass rate for canonical model-level scoring
• OSWorld-Verified and Toolathlon — sparse computer-use and tool-use signals; current score rows use clear mirror/self-reported mappings, with official sources used for benchmark semantics and exact-match validation
• MCP Atlas — real MCP-server tool-use tasks with claim-level scoring
• IFBench, AA Omniscience, and AA Long Context Reasoning v1.1 — instruction-following, factual-reliability, and long-document reasoning signals used for the Reliability dimension
• Epoch MirrorCode — ML,+Private,2L full-program solve rates shared by Programmatic Reasoning and Software Engineering
• Artificial Analysis model leaderboard, AA-AnalystAgent, and EnterpriseOps-Gym-AA — source-backed Knowledge Work measurements; AA-Briefcase uses Elo, AA-AnalystAgent uses pass^5, and the other D7 inputs retain their published percentage or normalized units
• BullshitBench v2 — a pushback / anti-sycophancy signal (whether a model challenges false premises) used for the Reliability dimension
• FACTS Grounding — Google's source-grounded long-form factuality benchmark, using the Score column from a Kaggle FACTS Grounding leaderboard capture for the Reliability dimension
• Epoch AI — FrontierMath Tier 1–3 and Tier 4 accuracy
• EQ-Bench 4 — emotional-intelligence Elo
• AttuneBench — emotional-attunement benchmark from real human-AI conversations

The Artificial Analysis Intelligence Index can list two rows for the same model under one display name when the same underlying model has both a reasoning and a non-reasoning configuration (the reasoning row is marked with a 💡 lightbulb icon). When the two configurations differ meaningfully on cost, latency, or quality, they are tracked as separate model entries (e.g. reasoning vs non-reasoning variants of the same release).