Methodology

AI IQ assigns each model an estimated IQ score across 6 equally weighted capability dimensions, each measured by source-backed benchmarks where coverage exists. Every benchmark uses a calibrated ladder: what score would correspond to IQ 70, 85, 100, 115, 130, 145, and 160 on that task? One source-backed benchmark is sufficient to estimate a dimension; additional coverage increases confidence. Missing benchmarks and dimensions are conservatively imputed inside the scoring pipeline, and every derived composite averages all six dimensions.

This page documents the full scoring system: how source data is captured, how the six dimensions are defined, how raw scores map to IQ, and how missing values are imputed without changing source-backed benchmark data. Every dimension value remains an estimate even at complete benchmark coverage. Emotional Reasoning (EQ) is a separate applied-capability domain and is excluded from Composite IQ.

The 6-Dimension Framework

AI IQ organizes Composite IQ evaluation into six equally weighted dimensions. Each dimension uses source-backed benchmarks that are averaged together, with missing benchmarks conservatively imputed only inside the scoring pipeline:

• Frontier benchmarks are hard, low-gameability tests where a large score gain at the frontier usually represents a large capability gain.
• Saturating benchmarks are easier, more gameable, or already clustered near the top. Their ladders still reach IQ 160 where appropriate, but the score expectations rise quickly near the high end so a merely high score does not overstate ability.

The composite IQ requires at least 2 of 6 scored dimensions to have source-backed data. Models with fewer scored dimensions do not receive a derived composite IQ.

Formulas

Each benchmark raw score \(s\) is converted to an IQ value by comparing it with that benchmark's expected score at fixed IQ levels. Let \(S(q_i)\) be the expected raw score for IQ level \(q_i\), where \(q_i \in \{70,85,100,115,130,145,160\}\). For a score between two adjacent expected scores, AI IQ uses piecewise-linear interpolation:

Each dimension averages the IQ values of its benchmarks. Missing benchmarks are conservatively imputed before averaging using the benchmark-level waterfall below, except in Abstract Reasoning once any source-backed ARC result exists: that dimension averages only the available source-backed, jointly calibrated ARC projections so later coverage reveals cause the smallest justified point-estimate change.

The six equally weighted scored dimensions:

\(f\) is piecewise-linear interpolation through each benchmark's expected-score ladder.

The composite IQ is the mean of all six scored dimension scores. At least 2 dimensions must be source-backed or predecessor-imputed before any missing whole dimensions are filled:

D1: Abstract Reasoning

Fluid abstraction is the ability to solve novel problems without relying on prior knowledge. This is the closest analogue to fluid intelligence in human psychometrics — raw problem-solving ability applied to patterns never seen before.

D2: Mathematical Reasoning

Mathematical reasoning and quantitative problem-solving — the ability to work with mathematical structures, proofs, and analytical frameworks. The hard benchmarks test novel quantitative reasoning that cannot be memorized from training data. Human references differ by benchmark: FrontierMath uses specialist and expert-team math references, ProofBench uses formalization ability, and MathArena/AIME use competition-math populations rather than the general public.

D3: Academic Reasoning

Academic Reasoning measures expert knowledge, research-style analysis, and difficult problem solving across academic fields. Its scored set uses Humanity's Last Exam, GPQA Diamond, CritPt, SciCode, MMLU-Pro, and MMMU-Pro. CritPt and SciCode remain here because their bottleneck is high-level scientific reasoning, even when mathematics or code is the medium.

Finance Domain

Finance is an applied domain outside Composite IQ. It measures financial analysis, long-document interpretation, spreadsheet modeling, banking operations, multimodal mortgage-tax extraction, and tax reasoning through Finance Agent v2, CorpFin v2, the Excel Modeling Benchmark, τ³-Banking, MortgageTax, and TaxEval v2.

Finance IQ balances three capability groups rather than averaging its scored benchmarks flat: Financial Analysis; Documents and Modeling; and Banking, Tax, and Operations. One source-backed scored benchmark is sufficient to produce a provisional estimate, matching the eligibility convention used by other applied domains. Missing scored benchmarks are filled at the lower quartile of capability-matched models, capped at the model's observed Finance benchmark average, before all three group means are weighted equally. This mirrors the conservative missing-coverage principle used by IQ dimensions and prevents absent results from improving a model's score. Coverage remains a confidence signal, not an eligibility gate. τ³-Banking remains visible as a diagnostic system benchmark, but is excluded from Finance IQ because retrieval strategy, reasoning effort, and agent scaffolding currently vary too much for clean model-level comparison.

These are model-plus-system evaluations. Finance Agent and Excel Modeling depend materially on tools and agent scaffolds; CorpFin and TaxEval use LLM judging. The July 24 calibration uses conservative occupational reference points rather than fitting the current model distribution: ordinary non-specialists sit in the low anchors, careful generalists or trainees around IQ 100, capable practitioners around IQ 115–130, and near-reliable expert work around IQ 145. Because matched professional-human score distributions are not published, every Finance projection remains provisional, sparse, or high-risk.

WebDev & Design Domain

WebDev & Design is an applied domain outside Composite IQ. It combines coding with product reasoning, user experience, interaction design, visual-spatial layout, and full-stack implementation.

Methodology caveat: the current set is intentionally focused on greenfield app and web-artifact quality, and its inputs are not four fully independent sub-skills. Arena.ai WebDev, DesignArena Frontend, and DesignArena Full Stack all contain overlapping product, UI, UX, prompt-following, and implementation-quality signals. Vibe Code Bench adds a more task-suite-like app-building signal, but this domain should still be read as a practical product-building cluster rather than a rigorously factor-separated measurement. Broader coverage across product architecture, user-intent inference, accessibility, maintainability, and long-horizon iteration would make the domain stronger.

Legal Domain

Legal is an applied domain outside Composite IQ. Legal IQ equally weights three capabilities after projecting each benchmark onto the AI IQ scale: broad legal reasoning, multi-source legal research, and long-horizon production of professional legal work. A model needs source-backed results on at least two of the three benchmarks; missing values are not imputed.

How to read the calibration tables: the column headings are IQ anchors, and the second row gives the raw benchmark score expected at each anchor. For example, Harvey LAB raw scores of 0%, 5%, 15%, 35%, and 70% map to IQ 100, 115, 130, 145, and 160 respectively.

Healthcare Domain

Healthcare is an applied domain outside Composite IQ. Healthcare IQ equally weights seven capabilities after projecting each benchmark onto the AI IQ scale: patient safety, clinical reasoning under uncertainty, complex diagnosis, clinical workflow execution, clinician communication, clinical documentation, and medical coding. A model needs source-backed results on at least two of the seven benchmarks; missing values are not imputed. Four benchmarks are sourced from the MAST clinical leaderboard, which is currently in preview, two from Vals.ai's private-data healthcare evaluations, and one from Medical Sphere's independent re-runs; snapshots are dated and scores move with their sources.

The projections are human-equivalent task-performance references, not psychometric IQ measurements of clinicians or models. IQ 100 represents an educated nonexpert or early trainee, IQ 115 a capable trainee or junior professional, IQ 130 an independent experienced professional, IQ 145 an exceptional specialist, and IQ 160 near-best human or expert-team performance. Exact same-protocol human results take priority; same-metric transfers and cohort comparisons come next. When no compatible human result exists, the ladder is an explicit lower-confidence estimate from task prerequisites, occupational competence, and the scoring floor and ceiling. Current model standings are inspected only after these anchors are fixed and are never used as the target.

Partial coverage changes confidence, not eligibility. The site first maps every result through its human-calibrated benchmark ladder, then fits a partially pooled two-way model across the sparse model-by-benchmark matrix. Each observation is represented as the model's latent Healthcare IQ plus a benchmark-specific AI task-profile effect. Those effects are learned only from models with the required two-result overlap and are constrained to average zero across the seven benchmarks, preserving the absolute human-anchored IQ scale. This corrects the tested-task mix without changing a benchmark's human calibration or treating leaderboard ranks as independent head-to-head wins.

The model estimate uses a weak IQ 120 prior with one-half benchmark of weight and shrinks benchmark effects with one benchmark of prior weight. These settings were selected by masking models with six or seven results down to every two- and three-benchmark subset: recovery RMSE fell from 8.48 IQ points for the available-only mean and 7.09 for the earlier fixed coverage adjustment to 2.48 and 2.05 respectively. Models with 2/7 or 3/7 coverage remain eligible and can rank first; they simply carry wider approximate 95% ranges. Tooltips disclose the raw available-benchmark mean, the net task-mix and pooling adjustment, coverage, and uncertainty range.

This estimator is currently a Healthcare-only pilot. Other applied domains and the six Composite IQ dimensions retain their existing scoring methods. Expansion requires prospective evidence that newly arriving benchmark results fall reasonably within the pilot's predicted ranges, sparse estimates converge without systematic bias as coverage grows, learned task-profile effects remain stable, and uncertainty ranges are not materially overconfident. Any later domain must tune and validate its own prior, pooling strength, and residual variance rather than inheriting Healthcare's settings.

Retrieval-augmented clinical products and medically fine-tuned model variants that appear on the source leaderboards are excluded: Healthcare IQ compares general-purpose foundation models under a common protocol.

D4: Programmatic Reasoning & the Software Engineering Domain

Programmatic Reasoning measures transferable problem solving through code: algorithmic reasoning, decomposition, execution, debugging, and task completion. It uses LiveCodeBench, IOI, Terminal-Bench 2.1, ProgramBench, FrontierSWE, and SWE-rebench. ProgramBench is scored using its Almost Resolved metric: the share of tasks passing at least 95% of hidden tests.

The broader Software Engineering domain also includes SWE-bench Verified, APEX-SWE, DeepSWE v1.1, and SWE Marathon. Those benchmarks are valuable measures of production repository work but are not separately counted in Composite IQ; ProgramBench, FrontierSWE, and SWE-rebench intentionally overlap the dimension and domain.

Academic Reasoning Benchmark Details

Academic Reasoning captures expert knowledge, research-style reasoning, and difficult problem analysis under uncertainty. The frontier benchmarks test whether a model can reason through questions that push the boundaries of human expertise itself, while the broader benchmarks add cross-field academic and graduate-level knowledge signals.

Programmatic benchmark detail: Terminal-Bench 2.1

Terminal-Bench 2.1 is part of Programmatic Reasoning because its tasks are typically coding-heavy and require deep reasoning about code and complex scripts. Terminal-Bench Hard is retained only as a historical raw benchmark; it is not scored because it is a hard subset of the current suite.

D5: Computer Use

Computer Use measures practical task execution across browsers, desktops, tools, and multi-step workflows. Its scored benchmarks are BrowseComp, OSWorld-Verified, Toolathlon, MCP Atlas, Arena.ai Agent Arena, and Agents' Last Exam.

D6: Reliability

Reliability captures factuality, instruction following, long-chain reasoning, metacognition and calibration, false-premise resistance, and source-grounded document work. Its scored set is SimpleQA Verified, AA Omniscience, BullshitBench v2, IFBench, MultiChallenge, AA Long Chain Reasoning, and FACTS Grounding.

This unified dimension is a deliberate interim choice. As benchmark coverage improves, Reliability may split into two narrower dimensions—Metacognition and Executive Control. Working Memory and Visual Reasoning are also candidates for future dimensions, but are not included yet.

Emotional Reasoning (EQ)

Emotional Reasoning (EQ) measures social, emotional, and conversational judgment as a domain, separate from Composite IQ. The available benchmark base is weaker than the scored IQ dimensions: EQ-Bench 3 uses a Claude/Anthropic-family judge, while AttuneBench is participant-grounded but still sparse.

Expected-Score Interpolation

Each benchmark defines expected raw scores at seven fixed IQ levels: 70, 85, 100, 115, 130, 145, and 160. For scores that fall between two adjacent expected scores, AI IQ uses piecewise-linear interpolation:

If the score is at or below the lowest expected score, the model receives IQ 70. If it is at or above the highest expected score, it receives IQ 160 for that benchmark. There is no extrapolation beyond the defined range.

This approach makes the calibration question explicit. Instead of first choosing a curve family, each benchmark asks: what would an IQ 70, 85, 100, 115, 130, 145, or 160 model-equivalent score on this source? Each segment can then have a different slope, allowing threshold-like benchmarks such as CritPt or FrontierMath T4 to behave differently from smoother bounded tasks such as ARC or Terminal-Bench.

Benchmark Calibration & Averaging

Each dimension gives its benchmarks equal weight. Most dimensions conservatively fill missing benchmark slots before averaging. Abstract Reasoning instead averages the available source-backed ARC projections once any direct ARC result exists; its three ladders are jointly calibrated so coverage reveals are order-invariant and cause the smallest justified score movement. Models with zero direct ARC coverage can still use conservative lineage estimates.

Why Use Expected Scores?

The calibration is framed as a human-readable judgment rather than as an abstract curve fit. For each benchmark, the table answers seven concrete questions: what score would a 70, 85, 100, 115, 130, 145, or 160 IQ-equivalent system be expected to achieve?

That means an extremely hard benchmark can assign a high implied IQ to a low-looking score. CritPt, for example, treats 0% as compatible with IQ 100 because a normal human would not be expected to score on it. AIME does the opposite at the high end: because frontier systems can approach saturation and the public problem set is contamination-sensitive, its calibration ladder puts IQ 145 and 160 outside the source's natural 0–100% range.

Benchmark-Level Imputation

When a model has partial benchmark coverage inside most dimensions, missing benchmarks are filled in before the dimension IQ is averaged. Abstract Reasoning is the exception: with one or more source-backed ARC results, its point estimate is the equal mean of only those direct projected IQs. Additional ARC coverage raises confidence and updates that mean; predecessor, momentum, and peer estimates do not enter the point estimate once direct ARC evidence exists.

Correlation estimates use source-backed overlap among benchmarks in the same dimension. If Benchmark A historically predicts Benchmark B, a model's score on A can estimate B, but only after the observed correlation is shrunk for small samples and an uncertainty penalty is subtracted. A one-benchmark dimension receives the largest penalty, and imputed benchmark IQs are capped below the observed benchmark average so sparse coverage cannot make a model look better than its actual evidence.

The ordinary within-dimension estimate uses two ingredients:

• The model's available-benchmark IQ average — how the model is performing on the benchmarks it does have in this dimension. This is the within-dimension signal: if a model is hitting IQ 130 on the dimension's other benchmarks, the missing one is probably also somewhere around 130.
• The benchmark's 80th-percentile IQ (\(P_{80}\)) — a per-benchmark cap derived from the actual data. Take every model that has a real score on that benchmark, convert each score to an implied IQ via the expected-score ladder, sort those IQs from low to high, and take the value at the 80th-percentile rank. So if 50 models have HLE scores yielding implied IQs ranging from 70 to 155, \(P_{80}(\text{HLE})\) is the implied IQ at the 80th-percentile rank in that sorted list. It is where strong-but-not-frontier measured models actually land on this benchmark.

The imputed value is the minimum of the two:

Benchmark Imputation Waterfall

ARC estimates are used only on scoring copies, never in raw benchmark tables. ARC-AGI-1/2 momentum uses a two-year window of source-backed transitions released before the target, keeps reasoning and non-reasoning histories separate, and gives each lab one vote in the global trend. ARC-AGI-3 carries a source-backed primary-lineage ancestor without uplift; without one, it requires three represented labs and caps its lower-quartile market fallback at 1%. Models that cannot use these rules continue through predecessor and lower-tail peer fallbacks. For ARC-AGI-2 and CritPt, missing values are treated as zero only after earlier imputation steps fail. The raw benchmark table still leaves values blank when no source row exists.

Predecessor imputation is constrained by model family, and non-reasoning variants do not impute from reasoning variants. These lineage choices are scoring assumptions only; they do not create source-backed raw benchmark values.

Correlation imputation is deliberately narrower than the generic within-dimension rule. It primarily improves one-benchmark cases; broader-coverage rows continue to use the ordinary within-dimension cap. If a dimension has no source-backed benchmark evidence, no correlation estimate is attempted — the dimension itself is either left missing or filled at the dimension level (see Composite IQ Calculation below). The Emotional Reasoning (EQ) domain is not used in Composite IQ.

Composite IQ Calculation

Step 1: Score Each Dimension

For every dimension with usable benchmark coverage, compute a dimension estimate from its source-backed and scoring-only imputed benchmark IQs. If coverage is partial, the estimate receives a coverage confidence score. Abstract Reasoning with direct ARC evidence is not confidence-shrunk: its point estimate is the equal mean of available source-backed ARC projections, while its 1/3, 2/3, or 3/3 coverage communicates uncertainty. Other partially covered dimensions continue to use the matched lower-quartile shrinkage rule.

If there is no source-backed evidence in a dimension, it is usually treated as missing and handled by dimension-level imputation; the main exception is Abstract Reasoning, where no-D1-source rows can receive conservative ARC-AGI-1/2 lower-tail peer estimates before the dimension is averaged.

Step 2: Dimension-Level Imputation

If a model has at least 2 scored dimensions but is missing some of the others, every missing dimension is imputed before the composite is averaged. The cap is matched to models with real data for that dimension and similar capability across the other dimensions:

where \(\overline{\mathrm{IQ}}_{\text{scored dims}}\) is the model's average IQ across the dimensions it does have. For the cap, we look at models released on or before the scored model that have real data on the missing dimension, compare their average IQ across the other dimensions, and use the lower-quartile missing-dimension IQ among the closest comparable models. If the comparable set is too thin, we fall back to the same-era lower quartile for that dimension. If no same-era real data exists for a dimension, we do not invent a neutral default; the model does not receive a derived all-dimension IQ.

In practice, comparable models are those whose average across the non-missing dimensions is within a small IQ band of the model being scored and whose release date is not later than the scored model. If fewer than three comparable models are available, we use the nearest same-era measured models instead. This keeps missing dimensions conservative without letting older models borrow strength from future benchmark cohorts.

All six scored dimensions are always used for derived IQ. Missing a hard dimension such as Abstract Reasoning or Programmatic Reasoning should not improve a model's score, so missing dimensions are filled conservatively rather than omitted from the average.

Dimension Imputation Waterfall

Step 3: Compute the Composite

where all six scored dimensions are used once missing dimensions are imputed.

Key rules:

• Minimum 2 dimensions required. Models with fewer than 2 scored dimensions do not receive a derived composite IQ.
• No omitted dimensions. Models with enough coverage for derived IQ always use a 6-dimension composite; missing dimensions are conservatively imputed.
• Transparent count. The display shows X/6 so readers can see how many scored dimensions had source-backed data before dimension-level imputation.
• Equal weighting. All dimensions contribute equally. Benchmark-specific expected-score ladders, not differential weighting, handle benchmark quality differences. This keeps Composite IQ transparent and neutral rather than introducing target-specific weights before there is enough external validation data to justify them.

Rank Status

Each model receives a rank status reflecting the completeness of its evaluation:

• Full — All 6 scored dimensions covered. The highest-confidence estimate.
• Partial — 2–4 dimensions scored. Composite is derived but based on incomplete coverage.
• Provisional — Only 1 dimension scored. Not enough for a derived composite.
• Unranked — No dimension data available.

Tracked Benchmarks & Exclusions

Some benchmarks are tracked internally or shown in standalone charts but are not part of the composite IQ calculation:

• DesignArena Code Categories — A broad code-category Elo leaderboard from DesignArena. It is tracked as a source-backed raw benchmark, but is not yet part of Programmatic Reasoning or the WebDev & Design and Software Engineering domains because it needs its own calibration ladder and overlap review.
• GSO-Bench — Agentic software optimization: improve a codebase's runtime performance while preserving correctness. Optimization and kernel-style performance work fits the Machine Learning Engineering domain better, so it lives on the Machine Learning Benchmarks page alongside KernelBench and PostTrainBench and is excluded from Composite IQ.

These benchmarks remain in the source-backed dataset for review and future charting — they are simply not included in the composite IQ computation.

Core Math Benchmarks

FrontierMath Tier 4, FrontierMath Tier 1–3, ProofBench, MathArena, and AIME are core inputs to the D2 (Mathematical Reasoning) dimension and are surfaced on the IQ page as standalone benchmark charts. The public charts are ordered hardest to easiest by the current top model's source-backed percent correct.

• FrontierMath Tier 4 — the hardest math chart by current top-model percent correct; novel research-level problems with very low gameability.
• FrontierMath Tier 1–3 — harder than AIME, easier than T4, with Epoch's expert-human baseline supporting the current IQ 130 region.
• ProofBench — formally-verified proof writing. A different cognitive task than the problem-solving benchmarks because the model has to construct a verified proof, not just give an answer.
• MathArena — IRT-derived expected performance across non-deprecated math competitions. It adds broad competition-math coverage beyond AIME and FrontierMath while using a stricter midrange ladder so midrange scores do not overstate Mathematical Reasoning IQ.
• AIME — a useful competition-math signal, but treated as saturating because frontier systems are near the source ceiling and historical problem contamination is plausible.

Emotional Reasoning Scoring

Emotional Reasoning (EQ) is a diagnostic domain, not a Composite IQ dimension. The two component signals below are mapped onto the shared 70–160 scale so users can inspect them together. These are not direct human IQ percentiles: EQ-Bench is AI-judged, and AttuneBench is participant-grounded but currently sparse.

The variable is written EQ above for historical continuity; it is a diagnostic Emotional Reasoning score averaged from the available source-backed component signals below.

One source-backed component is sufficient to produce a provisional Emotional Reasoning score, consistent with the one-benchmark minimum used across applied domains. Both components are averaged when available; missing components are not imputed. Coverage is a confidence signal, so a one-source estimate should be read as substantially less certain than a score supported by both sources.

EQ-Bench 3 Elo → Emotional Reasoning

EQ-Bench 3 produces Elo ratings from head-to-head emotional-roleplay matchups judged by Claude Opus 4.6. This makes it useful as a dedicated affective/emotional reasoning signal, but it is not neutral ground truth: the judge is Claude/Anthropic-family, so the benchmark can favor Claude-like response style and penalize models that solve the scenario in a substantially different voice. AI IQ treats the Elo scale as a relative contest scale and applies its own human-reference hypothesis: 1300 is the average-human reference point for this structured roleplay task, 1500 is strong, and current frontier Elo values are high-end but kept near the AttuneBench range because the source is subjective and future Elo scores can exceed current rows. The mapping therefore keeps headroom above 2000 Elo:

AttuneBench Composite → Emotional Reasoning

AttuneBench evaluates models against participant annotations from 200 real multi-turn conversations in its Default mode. The Composite is a normalized aggregate across the benchmark's primary human-annotated metrics. This is the strongest participant-grounded Emotional Reasoning source, and its small human-baseline pilot suggests human annotators can bracket or exceed current model ranges on some metrics. AI IQ therefore maps 50 near the average-human reference, treats 52.5 as strong, and reserves 55+ for high-end emotional attunement. Current public coverage is only 11 rows in a narrow frontier range, so AttuneBench remains sparse and is not imputed.

EQ-Bench Style-Sensitivity Adjustment

Because EQ-Bench 3 is judged by Claude, it is treated as a weak component rather than a standalone authority. To reduce family/style bias while preserving its dedicated task coverage, we subtract a 300-point Elo adjustment from the EQ-Bench component for Anthropic models before mapping to the implied Emotional Reasoning score. If stronger participant-grounded or independently judged emotional-reasoning benchmarks gain broader coverage, EQ-Bench 3 is a candidate for removal from the domain composite.

Cost & Speed Metrics

Sticker Price — published price for a typical workload

AI IQ's effective-cost views are anchored to 1M I/O Tokens: 1M input tokens plus 1M output tokens, priced at the model's published per-million-token rates. Sticker Price is the dollar amount to process that standard workload:

where \(p_{\text{in}}\) and \(p_{\text{out}}\) are the published per-million-token prices in dollars.

Task Efficiency — how much work does the model use?

Sticker price alone hides large per-task differences in how much work a model uses to solve a benchmark. We estimate this with a measured-or-imputed usage multiplier. For each benchmark, AI IQ first estimates the task cost expected from a model's published input and output prices. The usage signal is the residual: actual task cost divided by expected task cost. Validated direct token-usage data is included as an additional signal where available.

When validated direct token usage is available, AI IQ can use it directly. Benchmark-cost residuals are blended in when available, but a single benchmark-only residual is treated as provisional; benchmark-only effective cost requires at least two benchmark-cost signals. If a model has positive input/output pricing but no measured multiplier, AI IQ uses a conservative waterfall: one-generation-back same-family same-lineage multiplier, then two-generations-back same-family same-lineage multiplier, then the geometric average of the three closest measured peers, then a final assumed 1× multiplier. The Task Efficiency chart shows the inverse of the usage multiplier, so 2× means the model uses about half the task effort of the median model, and 0.5× means it uses about twice as much.

Effective Cost — what it actually costs to do the same task

The product of the two:

Reads as: what this model spends on a task after adjusting its 1M I/O Tokens sticker price by validated token usage and price-adjusted benchmark usage. Models below the diagonal (Effective Cost < Sticker Price) are task-efficient and cheaper than their sticker suggests; models above are task-hungry. This is the cost axis on every effective-cost-vs-quality chart.

Effective-cost charts require positive input and output token pricing. Free, zero-dollar, or rate-limited rows are not plotted as $0 effective cost. Open-weight models can enter these charts only when AI IQ records a specific nonzero hosted-provider price; temporarily subsidized or promotional rates should be avoided in favor of stable published pricing.

Token price and per-task cost

Alongside effective cost, the Cost page plots raw published token prices and two representative per-task cost blends. Token cost is a model's published input or output price per 1M tokens. Task cost weights those prices (plus cache-read pricing) by a realistic token mix for a kind of work over one million tokens:

• Coding task (cache-heavy): 800K cache-read + 100K fresh input + 100K output — 0.8·cache + 0.1·input + 0.1·output. Cache reads fall back to the input price when a provider publishes no cache pricing.
• Copywriting task (output-heavy): 150K input + 850K output — 0.15·input + 0.85·output.

These re-rank models by workload: the coding blend rewards cheap cache reads, while the copywriting blend rewards cheap output. Token and cache prices are published by each provider; the blends are AI IQ constructions documented here.

Speed metrics

AI IQ uses three user-facing speed terms. Latency is time to first token (TTFT): how long a model takes to begin responding. Throughput is tokens per second (TPS): how quickly the answer streams after generation begins. Completion time is time to last token (TTLT): how long the full answer takes to arrive.

The Speed page composes TTLT from a model's own measured parts rather than a single fixed figure: time to first answer token (which grows with input length and includes any hidden reasoning) plus output tokens ÷ TPS. Input- and output-length sliders set the workload. Time axes are reversed so faster models appear to the right.

Reading Chart Tooltips

Chart tooltips use the same structure across public chart surfaces. Click a model point, bar, or timeline label to open the tooltip; click outside it or scroll the page to dismiss it. Hover alone does not open a tooltip.

The first tab is a stable model summary rather than a chart-specific readout. Its order is always: IQ, Emotional Reasoning (EQ), Effective Cost/1M I/O, Task Efficiency, and Release Date. This makes models comparable across charts even when the chart axes are ordered differently.

The IQ and Cost tabs expose the supporting details. IQ shows the six scored dimension scores, with benchmark-level evidence nested inside each dimension. Emotional Reasoning is shown separately as a diagnostic domain derived from EQ-Bench 3 and AttuneBench. Cost shows input cost per 1M tokens, output cost per 1M tokens, sticker cost per 1M I/O tokens, task efficiency, and effective cost per 1M I/O tokens.

Dimension bell-curve charts distinguish measured-enough points from lower-coverage estimates. For Abstract Reasoning, models with at least two source-backed D1/Abstract benchmarks use the standard solid provider-colored dots and are eligible for labels. Lower-coverage D1 estimates remain visible as same-size dots with slightly lighter provider-colored fills and ordinary solid outlines, but are not labeled by default. This keeps sparse but useful estimates inspectable without making them look as certain as source-backed ARC coverage.

Limitations & Transparency

• Dimension coverage varies. Some models have data for all 6 scored dimensions; others have as few as 2 (with the rest imputed). Every score is an estimate, with confidence increasing as coverage rises. Always check the X/6 count and rank status.
• Benchmark mix matters. Two models with the same composite IQ may have very different underlying data quality. One might have all frontier benchmarks while another relies mostly on saturating or contamination-sensitive benchmarks. The rank status and dimension count help distinguish these cases.
• Imputation is conservative, not clairvoyant. Missing values are filled first from explicit direct-predecessor lineage when available, then from within-dimension benchmark evidence, then from comparable measured models at the dimension level. These are reasonable estimates, not ground truth — a model's true ability on an unevaluated benchmark could be significantly higher or lower.
• Anchor calibration is subjective. The mapping from raw scores to IQ involves judgment calls about what different performance levels mean relative to human cognitive ability. We document our rationale for each benchmark, but reasonable people can disagree.
• IQ is a metaphor. Human IQ tests measure a specific construct via standardized instruments under controlled conditions. AI benchmark performance is a different thing. The IQ scale provides an intuitive frame of reference, not a claim of equivalence.
• Calibration ladders are a design choice. The expected score at each IQ level directly affects which models benefit and which are penalized. Models that excel on saturating benchmarks will need very high raw scores to receive very high implied IQ, which may feel unfair if those benchmarks genuinely reflect high ability. We believe the trade-off — rewarding harder evaluation — is correct, but the specific ladder values are judgment calls.
• Benchmarks become stale. As models improve and training data evolves, benchmark ladders, gameability ratings, and source selection may need revision. This methodology is a living document.

High-End Calibration

The expected-score ladders intentionally get more demanding above IQ 140 on many benchmarks. Each additional raw point often contributes less to implied IQ in the superhuman range than in the human range. This reflects three realities:

1. Human IQ distributions thin out at the tails. The difference between IQ 100 and IQ 120 is much more common than the difference between IQ 140 and IQ 160.
2. Superhuman benchmark scores are driven by breadth, not depth. A model scoring 50% on FrontierMath T4 isn't twice as smart as one scoring 25% — it covers more mathematical branches rather than being fundamentally more capable in any single branch.
3. Practical discrimination. Without demanding high-end ladders, reasoning vs. non-reasoning configurations of the same model can produce unrealistically large IQ gaps. The ladder shape keeps those differences meaningful without letting one saturated benchmark dominate the composite.

A perfect or near-perfect score can still imply IQ 160 when that is a reasonable interpretation of the source. In the current calibration, several mature or contamination-sensitive benchmarks instead require off-scale performance for IQ 145 or 160. The point is not to cap hard benchmarks forever, but to make the high-end requirements explicit enough that future calibration passes can decide when a sudden 100% score on a very hard benchmark should map to an appropriately high implied IQ.

Data Process

AI IQ keeps source-backed data, extracted updates, and derived scoring separate. That separation matters because a raw benchmark chart should show what a public source actually reported, while the composite IQ can use conservative scoring-only imputations to avoid rewarding missing coverage.

Source-backed benchmark data can exist for models that are not yet shown on public chart surfaces. A model must have launch metadata, must not be hidden, and must have a derived IQ before it appears in the main IQ charts. This keeps placeholder rows inspectable in the data table without letting a single benchmark import promote them into public trend charts.

Manual source captures that may be hard to reproduce exactly are archived so the same raw data can be re-parsed later if the extraction rules improve. Larger generated scrapes can be refreshed from the original source and do not need to be treated as permanent public evidence in the same way.

Model Tiers

The tier filter on charts groups models into six levels: Flagship, Secondary, Tertiary, Compact, Ultralight, and Coding. Tiers describe where a model sits within its own lab's lineup, not how it scores against other labs' models — a lab's best current offering is always Flagship, and the levels below follow that lab's own ladder down to its smallest models. Coding-specialist releases sit in their own filter bucket. Open-weight releases count as part of the same lab's lineup. This keeps tier and measured intelligence independent: filtering to Flagship compares each lab's best model head-to-head, and the scores — not the tier — show how those bets actually perform. Because tiers are lab-relative, a Tertiary model from one lab can outscore a Flagship model from another; that gap is a finding, not an error. Not every lab fills every level, and tier assignments change only when a lab restructures its lineup, never when new benchmark results arrive.

Chart Inclusion

AI IQ separates model data from chart display policy. A model can exist in the dataset, have source-backed benchmark rows, and still be absent from a public chart if it does not meet that chart's policy or required fields.

Most public chart surfaces use a default policy based on publication status, derived IQ availability, model type, provider tier, and generation recency. Each major lab's main model line keeps several previous generations on charts; other top-tier models keep one previous generation; lower-tier variants such as mini, nano, Sonnet, Haiku, Flash, and smaller open-weight sizes are included only for the current generation. Archive and hidden rows are excluded unless explicitly overridden.

The IQ Over Time chart uses a stricter frontier-timeline policy. It requires a release date, derived IQ, a public model row, a general-purpose model type, membership in a top-lab provider grouping, and either top-tier status or membership in one of the lab's recognized main model lines — the latter keeps a lab's historical flagship line (for example Opus) on the timeline after the lab restructures its lineup above it. Provider lines then connect non-decreasing IQ checkpoints rather than every model from that provider.

The policy fields are maintained in the admin dashboard: publication status, model type, provider tier, model line, generation offset, and display override. Provider tier follows each lab's own ladder as described under Model Tiers; lineup names such as OpenAI mini/nano, Anthropic Sonnet/Haiku, Google Flash, and Qwen size tiers map onto that ladder per lab rather than acting as generic cross-provider role names.

Sources

Benchmark scores, prices, and token usage come from publicly published leaderboards. Each source is sampled periodically and reconciled against published numbers before being applied.

• Artificial Analysis Intelligence Index — the primary aggregator. Provides scores for AIME, GPQA Diamond, SWE-Bench Verified, HLE, SciCode, Terminal-Bench 2.1, Terminal-Bench Hard, CritPt, LiveCodeBench, IFBench, MMMU-Pro, and the AA composite indices (Omniscience, GDPval, τ₂-Bench Telecom, Long Chain Reasoning), plus per-model pricing, response time, median throughput, total evaluation cost for the AA suite, and token-usage data used for task efficiency.
• Arena.ai Overall — Arena.ai's head-to-head text Elo ratings and ranks
• Arena.ai WebDev — pairwise web-app development evaluations using Bradley-Terry ratings
• DesignArena Full Stack — human-preference Elo for end-to-end frontend + backend builds
• DesignArena Code Categories — broad code-category Elo tracked as a raw benchmark pending calibration
• ARC Prize leaderboard — ARC-AGI-1, ARC-AGI-2, ARC-AGI-3 scores, and ARC-AGI per-task cost where published
• Vals.ai — the Vals Index, AIME, ProofBench, SWE-Bench, and LiveCodeBench source views where available
• Vals.ai LegalBench, Legal Research Bench, and Harvey Legal Agent Benchmark — the three equally weighted Legal-domain source views
• Scale Labs SWE-Bench Pro Public Dataset — public SWE-Bench Pro Resolve Rate source
• SWE Marathon — long-horizon software-engineering Resolution Rate (Pass@1), using best published agent/harness rows for canonical model-level scoring
• Cognition FrontierCode — FrontierCode Diamond score for high-quality production-code tasks, using canonical public model rows where the source model identity is clear
• Mercor APEX-SWE — professional software-engineering overall Pass@1 scores
• SWE-rebench — continuously evolving and decontaminated software-engineering leaderboard rows
• MathArena — model-level expected performance across non-deprecated math competitions
• DeepSWE v1.1 — Datacurve's live leaderboard for original long-horizon software-engineering tasks
• GSO-Bench — software optimization Opt@1 leaderboard using OpenHands scaffold rows for canonical model-level scoring
• SWE-Bench — SWE-Bench Verified leaderboard rows, using clear single-model agent/model pairs for model-level scoring
• LiveCodeBench — continuously refreshed coding-problem benchmark sourced from live contests
• Terminal-Bench — Terminal-Bench 2.1 and Terminal-Bench Hard task accuracy
• BrowseComp — hard browsing benchmark with published human-trainer performance used for calibration
• Humanity's Last Exam — benchmark semantics for the HLE calibration caveat
• CritPt paper — research-physics benchmark semantics for the CritPt calibration caveat
• SciCode — scientific coding benchmark results
• Agents' Last Exam — agent/harness leaderboard for real-world professional workflows, using Full / Overall pass rate for canonical model-level scoring
• OSWorld-Verified and Toolathlon — sparse computer-use and tool-use signals; current score rows use clear mirror/self-reported mappings, with official sources used for benchmark semantics and exact-match validation
• MCP Atlas — real MCP-server tool-use tasks with claim-level scoring
• IFBench, AA Omniscience, and AA Long Chain Reasoning — instruction-following, factual-reliability, and long-document reasoning signals used for the Reliability dimension
• BullshitBench v2 — a pushback / anti-sycophancy signal (whether a model challenges false premises) used for the Reliability dimension
• FACTS Grounding — Google's source-grounded long-form factuality benchmark, using the Score column from a Kaggle FACTS Grounding leaderboard capture for the Reliability dimension
• Epoch AI — FrontierMath Tier 1–3 and Tier 4 accuracy
• EQ-Bench 3 — emotional-intelligence Elo
• AttuneBench — emotional-attunement benchmark from real human-AI conversations

The Artificial Analysis Intelligence Index can list two rows for the same model under one display name when the same underlying model has both a reasoning and a non-reasoning configuration (the reasoning row is marked with a 💡 lightbulb icon). When the two configurations differ meaningfully on cost, latency, or quality, they are tracked as separate model entries (e.g. reasoning vs non-reasoning variants of the same release).