Methodology

AI IQ assigns each model an estimated IQ score by evaluating performance across 5 cognitive dimensions, each measured by multiple benchmarks. Hard, ungameable benchmarks retain full IQ curves, while easier or gameable benchmarks have compressed ceilings that limit their influence. Missing benchmarks and dimensions are conservatively imputed inside the scoring pipeline, and every derived composite IQ averages all five dimension scores.

This page documents the full scoring system: how source data is captured, how the five dimensions are defined, how raw scores map to IQ via piecewise-linear interpolation, how benchmark ceilings are compressed for gameability, and how missing values are imputed without changing the source-backed benchmark table.

The 5-Dimension Framework

AI IQ organizes evaluation into five cognitive dimensions. Each dimension uses multiple benchmarks that are averaged together, with missing benchmarks conservatively imputed:

• Hard benchmarks are frontier-discriminating tests with low gameability. They retain full IQ curves with ceilings of 143–158 and can differentiate between the strongest models.
• Compressed benchmarks are easier or more gameable tests. Their anchor curves are compressed to lower ceilings (128–140), limiting how much a high score on a gameable benchmark can inflate the composite.

The composite IQ requires at least 2 of 5 dimensions to have data. Models with fewer scored dimensions do not receive a derived IQ and are skipped by IQ-ranked chart surfaces.

Formulas

Each benchmark raw score \(s\) is converted to an IQ value via piecewise-linear interpolation over that benchmark's anchor points \(\mathbf{A} = [(s_0, a_0),\, (s_1, a_1), \ldots]\):

Each dimension averages the IQ values of its benchmarks. Missing benchmarks are conservatively imputed before averaging using the benchmark-level waterfall below (see Benchmark-Level Imputation).

The five dimensions:

Superscripts denote compressed ceilings. \(f\) is piecewise-linear interpolation over each benchmark's anchor curve.

The composite IQ is the mean of all five dimension scores. At least 2 dimensions must be source-backed or predecessor-imputed before any missing whole dimensions are filled:

D1: Fluid Abstraction

Fluid abstraction is the ability to solve novel problems without relying on prior knowledge. This is the closest analogue to fluid intelligence in human psychometrics — raw problem-solving ability applied to patterns never seen before.

D2: Mathematical Reasoning

Mathematical reasoning and quantitative problem-solving — the ability to work with mathematical structures, proofs, and analytical frameworks. The hard benchmarks test novel quantitative reasoning that cannot be memorized from training data.

D3: Programmatic Reasoning

Practical engineering ability — the capacity to solve real-world technical problems in code and software systems. Programmatic Reasoning focuses on code repair, implementation, and algorithmic coding, while scientific research problems that use code as an execution medium live in Critical Reasoning.

D4: Critical Reasoning

Critical reasoning captures expert judgment, scientific literacy, and difficult problem analysis under uncertainty. The hard benchmarks test whether a model can reason through questions that push the boundaries of human expertise itself, while the compressed benchmark tests graduate-level science knowledge.

D5: Agentic Reasoning

Agentic reasoning measures practical task execution across environments: navigating tools, using context, taking multi-step actions, and recovering enough from intermediate state to finish the task.

Piecewise-Linear Interpolation

Each benchmark defines a set of anchor points mapping raw scores to IQ values. For scores that fall between two anchors, we use piecewise-linear interpolation:

If the score is at or below the lowest anchor, the model receives that anchor's IQ. If at or above the highest, it receives the ceiling IQ. There is no extrapolation beyond the defined range.

This approach avoids assumptions about the distribution shape between anchors. Each segment can have a different slope, allowing the curve to be steeper where small score improvements represent large cognitive leaps (e.g., going from 0% to 5% on HLE) and flatter where additional points reflect diminishing differentiation.

Benchmark Averaging & Compression

Each dimension averages all its benchmarks together, with missing benchmarks conservatively imputed. Rather than separating benchmarks into primary/fallback tiers with a hard cap, we use compressed anchor curves to limit the influence of easier or gameable benchmarks.

How Compression Works

For compressed benchmarks, the anchor curve is rescaled so that IQ values above 100 are proportionally reduced toward a lower ceiling:

Values at or below IQ 100 are unchanged. This preserves the low-end of the curve (where models genuinely struggle) while compressing the high-end where gameable benchmarks over-reward.

Benchmark-Level Imputation

When a model has scores for some but not all of a dimension's benchmarks, the missing benchmarks are filled in before the dimension IQ is averaged. We use two ingredients:

• The model's available-benchmark IQ average — how the model is performing on the benchmarks it does have in this dimension. This is the within-dimension signal: if a model is hitting IQ 130 on the dimension's other benchmarks, the missing one is probably also somewhere around 130.
• The benchmark's 80th-percentile IQ (\(P_{80}\)) — a per-benchmark ceiling derived from the actual data. Take every model that has a real score on that benchmark, convert each score to an implied IQ via the anchor curve, sort those IQs from low to high, and take the value at the 80th-percentile rank. So if 50 models have HLE scores yielding implied IQs ranging from 70 to 155, \(P_{80}(\text{HLE})\) is the implied IQ at the 80th-percentile rank in that sorted list. It is where strong-but-not-frontier measured models actually land on this benchmark.

The imputed value is the minimum of the two:

Benchmark Imputation Waterfall

For ARC-AGI-2 and CritPt, missing values are treated as zero in the scoring pipeline because historical frontier models generally scored at or near zero until directly shown otherwise. For models released before April 2025, the same scoring-only zero assumption is also applied to FrontierMath T4, ProofBench, and Terminal-Bench 2.0. The raw benchmark table still leaves those values blank when no source row exists. Other missing benchmarks use the ordinary imputation waterfall.

Predecessor imputation is constrained by model family, and non-reasoning variants do not impute from reasoning variants. These lineage choices are scoring assumptions only; they do not create source-backed raw benchmark values.

Imputation only fires inside a dimension that has at least one real or hard-zero benchmark. If a dimension has no usable benchmark value for a given model, no benchmark-level imputation runs there — the dimension itself is either left missing or filled at the dimension level (see Composite IQ Calculation below).

Composite IQ Calculation

Step 1: Score Each Dimension

For every dimension where the model has at least one real benchmark, compute the dimension IQ as the average of its benchmarks. If some benchmarks within the dimension are missing, fill them via the conservative imputation above before averaging.

Step 2: Dimension-Level Imputation

If a model has at least 2 scored dimensions but is missing some of the others, every missing dimension is imputed before the composite is averaged. The cap is matched to models with real data for that dimension and similar capability across the other dimensions:

where \(\overline{\mathrm{IQ}}_{\text{scored dims}}\) is the model's average IQ across the dimensions it does have. For the cap, we look at models released on or before the scored model that have real data on the missing dimension, compare their average IQ across the other dimensions, and use the lower-quartile missing-dimension IQ among the closest comparable models. If the comparable set is too thin, we fall back to the same-era lower quartile for that dimension. If no same-era real data exists for a dimension, we do not invent a neutral default; the model does not receive a derived all-dimension IQ.

In practice, comparable models are those whose average across the non-missing dimensions is within a small IQ band of the model being scored and whose release date is not later than the scored model. If fewer than three comparable models are available, we use the nearest same-era measured models instead. This keeps missing dimensions conservative without letting older models borrow strength from future benchmark cohorts.

All five dimensions are always used for derived IQ. Missing a hard dimension such as Fluid Abstraction should not improve a model's score, so missing dimensions are filled conservatively rather than omitted from the average.

Dimension Imputation Waterfall

Step 3: Compute the Composite

where all five dimensions are used once missing dimensions are imputed.

Key rules:

• Minimum 2 dimensions required. Models with fewer than 2 scored dimensions do not receive a derived composite IQ.
• No omitted dimensions. Models with enough coverage for derived IQ always use a 5-dimension composite; missing dimensions are conservatively imputed.
• Transparent count. The display shows X/5 so readers can see how many dimensions had source-backed data before dimension-level imputation.
• Equal weighting. All dimensions contribute equally. Compressed ceilings (not differential weighting) handle benchmark quality differences.

Rank Status

Each model receives a rank status reflecting the completeness of its evaluation:

• Full — All 5 dimensions scored. The most reliable composite.
• Partial — 2–4 dimensions scored. Composite is derived but based on incomplete coverage.
• Provisional — Only 1 dimension scored. Not enough for a derived composite.
• Unranked — No dimension data available.

Tracked Benchmarks & Exclusions

Some benchmarks are tracked in the models table or shown in standalone charts but are not part of the composite IQ calculation:

• MMLU-Pro — A 10-choice multiple-choice knowledge test. Overlaps with the Critical Reasoning dimension (GPQA/HLE) and adds limited discrimination at the frontier. Models have converged to similar high scores.
• MMMU-Pro — Multimodal academic questions. While the vision component is interesting, most frontier model evaluation focuses on text-based reasoning. This benchmark is tracked in the data but excluded from the IQ composite.

These benchmarks remain in the database and are viewable on the models page — they are simply not included in the composite IQ computation.

Core Math Benchmarks

FrontierMath Tier 1–3 and ProofBench are core inputs to the D2 (Mathematical Reasoning) dimension and are surfaced on the IQ page as standalone cost-scatters.

• FrontierMath Tier 1–3 (ceiling 152) — harder than AIME, easier than T4. Top general models score ~50%. Slots between AIME and T4 in difficulty and gives more discrimination in the middle of the math distribution where T4 is too sparse.
• ProofBench (ceiling 158) — formally-verified proof writing. A different cognitive task than the problem-solving benchmarks (you have to construct a verified proof, not just give an answer). Top general models ~56%; specialized math models ~71%.

EQ Scoring

AI IQ estimates an Emotional Quotient (EQ) for each model from three interaction-quality signals: Arena Elo (broad conversational quality as ranked by human-preference voting), IFBench (instruction-following and constraint adherence), and EQ-Bench 3 Elo (AI-judged emotional/social reasoning in challenging roleplays). EQ-Bench 3 is retained because it is the strongest recent dedicated emotional/social reasoning signal, but it is style-sensitive and judged by Claude, so it is treated as one component rather than as neutral ground truth. Each source score is mapped to an implied EQ via a hand-calibrated anchor curve, then the available components are averaged when at least two source-backed components are present.

If only one source is available, the row remains eligible for that component benchmark chart, but it does not receive a composite EQ. This keeps single-benchmark coverage from outranking models with broader interaction-quality evidence.

EQ-Bench 3 Elo → EQ

EQ-Bench 3 produces Elo ratings from head-to-head emotional-roleplay matchups judged by Claude. This makes it valuable as a dedicated affective/social reasoning signal, but also sensitive to Claude-like response style. The Elo range observed in production runs roughly from 200 (very weak) to 2000 (top frontier). The mapping:

Arena Elo → EQ

LM Arena Elo reflects broad conversational quality as judged by human voters in head-to-head matchups. The observed Elo range is tighter (~1100–1520), so the anchor curve is calibrated separately:

IFBench → EQ

IFBench captures whether the model listens to instructions and respects user constraints. This is not emotional intelligence in the narrow sense, but it is part of practical interaction quality.

EQ-Bench Style-Sensitivity Adjustment

Because EQ-Bench 3 is judged by Claude, it can favor Claude-like response style and penalize models that solve emotional/social scenarios in a substantially different voice. To reduce that family/style bias while preserving EQ-Bench as a useful dedicated signal, we subtract a 100-point Elo adjustment from the EQ-Bench component for Anthropic models before mapping to implied EQ. Arena and IFBench are unaffected.

Cost & Speed Metrics

Sticker Price — published price for a typical workload

AI IQ's effective-cost views are anchored to a 2:1 input-to-output token mix — a deliberately input-heavy workload that reflects most real applications (RAG, long-context reasoning, agent loops). Sticker Price is the dollar amount to process 2M input tokens and generate 1M output tokens at a model's published rates:

where \(p_{\text{in}}\) and \(p_{\text{out}}\) are the published per-million-token prices in dollars.

Task Efficiency — how much work does the model use?

Sticker price alone hides large per-task differences in how much work a model uses to solve a benchmark. We estimate this with a blended usage multiplier. For each benchmark, AI IQ first estimates the task cost expected from a model's published input and output prices. The usage signal is the residual: actual task cost divided by expected task cost. Direct token-usage data is included as an additional signal where available.

When only one channel is available, AI IQ uses that channel directly. The Task Efficiency chart shows the inverse of the usage multiplier, so 2× means the model uses about half the task effort of the median model, and 0.5× means it uses about twice as much.

Effective Cost — what it actually costs to do the same task

The product of the two:

Reads as: what this model spends on a task after adjusting its published price by observed and price-adjusted benchmark usage. Models below the diagonal (Effective Cost < Sticker Price) are task-efficient and cheaper than their sticker suggests; models above are task-hungry. This is the cost axis on every effective-cost-vs-quality chart.

Response Time

Response time is the median seconds to a complete answer (lower is better), shown on a logarithmic scale. The IQ vs Response Time chart reverses the X axis so the upper-right corner represents the ideal — high intelligence at low latency.

Limitations & Transparency

• Dimension coverage varies. Some models have data for all 5 dimensions; others have as few as 2 (with the rest imputed). A model's composite IQ is most reliable when all dimensions are scored. Always check the X/5 count and rank status.
• Benchmark mix matters. Two models with the same composite IQ may have very different underlying data quality. One might have all hard benchmarks (ungameable tests with full curves) while another relies mostly on compressed benchmarks (with lower ceilings). The rank status and dimension count help distinguish these cases.
• Imputation is conservative, not clairvoyant. Missing values are filled first from explicit direct-predecessor lineage when available, then from within-dimension benchmark evidence, then from comparable measured models at the dimension level. These are reasonable estimates, not ground truth — a model's true ability on an unevaluated benchmark could be significantly higher or lower.
• Anchor calibration is subjective. The mapping from raw scores to IQ involves judgment calls about what different performance levels mean relative to human cognitive ability. We document our rationale for each benchmark, but reasonable people can disagree.
• IQ is a metaphor. Human IQ tests measure a specific construct via standardized instruments under controlled conditions. AI benchmark performance is a different thing. The IQ scale provides an intuitive frame of reference, not a claim of equivalence.
• Compressed ceilings are a design choice. The ceiling values directly affect which models benefit and which are penalized. Models that excel on compressed benchmarks will have their contributions limited, which may feel unfair if those benchmarks genuinely reflect high ability. We believe the trade-off — rewarding harder evaluation — is correct, but the specific ceiling values are judgment calls.
• Benchmarks become stale. As models improve and training data evolves, benchmark ceilings, gameability ratings, and compression levels may need revision. This methodology is a living document.

Asymptotic Compression Above IQ 140

The anchor point curves intentionally compress above IQ 140. Each additional percentage point on a benchmark contributes less to the IQ score in the superhuman range than in the human range. This reflects three realities:

1. Human IQ distributions compress at the tails. The difference between IQ 100 and IQ 120 is much more common than the difference between IQ 140 and IQ 160.
2. Superhuman benchmark scores are driven by breadth, not depth. A model scoring 50% on FrontierMath T4 isn't twice as smart as one scoring 25% — it covers more mathematical branches rather than being fundamentally more capable in any single branch.
3. Practical discrimination. Without compression, reasoning vs. non-reasoning configurations of the same model produce 20+ point IQ gaps, which is unrealistic. With compression, the gap narrows to ~10-12 points (“smart” vs. “very smart” rather than “above average” vs. “genius”).

The compression ensures that no benchmark can single-handedly produce IQ values above ~155, regardless of raw score. The theoretical ceiling of the composite is approximately 150–155 under current benchmarks.

Data Process

AI IQ keeps source-backed data, extracted updates, and derived scoring separate. That separation matters because a raw benchmark chart should show what a public source actually reported, while the composite IQ can use conservative scoring-only imputations to avoid rewarding missing coverage.

Source-backed benchmark data can exist for models that are not yet shown on public chart surfaces. A model must have launch metadata, must not be hidden, and must have a derived IQ before it appears in the main IQ charts. This keeps placeholder rows inspectable in the data table without letting a single benchmark import promote them into public trend charts.

Manual source captures that may be hard to reproduce exactly are archived so the same raw data can be re-parsed later if the extraction rules improve. Larger generated scrapes can be refreshed from the original source and do not need to be treated as permanent public evidence in the same way.

Chart Inclusion

AI IQ separates model data from chart display policy. A model can exist in the dataset, have source-backed benchmark rows, and still be absent from a public chart if it does not meet that chart's policy or required fields.

Most public chart surfaces use a default policy based on publication status, derived IQ availability, model type, provider-specific tier, and generation recency. Current and previous generations of provider-tier 0 or higher models are included; lower-tier variants such as mini, nano, Sonnet, Haiku, Flash, and smaller open-weight sizes are included only for the current generation. Archive and hidden rows are excluded unless explicitly overridden.

The IQ Over Time chart uses a stricter frontier-timeline policy. It requires a release date, derived IQ, a public model row, a general-purpose model type, provider tier 0 or higher, and membership in a top-lab provider grouping. Provider lines then connect non-decreasing IQ checkpoints rather than every model from that provider.

The policy fields are maintained in the admin dashboard: publication status, model type, provider tier, model line, generation offset, and display override. Provider tier is provider-specific; for example OpenAI mini/nano, Anthropic Sonnet/Haiku, Google Flash, NVIDIA Nano, and Qwen size tiers are not treated as generic cross-provider role names.

Sources

Benchmark scores, prices, and token usage come from publicly published leaderboards. Each source is sampled periodically and reconciled against published numbers before being applied.

• Artificial Analysis Intelligence Index — the primary aggregator. Provides scores for AIME, GPQA Diamond, SWE-Bench Verified, HLE, SciCode, Terminal-Bench 2.0, CritPt, LiveCodeBench, IFBench, MMMU-Pro, and the AA composite indices (Omniscience, GDPval, τ₂-Bench Telecom, LCR), plus per-model pricing, response time, median throughput, total evaluation cost for the AA suite, and token-usage data used for task efficiency.
• LM Arena — head-to-head Elo ratings and ranks
• ARC Prize leaderboard — ARC-AGI-1 and ARC-AGI-2 scores and per-task cost; ARC-AGI-3 is tracked in the admin/models-table pipeline but is not yet used in IQ scoring
• Vals.ai — the Vals Index, AIME, ProofBench, SWE-Bench, SWE-Bench Pro, and LiveCodeBench source views where available
• SWE-Bench — SWE-Bench Verified leaderboard rows, using clear single-model agent/model pairs for model-level scoring
• Terminal-Bench — Terminal-Bench 2.0 and Terminal-Bench Hard task accuracy
• SciCode — scientific coding benchmark results
• BrowseComp, OSWorld-Verified, and Toolathlon — curated source captures used for the Agentic Reasoning dimension
• Epoch AI — FrontierMath Tier 1–3 and Tier 4 accuracy
• EQ-Bench 3 — emotional-intelligence Elo

The Artificial Analysis Intelligence Index can list two rows for the same model under one display name when the same underlying model has both a reasoning and a non-reasoning configuration (the reasoning row is marked with a 💡 lightbulb icon). When the two configurations differ meaningfully on cost, latency, or quality, they are tracked as separate model entries (e.g. reasoning vs non-reasoning variants of the same release).