IQ vs Latency

Latency is time to first token (TTFT): how long a model takes to begin responding. Drag the input-length slider from 1K to 10K tokens to interpolate between AA's two measured TTFT anchors.

IQ vs Latency
Latency is time to first token (TTFT): how long a model takes to begin responding. Drag the input-length slider from 1K to 10K tokens to interpolate between AA's two measured TTFT anchors.
Controls:
Input10K tok
IQ1:1Latency

How to read this chart

Latency is time to first token (TTFT): how long a model takes to begin responding. Drag the input-length slider from 1K to 10K tokens to interpolate between AA's two measured TTFT anchors.