LLM Usage Metrics: Token Consumption, Measurement Baselines, and How to Read Usage Trends Correctly
A practical guide to interpreting large language model usage data. It breaks down the token types behind a single call (input, output, reasoning, cached), explains why production traffic, offline evaluation, and distribution counts measure three different things, and shows why usage figures from different platforms cannot be added together. It also covers day-over-day comparison, 7-day moving averages, baseline backfill, and why pricing and rate limits are not usage data — ending with a checklist teams can apply when reporting or benchmarking model usage.
Why usage numbers are so often misread
When people assess the real impact of a large language model, metrics such as call volume, token consumption, and request counts are often compared directly. A common trap is to place numbers from different sources and different measurement baselines into the same table — or even add them together. To interpret these figures correctly, you first need to know what each one represents, under which conditions it was produced, and which metrics are fundamentally incomparable.
Usage data misleads for three main reasons:
- Inconsistent measurement baselines. Some figures are per day, some are cumulative, some are given only as ranges.
- Mixed data types. Production traffic, offline evaluation, and software distribution downloads get lumped together.
- Different time scopes. Real-time data, next-day backfills, and monthly aggregates are used interchangeably.
The sections below unpack each layer.
1. Start with the token breakdown of a single call
To understand usage, you first need to understand how tokens are composed. In a typical large model call, billing and metering usually involve these categories:
- Input tokens (prompt): the context a user or system sends to the model, including prompts, conversation history, and retrieved reference material. In retrieval-augmented or long-context scenarios, input tokens often make up the majority of the total.
- Output tokens (completion): the content the model generates. This is usually the most expensive per token.
- Reasoning tokens: for models with chain-of-thought capability, the intermediate reasoning process may be metered separately, and it can affect both cost and latency reporting.
- Cached tokens: when a large amount of repeated context hits the cache, these tokens are often billed at a significantly lower rate. The cache hit rate directly determines the real cost of long-context applications.
Abstractly, the usage of a single request can be modeled as follows:
from dataclasses import dataclass@dataclassclass TokenUsage:prompt_tokens: int # inputcompletion_tokens: int # outputreasoning_tokens: int # reasoning (optional)cached_tokens: int # cached portion of the input tokens@propertydef billable_input(self) -> int:# cached tokens are usually priced separately at a lower ratereturn self.prompt_tokens - self.cached_tokens@propertydef total(self) -> int:return self.prompt_tokens + self.completion_tokens + self.reasoning_tokens
The key insight: within a seemingly large token total, most of it may be input tokens served from cache at a low price. Looking at the total alone can therefore significantly overestimate both real load and real cost. Any sound analysis has to distinguish the billing view from the physical view.
2. Three categories that must be kept apart: production, evaluation, distribution
This is where usage interpretation most often goes wrong. The numbers may all be called “usage,” “tokens,” or “downloads,” but they measure fundamentally different things.
1. Production usage (real production calls)
This is the traffic an application actually generates in a production environment, and it best reflects real demand. It is usually reported along dimensions such as request counts, token consumption, active users, and session counts. The key checks for production data are whether it keeps updating over time and whether it carries a breakdown (input / output / reasoning / cache). Reliable production data should have a clearly defined measurement period, a traceable historical baseline, and a time series with daily granularity.
2. Evaluation usage (offline benchmarks)
Evaluation data comes from standardized benchmark sets and measures how a model performs on specific tasks — for example, the number of evaluation tokens consumed under a given budget. It measures how much was tested, not how much was used. Putting evaluation tokens next to production tokens in the same comparison is a classic conceptual confusion: evaluation data reflects the boundaries of model capability, not market demand.
3. Distribution volume (downloads, pulls, installs)
Package download counts, image pull counts, and model weight download counts measure the breadth of a distribution channel. They reflect how many environments have obtained the model, not how many times it has been called. A model may have hundreds of thousands of downloads while most of those environments never issue a production call; conversely, a hosted cloud service may publish no download numbers at all while serving enormous online request volumes. There is no stable conversion factor between distribution volume and production volume.
In one sentence: production volume shows demand, evaluation volume shows capability, and distribution volume shows reach. None can substitute for another, and they must never be combined.
3. Why you cannot sum across platforms
A frequent mistake is adding up token figures from several platforms to produce a single “total usage.” This does not hold up methodologically, for three reasons:
- Overlapping traffic. The same users and applications may go through multiple entry points at the same time — for example, calling through an aggregation gateway and also directly through a local tool — so summing across platforms double-counts.
- Inconsistent baselines. One source may report “new per day” while another reports “cumulative total.” Adding them mixes flow with stock.
- Incomparable units. Some sources report request counts, some report tokens, some report downloads. Different units simply cannot be added.
The correct approach is to first build a comparable time series within a single source, and then perform trend analysis on it. Across sources, only qualitative comparison is defensible (for example, “one source is in the tens-of-billions daily range, another in the hundreds-of-billions range”) — not quantitative summation.
4. Reading traffic trends: day-over-day change and moving averages
A single day’s usage number fluctuates a lot, and drawing conclusions from one day alone is easy to get wrong. A more robust approach introduces the time dimension:
- Day-over-day change: compare two adjacent days to observe short-term movement. For example, a value of 434.8B against 573.9B the previous day is roughly a 24% drop, indicating a pullback in traffic that day — but a single day is not enough to judge the long-term trend.
- 7-day moving average: smoothing daily noise with the mean of the last seven days better represents the “current level” than any single day. When the 7-day average sits at about 413B per day, then even if one day spikes to 574B or falls to 300B, the average should remain the reference point.
- Baseline backfill: when real-time data is missing, retain the most recent reliable baseline and label it explicitly rather than leaving it blank or guessing.
Expressing the moving average and day-over-day change in code:
def summarize(series: dict[str, float]) -> dict[str, float]:"""series: {date: tokens consumed that day}; units must be normalized first"""dates = sorted(series)latest = series[dates[-1]]prev = series[dates[-2]]dod = (latest - prev) / prev # day-over-day changelast7 = [series[d] for d in dates[-7:] if d in series]ma7 = sum(last7) / len(last7) # 7-day averagereturn {"latest": latest, "dod_pct": dod, "ma7": ma7}
One point worth stressing: when describing the scale of a given source, use the 7-day average or a stable baseline and state it as “on the order of X,” rather than picking a peak day or a trough day as your evidence.
5. Price and rate limits are not usage
Another common misunderstanding is treating a platform’s price list (input and output price per million tokens, cache price) or its rate limit thresholds (requests per minute, tokens per minute) as usage data.
- Price is a cost parameter. It determines how much each unit of calls costs, and says nothing about how much was actually called.
- Rate limits (RPM/TPM) are a system capacity ceiling — the maximum you can call, not what you did call.
Even if a platform publishes a detailed price list and rate limit configuration, you cannot infer its market share or load from them without public real-time call volumes. When analyzing usage, keep price, rate limits, and usage in strictly separate columns so they do not contaminate each other.
6. Practical recommendations for teams
Pulling the above together, here is a methodology worth following when dealing with large model usage data:
- Classify first, compare second. Assign every number to one category — production, evaluation, distribution, price, or rate limit. Only compare within a category.
- Prefer time series. Build a time series with daily granularity and a consistent baseline, and use the 7-day moving average as the reference level.
- Decompose token structure. Separate input, output, reasoning, and cache, and pay particular attention to how the cache hit rate affects cost and load.
- Never sum across platforms. Compare scale qualitatively across platforms; do not add quantities.
- Label data freshness. When real-time data is missing, retain and label the most recent reliable baseline instead of inventing one.
- Back it with observability. On the production side, use logging services, monitoring and alerting, and metrics collection to build your own usage dashboard. On Baidu AI Cloud, for instance, general-purpose capabilities such as the log service, monitoring and alerting, and object storage can be used to accumulate the token structure, latency, and cost of each call into traceable metrics, forming a consistent internal usage baseline.
Conclusion
The real value of large model usage data lies not in some isolated, impressive-looking number, but in clear baselines, decomposable structure, and explainable trends. Separate the five categories — production, evaluation, distribution, price, and rate limits — replace single-point snapshots with time series, and decompose token composition instead of looking only at totals. Only then can you make a rigorous judgment about the true impact of a model or a class of service. That is also the key step in moving from “looking at numbers” to “reading data.”
