LLM API Call Volume Weekly Trend Tracking: Metrics, Attribution, and Alerting
A practical guide to building a weekly trend-tracking process for LLM API call volume. It defines a core metric set (total tokens, text requests, average tokens per request, week-over-week growth), explains the three statistical calibers (routing alias, backend version, and merged old/new), and covers gateway-side instrumentation backed by a unified call-log schema. It then walks through weekly aggregation and week-over-week SQL, three-layer attribution (volume-price decomposition, structural effects, scenario drill-down, and trend fitting), MAD-based anomaly detection with tiered alerts, and how weekly data drives cost, capacity, and quality decisions. Python and SQL snippets illustrate each stage.
1. Why LLM Call Volume Needs Weekly Observation
LLM applications iterate on a weekly cadence. New releases, prompt changes, routing-strategy updates, and cache-strategy switches mostly roll out week by week, sometimes day by day. Call volume follows a similarly clear cyclical pattern:
- Total volume can grow several times over within a few months, shifting from “ten-thousand requests” to the “trillion-token” scale;
- Old and new versions of the same model coexist, with traffic migrating between them;
- Scenario launches, feature canaries, and marketing campaigns produce distinct pulse-like spikes.
Daily reports are too granular and get distorted by weekday effects and random noise. Monthly reports are too lagging, often surfacing a problem or opportunity two or three weeks after it appears. Weekly trend tracking sits between the two: it smooths intraday noise while aligning with the cadence of releases and capacity expansion, which makes it a natural baseline rhythm for call-volume governance.
Public monitoring data illustrates this (illustrative): total tokens grew about 9% week over week (WoW) in a given week, with the new version ramp being the main source of the increase. Behind any total-volume change, however, two kinds of change usually coexist:
- Total-volume change: the call scale itself is increasing or decreasing;
- Structural change: traffic is being reallocated across models, versions, and business scenarios.
Looking at the total alone, it is easy to mistake a “structural shift” for “real growth.” Take one model as an example: its old and new versions together account for about 27% of the total, growing about 35%. Looking at a single version could lead to the opposite conclusion. So the goal of weekly tracking is not only to answer “how much did it grow,” but also “where did the growth come from,” “who is driving it,” and “how long can it last.”
2. Metric System and Caliber Design
2.1 Four Base Metrics
| Metric | Definition | Primary use |
|---|---|---|
| Total tokens | Sum of input, output, and reasoning tokens over the period, without cross-version deduplication | Cost estimation, capacity planning |
| Text request count | Requests that returned successfully and produced valid generation; failures and empty responses excluded | Measuring real usage activity |
| Average tokens per request | Total tokens ÷ text request count | Identifying structural changes such as longer context or longer output |
| Week-over-week growth rate | (This week − last week) ÷ last week | Describing trend direction and slope |
Among these, restricting “text request count” to text requests is deliberate—it excludes other modalities such as image, audio, and video so that numbers across weeks stay comparable. Average tokens per request is a frequently overlooked bridge metric: when total volume grows, it reveals whether the growth comes from “more people calling” or “each call consuming more.”
The illustrative data reads like this:
- Total tokens grew about +9% WoW while request count grew about +4.7% WoW. Total volume is growing noticeably faster than request count, which shows that growth comes not only from call frequency but also from increased per-call consumption—longer context or larger output are likely causes.
- The difference between the two growth rates is roughly the growth rate of average tokens per request (about +4.1%). The relationship between the ratios can be approximated as ΔT% ≈ ΔQ% + ΔS%, where T is total tokens, Q is request count, and S is average tokens per request.
2.2 Three Statistical Calibers
For the same set of calls, different calibers can lead to very different conclusions:
- Routing alias caliber: counts by the model name or routing identifier the business requested. When a new version is first integrated, the alias is often already switched, but that version’s tokens are backfilled only after the statistical period ends, causing the current week to look low and the next week to spike as it catches up. Some platforms’ dashboards use this caliber by default.
- Backend actual-version caliber: counts by the model version a request actually hit. It accurately reflects traffic migration between versions, but depends on the inference service faithfully returning version information.
- Merged old/new version caliber: treats adjacent versions of the same model as a whole, describing evolution with “combined scale + version share.” This caliber best answers “is this model growing or shrinking overall.”
Using the illustrative data for old v4 and new v4.1:
| Caliber | Observation | Risk note |
|---|---|---|
| New version only | The new version alone shows a clear ramp curve | Overstates the new version’s independent impact; ignores the old version’s stock |
| Old + new combined | Combined ~49.55 trillion tokens, about +35.5% WoW | The main basis for judging “whether the model as a whole is expanding” |
| Routing alias caliber | A leading model shows ~8.26 trillion tokens, about +36.5% WoW | Alias definitions and mapping rules differ across platforms; unify the caliber before comparing horizontally |
So the report should fix a primary caliber and display one comparison caliber alongside it, avoiding “the same thing, two stories.” In addition, naming should follow a consistent normalization rule. Industry monitoring commonly uses a “major version - variant identifier” convention (for example, “major version - variant”), and internal naming should keep this structure so a new version can automatically roll up under its corresponding model when it launches.
2.3 Time Window
- Natural week: fixed start and end each week, convenient for aligning with business weekly reports and operations meetings;
- Rolling 7 days: look back 7 days from each day, faster to respond and removes weekday effects.
The two can run in parallel: rolling 7 days for daily monitoring, natural week for weekly review meetings.
3. Data Pipeline and Instrumentation Design
3.1 Three-Layer Data View
- Access layer: applications call the model service through a unified gateway or SDK, and the gateway records the call result. This is the starting point for building the data view;
- Aggregation layer: pre-aggregate metrics such as tokens and request counts by hour or by day, and store them in partitioned tables in the data warehouse;
- Presentation layer: weekly reports, trend dashboards, and alerting rules.
The purpose of layering is that “raw details are traceable, and aggregate results are recomputable.”
3.2 Unified Call-Log Model
Instrumentation across all pipelines should write into a single call-log table. The core fields are as follows:
| Field group | Typical fields | Purpose |
|---|---|---|
| Identity | request_id, trace_id, tenant_id | Deduplication, end-to-end correlation, per-tenant analysis |
| Routing | requested model alias, actual routed backend | Distinguishing the three statistical calibers |
| Metering | input tokens, output tokens, reasoning tokens (if any), cached-token hits | Cost attribution and cache-effectiveness evaluation |
| Attributes | business scenario, application identifier, model capability tags (e.g., reasoning/non-reasoning) | Attribution along business dimensions |
| Performance | first-token latency, total latency, status code | Distinguishing successful calls from failed retries |
One point deserves emphasis: a unified token caliber is a prerequisite for data usability. If different pipelines handle reasoning tokens and cache-hit tokens inconsistently, the same business’s total will not reconcile across reports.
3.3 Unified Instrumentation at the Gateway
Instrumentation should sit as close to the unified call entry point as possible, avoiding each business team counting on its own. The following Python pseudocode shows how to write the log after the gateway processes the response; field names are illustrative:
import timefrom collections import namedtupleRequestLog = namedtuple("RequestLog",["request_id", # Unique request identifier, generated by the gateway and passed through the whole chain"ts", # Use UTC timestamps to avoid inconsistent time-zone calibers"tenant_id", # Tenant/business identifier, used for attribution analysis"scenario", # Business scenario label, e.g., intelligent customer service, code assistant, document writing"requested_as", # Model alias used at request time (routing alias caliber)"routed_to", # Actual model and version routed to (backend caliber)"input_tokens","output_tokens","reasoning_tokens", # Record reasoning tokens if a reasoning model produces them, otherwise 0"cached_tokens", # Input tokens that hit the cache"status", # success / failure"retry_count", # Number of retries, used to exclude failure amplification],)def log_llm_call(ctx, result):"""Log the call uniformly after the gateway processes the model response"""usage = result.usage # Metering information returned by the model servicerecord = RequestLog(request_id=ctx.request_id,ts=time.time(),tenant_id=ctx.tenant_id,scenario=ctx.scenario,requested_as=ctx.requested_alias,routed_to=result.model_version,input_tokens=usage.input_tokens,output_tokens=usage.output_tokens,reasoning_tokens=getattr(usage, "reasoning_tokens", 0),cached_tokens=usage.cached_tokens,status=result.status,retry_count=ctx.retry_count,)# Asynchronously enqueue to the message queue; downstream consumers handle persistence and aggregationenqueue_to_queue("llm_call_log", record)
Once the log is enqueued to a message queue, the consumer performs cleaning, deduplication, and persistence. This decouples “recording” from “analysis” and keeps statistical logic from invading business code.
4. Weekly Aggregation and Week-Over-Week Calculation
On top of the detail data, you can aggregate by “time × model × scenario × version” and compute WoW with window functions:
-- Example SQL: count per-version call volume by natural week and compute week-over-week-- week_start is a date representing the first day of each week (e.g., Monday)WITH weekly AS (SELECTrouted_to AS model_version,week_start,SUM(input_tokens + output_tokens + reasoning_tokens) AS total_tokens,COUNT(DISTINCT CASE WHEN status = 'success' THEN request_id END) AS requestsFROM llm_call_logWHERE ts >= DATE '2025-09-29' -- Analysis start date (Monday)AND ts < DATE '2025-11-03' -- Analysis end date (next Monday, exclusive)GROUP BY routed_to, week_start)SELECTmodel_version,week_start,total_tokens,requests,-- Week-over-week: (this week - last week) / last week; return NULL when last week is 0 to avoid division by zeroROUND(100.0 * (total_tokens - LAG(total_tokens) OVER w)/ NULLIF(LAG(total_tokens) OVER w, 0), 1) AS wow_pct,-- Version share: this version's tokens this week / all versions' tokens this weekROUND(100.0 * total_tokens / SUM(total_tokens) OVER (PARTITION BY week_start), 1) AS share_pctFROM weeklyWINDOW w AS (PARTITION BY model_version ORDER BY week_start)ORDER BY week_start, total_tokens DESC;
Illustrative output (values are for explanation only, not real data):
| model_version | week_start | total_tokens (trillions) | requests (100M) | wow_pct | share_pct |
|---|---|---|---|---|---|
| ModelA-v4.1 (new) | Oct 20 | 49.55 | — | +35.5 | 26.9 |
| ModelB-v3.1-Flash | Oct 20 | 14.13 | — | new | 7.7 |
| ModelC (routing alias caliber) | Oct 20 | 8.26 | — | +36.5 | 4.5 |
| ModelD-Preview | Oct 20 | 2.84 | — | -79.6 | 1.5 |
| All versions total | Oct 20 | 184.01 | 70.18 | +9.3 | 100.0 |
Three conclusions follow:
- A new version’s ramp usually leads overall growth and is a leading signal for total-volume change;
- When a preview or test version declines sharply, verify whether it is a planned retirement or a migration to the stable version, to prevent resource misallocation;
- The version-share curve is the most direct basis for judging traffic migration.
5. Attribution Analysis: From “How Much Did It Grow” to “Why Did It Grow”
5.1 First Layer: Volume-Price Decomposition
Total tokens can be decomposed into “request count × average tokens per request.” For additive totals such as call volume and revenue, the WoW change can be decomposed precisely as follows:
def decompose_delta(prev: dict, curr: dict) -> dict:"""Decompose the week-over-week change of an additive total (e.g., tokens, requests)prev / curr look like {"ModelA-new": 40, "ModelB": 30}Return each item's contribution to the base-period total (percentage points), plus the cross term"""total_prev, total_curr = sum(prev.values()), sum(curr.values())if total_prev == 0:raise ValueError("Base-period total is 0, cannot compute week-over-week")keys = set(prev) | set(curr)contributions = {k: (curr.get(k, 0) - prev.get(k, 0)) / total_prev * 100for k in keys}# Cross term = (total change - sum of item changes) / base-period totalcross = ((total_curr - total_prev)- sum(curr.get(k, 0) - prev.get(k, 0) for k in keys)) / total_prev * 100return {"by_item": contributions, "cross": cross}
Using the illustrative data, a total WoW of +9.3% can be split into three layers:
- Volume-price decomposition: request count contributes about +4.7 percentage points, average tokens per request contributes about +4.1 percentage points, and about +0.5 percentage points comes from the cross effect between the two. This shows that longer per-call consumption is an important source of this round of growth.
- Structural decomposition: the ramp of the new version and the former mainstream version is the main increment, while Model D’s preview version falling sharply partly offsets that increment.
- Scenario decomposition: drill the increment further down into scenarios such as “intelligent customer service, code assistant, document writing” to judge whether the growth comes from natural business growth or from a concentrated ramp in a single scenario.
Only when the three layers drill down progressively does the attribution conclusion become actionable.
5.2 Second Layer: Structural Effects and Common Fluctuation Patterns
A structural effect means the total-volume growth mainly comes from migration from old to new versions rather than net-new calls. Because per-token prices differ across versions, a rise in total volume does not translate into a proportional rise in cost, so you need weighted estimates combined with each version’s unit cost. If the new version has a lower unit price, the migration itself lowers cost; otherwise you must do a budget assessment before migrating.
Common fluctuation patterns and troubleshooting actions:
| Symptom | Possible cause | Troubleshooting action |
|---|---|---|
| A version’s tokens surge while request count stays flat | Longer context, longer output | Analyze the distribution of average input/output length |
| Request count surges, tokens grow even faster | More retries, failure amplification | Check status codes and retry counts |
| A version’s tokens far exceed its request share | Reasoning models generating extra reasoning tokens | Meter and attribute reasoning tokens separately |
| A business’s usage drops sharply | Business retirement, quota limits, pipeline failure | Drill down by tenant and scenario |
| A total spike concentrated on a few dates | Duplicate reporting, load-test traffic mixed in | Deduplicate by request_id, isolate with tags |
That last point especially warrants caution: when load-test traffic mixes with real user traffic, the weekly curve shows spikes that cannot be explained. It is recommended to tag load-test and canary traffic separately at the access layer.
5.3 Third Layer: Medium- and Long-Term Trend Fitting
Applying linear regression or a moving average to several consecutive weeks of data yields a baseline:
def trend_baseline(series: list[dict], window: int = 4) -> dict:"""Fit a linear trend based on the last `window` weeks of data; return slope and next-period forecastseries sorted ascending by time, elements look like {"week": "2025-10-20", "tokens": 184.01e12}"""recent = series[-window:]n = len(recent)xs = list(range(n))ys = [p["tokens"] for p in recent]mean_x, mean_y = sum(xs) / n, sum(ys) / ndenom = sum((x - mean_x) ** 2 for x in xs)slope = sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / denomintercept = mean_y - slope * mean_xreturn {"weekly_growth": slope, # Average weekly increment (tokens)"next_week_forecast": intercept + slope * n,}
When pulse-like ramps are caused by reasons such as new-version releases or major marketing campaigns, you should exclude the anomalous weeks before fitting; otherwise the baseline gets significantly inflated, and later normal fluctuations end up being misreported as a decline.
6. Anomaly Detection and Tiered Alerting
The value of trend tracking ultimately shows up in “detecting a problem before users complain.” A single-threshold alert is easily triggered by business growth, so combining a “WoW baseline + statistical outlier detection” is recommended:
import statistics as statsdef flag_anomalies(series: list[dict], k: float = 3.5) -> list[dict]:"""Robust outlier detection based on median absolute deviation (MAD); more robust for skewed dataCorrection factor 0.6745 makes MAD comparable to standard deviation under a normal distributionDo not judge when there are fewer than 8 consecutive weeks, to avoid false alarms from insufficient samples"""if len(series) < 8:return []values = [p["tokens"] for p in series]med = stats.median(values)mad = stats.median(abs(v - med) for v in values)if mad == 0:return []flags = []for p in series:score = 0.6745 * (p["tokens"] - med) / madif abs(score) > k:flags.append({**p, "robust_z": round(score, 2)})return flags
Based on the detection results, you can set up three alert tiers:
- Info level: WoW deviation from the baseline beyond a set margin, or a version-share weekly change exceeding a set threshold;
- Warning level: total tokens deviating from the trend for two consecutive weeks, or average tokens per request rising noticeably;
- Critical level: usage dropping or surging sharply (e.g., single-week WoW beyond ±50%), accompanied by a rising error rate.
Critical-level alerts must notify on-call staff immediately and integrate with logging and monitoring-alerting services to quickly pinpoint the specific tenant, scenario, or version.
7. From Observation to Action: Cost, Capacity, and Quality
Weekly data is valuable only when it turns into action. It can support at least three kinds of decisions.
1. Cost management
- Cost attribution: aggregate token consumption by business line, scenario, and version, combine it with each version’s unit price to compute cost contribution, and identify high-consumption, low-value scenarios;
- Unit cost: use “unit cost = total cost ÷ valid request count” to measure efficiency, and split its change into price change and structural change;
- Optimization directions:
- Trim prompts and retrieval context;
- Enable input caching to reuse repeated prefixes;
- Use layered summarization for long-document tasks;
- Route low-risk requests to lightweight models;
- Reduce redundant output through structured-output constraints;
- Cache effect: bringing cache-hit tokens into the metric system helps catch cost increases caused by a declining cache hit rate early.
2. Capacity planning
- Capacity headroom should be measured by peak rather than average, so it is recommended to track the weekly peak’s P95/P99 as well;
- Set expansion plans by overlaying a safety margin on the trend baseline to avoid queuing from a concentrated ramp on release day;
- Under a multi-model routing architecture, backends share a total quota pool, so a new version’s ramp requires assessing its impact on the overall quota pool in advance;
- Set quotas and rate limits for critical businesses to prevent an anomalous spike in one scenario from dragging down the whole system.
3. Quality and version governance
- Watch quality metrics such as “failures per thousand requests,” “first-token latency P95,” and “user feedback rate,” which together with call volume form version health;
- When a preview version declines rapidly, confirm promptly whether it is a planned retirement or an anomalous failure, and guide existing traffic to migrate in advance;
- Adopt a “canary ramp + weekly review” rhythm for new-version ramp: each week, review three data points—call share, unit cost, and quality metrics—and expand the canary scope only after they meet targets;
- For business parties with large call volumes, use object storage for long-cycle archiving to provide a basis for later training-data screening and audit retention.
8. Rollout Cadence and Common Pitfalls
It is recommended to proceed in three phases:
- Weeks 1–2: Unify the caliber. Complete instrumentation at the unified gateway, and confirm token calculation rules and naming conventions;
- Weeks 3–4: Complete attribution. Build attribution reports along scenario and version dimensions, and produce the first weekly report;
- Week 5 onward: Close the loop. Configure tiered alerts and link them with cost, capacity, and quality metrics for joint review.
During implementation, pay special attention to the following pitfalls:
- Caliber drift: failing to update the mapping after a model alias or routing rule changes, causing a break in the WoW data;
- Substituting total for structure: total growth masking a single-scenario decline, or a new-version ramp being misread as overall expansion;
- Ignoring delayed data: some platforms have backfill delays in token statistics, and locking the weekly report too early causes numbers to be retroactively corrected in the next period. Set an explicit data-freeze point (e.g., Tuesday of the following week) and mark the revision status;
- Not excluding anomalous weeks: mixing release, big-promotion, and failure weeks directly into the baseline, which keeps alerts ineffective for a long time;
- Reporting numbers without conclusions: every report should include conclusions and action items, such as “Model X’s share is rising; recommend expanding the canary next period.”
9. Conclusion
Weekly trend tracking for LLM call volume is essentially about building an operational loop that is “measurable, attributable, and alertable”:
- Measurement: unify the token caliber and naming conventions to solve the “numbers don’t reconcile” problem;
- Attribution: complete volume-price decomposition, structural decomposition, and scenario drill-down in sequence to answer “where does the growth come from”;
- Alerting: use a baseline plus robust statistics to catch anomalies early and avoid reactive firefighting.
In practice, you can build this pipeline on general cloud capabilities such as data ingestion, message queues, container platforms, logging, and monitoring-alerting. Taking Baidu AI Cloud as an example, developers can pipe call logs through a message queue into a data warehouse for aggregation, then use the monitoring-alerting service for tiered notifications, and integrate the trend dashboard with cost, capacity, and quality metrics in a single view to form a complete call-volume governance loop.
When call volume is measured in trillions of tokens, weekly trends stop being just a numbers game and become a shared basis for version decisions, capacity planning, and cost control.
