Reverse-Engineering LLM Token Costs: Fitting Unit Prices and Cutting Spend with Tiered Routing
Credit-based AI coding and agent plans hide per-token pricing behind time-window quotas, making cost invisible and unpredictable. This article presents a reproducible methodology for turning everyday usage logs into regression data, fitting per-token unit prices for input, cache-hit, and output tokens, and validating the model with R² and residual analysis. It shows how to convert those prices into a per-window token budget, quantifies the throughput gap between flagship and lightweight tiers, and explains how to build a tiered routing strategy that sends "thinking and review" work to expensive models and "doing" work to cheap ones. Readers will learn how to treat model selection as an engineering decision backed by data, along with guardrails to prevent silent quality degradation. All numeric examples are illustrative and only demonstrate the calculation method.
1. The Starting Point: Credit-Based Plans Make Cost Invisible
Time-window quotas (such as “every 5 hours” or “weekly”) measured in credits are now a common way to supply intelligent coding and agent tools. The upside is that they hide fine-grained billing. The downside is equally obvious: you never see a unit price, only whether credits “drain fast” or “drain slowly.”
This produces three typical pain points:
- Unpredictable budget: The same task can consume one to two orders of magnitude more or fewer credits depending on the model tier or how you phrase a prompt, so teams can’t reliably plan around it.
- Selection by gut feeling: Developers often decide by intuition — “expensive must be stronger, cheap must be rough” — with no quantifiable cost-benefit basis.
- Misallocated cost focus: Using the strongest model for deterministic tasks like formatting or batch replacement is the most common form of waste.
The fix isn’t to wait for an official price list. It’s to treat your everyday usage records as observational data and reverse-engineer the unit prices yourself. Below is a complete, reproducible methodology.
2. Turning “Usage Records” into a Regression Problem
2.1 First, Standardize the Measurement Basis
A failed fit is usually not a math problem but a measurement-consistency problem. Before collecting anything, lock down the following, otherwise samples will be internally inconsistent and the regression will inevitably drift:
- Observation boundary: Use a single independent request (or one complete session) as the smallest unit. Record the credit delta before and after that request, not the day’s total delta.
- Token breakdown: Count input, cache-hit, and output tokens separately. Their unit prices differ enormously (a cache hit is often just a single-digit percentage of the normal input price), and mixing them together will absorb the coefficients.
- Special consumption: Long-context retrieval, intermediate reasoning/thinking tokens, and the round-trip overhead of tool calls must be tagged separately, not stuffed into the ordinary input column.
- Single-model collection: Run only one model tier per observation. Mixing calls leaves the system of equations unsolvable.
Suggested observation record structure:
{"obs_id": "0421-0137","tier": "midrange", // model tier identifier, for grouping only"input_tokens": 184230,"cache_hit_tokens": 152000,"output_tokens": 9120,"points_before": 42310,"points_after": 38935,"window": "5h","tags": ["retrieval", "tool_call"]}
2.2 Build a Linear Model
Credit consumption is approximately linear in token count, so it can be described with a three-variable relation:
P = a · T_in + b · T_cache + c · T_out
where a, b, and c are the unit prices (credits/token) for input, cache-hit, and output tokens respectively. There are only three unknowns, so in theory, three or more mutually linearly-independent observations are enough to solve for them. In practice, collect 30–50 observations per tier to average out measurement noise and fixed per-session overhead.
Pseudocode for solving and validating:
# X: one observation per row, columns are [input tokens, cache-hit tokens, output tokens]# y: corresponding credit consumptionX = [[t_in, t_cache, t_out], ...]y = [p_before - p_after, ...]w = least_squares_fit(X, y) # solve for coefficients minimizing residual sum of squaresa, b, c = wper_1M = [a * 1_000_000, b * 1_000_000, c * 1_000_000]pred = matmul(X, w)ss_res = sum((y - pred) ** 2)ss_tot = sum((y - mean(y)) ** 2)r2 = 1 - ss_res / ss_tot # goodness of fitresidual = y - pred # per-observation residuals, to locate outliers
Two signals you must check:
- R² close to 1 with residuals near zero: This confirms the model is billed linearly per token and the fit is trustworthy.
- Rank deficiency (rank < 3): This usually happens because cache-hit values are too uniform in the sample, or because several observations have highly similar input/output ratios, making the coefficients collinear. In that case, deliberately collect more “output-dominant” and “cache-hit-dominant” samples.
2.3 Express Results as “Per Million Tokens”
Multiplying the coefficients by 10⁶ to convert them into “credits per million tokens” produces a unit that people can compare directly. The following is an illustrative setup (used only to demonstrate the calculation flow; it does not represent any real product’s pricing):
| Tier | Input (per 1M) | Cache hit (per 1M) | Output (per 1M) |
|---|---|---|---|
| Flagship reasoning | 60000 | 3000 | 300000 |
| Midrange reasoning | 25000 | 1200 | 90000 |
| High value / fast | 3000 | 60 | 6000 |
| Lightweight | 1500 | 1500 | 1500 |
Two conclusions that can immediately change decisions:
- The flagship tier’s output price is 200 times that of the lightweight tier. Having a flagship model generate large blocks of boilerplate code is using the most expensive resource for the cheapest work.
- After a cache hit, the high-value tier’s unit price drops to just 2% of normal input. This means prompt-prefix stability translates directly into cost — if the system prompt, tool definitions, and long-document context change every turn, the cache can’t hit, and that leverage is simply thrown away.
3. From Unit Price to Throughput: How Many Tokens a Window Buys
In real workloads, input far exceeds output. Using the empirical assumption of 80% input + 20% output (ignoring cache hits for now), you can define a blended effective unit price:
def blended_cost_per_1m(a_in, a_out, p_in=0.80, p_out=0.20):# a_in / a_out are both in "credits per 1M tokens"return p_in * a_in + p_out * a_outdef tokens_per_window(budget, a_in, a_out, p_in=0.80, p_out=0.20):return budget / blended_cost_per_1m(a_in, a_out, p_in, p_out) * 1_000_000
Plugging in the table above, using a 60,000-credit / 5-hour window:
- Flagship reasoning: 108000 credits/1M → about 556,000 tokens
- Midrange reasoning: 38000 credits/1M → about 1,579,000 tokens
- High value: 3600 credits/1M → about 16.67 million tokens
- Lightweight: 1500 credits/1M → about 40 million tokens
The throughput gap between the highest and lowest tiers is roughly 72x. This ratio — not any single unit price — is the real space of resources a team can allocate within a 5-hour window. It directly dictates architecture: if a step can only run a few hundred thousand tokens on the flagship tier, it inherently cannot handle tasks like “read the entire codebase.” Such work must be pushed down to a low-cost tier or have its context trimmed at the retrieval stage.
4. Tiered Routing: Expensive Models “Think,” Cheap Models “Do”
With unit prices in hand, routing strategy gains a quantitative basis. The core principle in one sentence: allocate scarce resources based on reasoning depth, not on task importance.
4.1 Basis for Task Tiering
| Level | Typical tasks | Resource profile |
|---|---|---|
| Planning & final review | Requirement breakdown, interface design, code review, decision-making between options | Few calls, high value per call |
| Complex reasoning | Debugging hard defects, cross-module refactoring, algorithm design | Short output, long reasoning chains |
| Main execution | Feature implementation, routine refactoring, writing tests | Large input and output, the bulk of volume |
| Batch / simple | Formatting, renaming, comments, batch migrations, log cleanup | Highly deterministic, massive scale |
The key is that only “few calls, high value per call” tasks deserve the flagship tier. Planning and final review typically account for less than 5% of total tokens yet determine the correctness of the overall output, while the 80%+ of execution work must have its unit cost pushed to a lower tier.
4.2 A Practical Routing Implementation
def dispatch(task):# 1) Deterministic batch tasks: go to the cheapest tier, with a budget capif task.kind in {"format", "rename", "docstring", "log_summary"}:if task.estimated_tokens <= CHEAP_LIMIT:return lightweight# 2) Final review and planning: prefer expensive over wrongif task.kind in {"review", "architecture", "plan"}:return flagship# 3) Implementation needing multi-step reasoning: midrange, with an escalation pathif task.requires_reasoning_depth >= 3 or task.crosses_modules:return midrange# 4) Remaining main execution tasksreturn fast_cheap
Three supporting pieces are needed to make this actually work:
- Unified access layer: Route all model calls through a single entry point, making unified measurement, rate limiting, and canary switching straightforward. Call logs land in a logging service and object storage for later regression fitting.
- Escalation path: When a lower tier fails repeatedly past a threshold (e.g., two failed validations), automatically escalate to a higher tier and retry, recording the escalation rate. The escalation rate is a core metric for judging “whether the tiering is cut correctly” — a sharp rise means the tier boundaries are too conservative.
- Cost quotas and alerts: Set per-window credit budgets by project/member, with monitoring alerts at thresholds to prevent a single long task from eating the entire window.
4.3 Quality Guardrails
Cost reduction is most dangerous when quality degrades silently. Maintain a fixed evaluation set (covering representative tasks from real repositories) and run a regression pass every time you adjust tier mapping or prompt templates, tracking three metrics: task pass rate, average retry count, and average credits per task. Only when all three improve simultaneously is it a genuine optimization. If cost drops but pass rate falls, the routing is cut in the wrong place, not saving in the right one.
5. Common Pitfalls and Risks
- Watching only output price: In long-context scenarios, the absolute volume of input and cache hits is often several times that of output. Ignoring the first two severely underestimates cost.
- Ignoring cache invalidation: Dynamic timestamps, random IDs, and history concatenated line by line all break the prefix and drive the cache hit rate to zero. Fixing the system prompt and moving variable content to the end is the most effective low-cost optimization.
- Sample collinearity distorting coefficients: Fitted negative coefficients or abnormally large coefficients almost always stem from a flaw in observation design — go back to the collection stage rather than tuning the algorithm.
- The rules will change: Credit multipliers, caching strategy, and rate limits can all be adjusted, so the reverse-engineered unit prices are time-sensitive. Make the fit a recurring job (e.g., rerun weekly) and use the residual trend as an early indicator of rule changes.
- The numbers here are illustrative: All unit prices and throughput figures in this article are hypothetical setups that demonstrate the calculation method. Real values must come from fitting your own measured records.
6. Conclusion
The real value of credit-based plans is not that they hide unit prices, but that they force users to build their own cost model. Once you master the chain of “itemized measurement → linear regression → residual validation → throughput conversion → tiered routing,” model selection stops being a gut-feeling decision and becomes an engineering decision you can recheck with data at any time.
In one sentence, the takeaway of this method: let expensive models focus on thinking and vetting, let cheap models handle large-scale execution, and use continuous regression to ensure this trade is never made at the expense of quality.
