Multi-Model API Cost Accounting Guide: Pricing Dimensions and Cost Modeling

    Comparing model API price sheets head-to-head often leads to the wrong conclusion, because a unit price is not a cost. This guide breaks the pricing dimensions—asymmetric input/output rates, prompt-cache hit and miss tiers, cache write and storage holding costs, time-of-day multipliers, batch discounts, and discount stacking/exclusion rules—into orthogonal parameters, then builds a unified cost function with runnable Python code. Worked examples expose the break-even point behind each optimization: when caching beats batch, how many times a cache write must be reused to pay off, and how tiered routing lowers spend. The method is vendor-independent—swap in a new price sheet, replace the parameters, and recompute.

    1. The Starting Problem: Why Price Sheets Can’t Be Compared Head-to-Head

    The most common move in model selection is to lay out each vendor’s price sheet and compare them side by side. In most cases this leads to the wrong conclusion. The problem isn’t that the sheet is wrong—it’s that the unit price itself is not the cost.

    A complete price sheet usually contains at least these columns: input (cache miss), input (cache hit), output, cache write, and cache storage (priced per million tokens per hour). On top of that come several multipliers: peak-hour rates, batch discount factors, and discount-stacking rules. In other words, the unit price is “the price per unit of resource,” while cost is “the product of total resource consumption and the price structure.” If two models differ by several times in their quoted input price, but one supports high-hit-rate context caching and the workload is a long system prompt with frequent short requests, the actual bill can end up reversed.

    So the approach here is to first decompose pricing into a set of orthogonal parameters, then build a unified cost function, and finally use a handful of worked examples to find the critical conditions for each type of optimization. The resulting conclusions don’t depend on any specific vendor’s price sheet—swap in different parameters and they apply to any provider.

    2. A Complete Breakdown of the Pricing Dimensions

    2.1 Asymmetric Pricing for Input and Output

    Nearly every model service prices input and output tokens separately, and the output price is usually several times the input price. This isn’t a pricing preference—it follows from the compute characteristics of inference: the input stage can process the entire context in parallel, while the output stage must generate tokens autoregressively one at a time, which is far more demanding in per-token compute and memory.

    The first engineering takeaway follows: in most workloads, the dominant cost term sits on the output side, not the input side. Compressing output usually pays off more than compressing input, and the levers for compressing output are also more direct—cap the output length, constrain the output format to structured schemas, and cut redundant restatement and explanation.

    2.2 Cache-Hit and Cache-Miss Tiers

    Prompt caching reuses tokens that repeat in the request prefix, avoiding redundant computation. In pricing terms, the input is split into two tiers: the hit portion is billed at a discounted rate far below the miss price, and the miss portion at the standard rate.

    The key point is that whether the cache hits depends on the prefix being stable token by token. This means the engineering design must arrange content as “stable prefix, variable suffix”—system prompts, tool and function definitions, few-shot examples, and domain knowledge (static content) should go as far to the front as possible, while timestamps, user input, random IDs, and other dynamic content go at the back. A single change that inserts a variable at the start of the prompt invalidates the entire prefix and drives the hit rate to zero.

    2.3 The Holding Costs of Cache Writes and Cache Storage

    Caching isn’t free. Most platforms charge a one-time write fee for creating a new cache (typically on the order of the input-miss price or slightly higher) and bill for the cache’s residence by “million tokens × hour.” This brings both a one-time cost and a holding cost into the model:

    • The write cost decides “whether it’s worth building the cache,” and depends directly on the number of reuses;
    • The storage cost decides “how long the cache should stay,” and is strongly tied to the TTL and off-peak traffic.

    Ignoring these two items systematically overestimates cache benefits; in low-traffic, long-TTL scenarios, ignoring them can even make an overall caching strategy net-negative.

    2.4 Time-of-Day Pricing and the Attribution Cutoff

    Some platforms apply a multiplier by the time the request occurs: weekday daytime is the peak tier, while nights, weekends, and public holidays are off-peak (a common approach prices off-peak at half the peak). This design turns “time” into a schedulable cost variable.

    The trap here isn’t the multiplier itself but the cutoff used to attribute a request to a time period: is it based on when the request is initiated, when the first byte is returned, or when the request fully completes? For a long-output request that straddles the peak cutoff, billing by completion time versus billing by initiation time can differ by a factor of two. When doing your own accounting, you must align with the same cutoff as your recorded call times—otherwise your monthly reconciliation will never quite add up.

    Also note the minimum billing unit and rounding method (for example, whether billing rounds up per thousand tokens). In scenarios with dense very-short requests, this amplifies the effective unit price.

    2.5 Batch Discounts: Trading Latency for Price

    Batch APIs collect non-real-time requests into batches and run them asynchronously, trading a noticeably longer completion latency for a discount (commonly half price or 60%). They suit workloads that don’t require second-level response: offline processing, bulk extraction, content-moderation backfills, and log analysis.

    Batch and context caching solve different problems: batch reduces the premium from resource contention within a time unit, while caching reduces redundant computation. The two aren’t inherently in conflict at the business level, but they often are mutually exclusive in billing rules—that’s the focus of the next section.

    2.6 Discount Stacking and Mutual-Exclusion Rules

    A real bill is made up of “base price × each effective factor,” and platforms usually spell out which factors can apply simultaneously and which are mutually exclusive. The most typical rule: batch discounts and context-cache discounts often can’t be enjoyed at the same time—the system picks one when billing, or applies one per rule.

    This means “batch + cache” can’t be stacked into “an even cheaper cache.” Whether to use batch requires an explicit comparison: on the same workload, is the effective unit price via the cache path or via the batch path lower? This critical point can be solved analytically—see Section 4.

    2.7 Other Easily Missed Pricing Items

    • Whether reasoning/thinking content is billed: If the model’s output includes a reasoning process, whether that part counts as output tokens directly determines the bill’s magnitude; long-thinking tasks need separate accounting.
    • Context-tiered pricing: Some platforms price by context-length brackets, so long-context requests effectively cost more than the sticker price.
    • Multimodal input: Image, audio, and other inputs are often converted at separate units or priced separately.
    • Cache-discount caps or activation conditions: For example, only prefixes longer than a certain length qualify.

    3. A Unified Cost Model

    Parameterizing the dimensions above, the cost of a single request can be written as follows. Let:

    • P_in: input cache-miss unit price (yuan per million tokens)
    • P_hit: input cache-hit unit price
    • P_out: output unit price
    • P_w: cache write unit price
    • P_s: cache storage unit price (yuan per million tokens per hour)
    • h: the request’s cache hit rate—the ratio of cache-hit tokens to input tokens
    • k_peak: time-of-day multiplier (off-peak is a discount factor of 1 or less)
    • k_batch: batch factor (1 if not batched)

    Then the single-request cost:

    1. Cost = [ (1 - h) · In · P_in + h · In · P_hit + Out · P_out ] · k_peak · k_batch
    2. + Write · P_w + Cached · Hours · P_s

    where In is the number of input tokens, Out the number of output tokens, Write the number of tokens written to cache this time, Cached the number of tokens residing in cache, and Hours the residence hours of that cache. The cache-storage term generally doesn’t change with batch or time-of-day discounts and should be accounted for separately.

    From this we can define two intermediate quantities that are useful for decisions:

    Effective input price (folding the hit rate into the price):

    1. P_in_eff = h · P_hit + (1 - h) · P_in

    Cost per 1,000 requests, used to convert a token-dimension model into business-dimension capacity and budget:

    1. Cost_per_1k = 1000 · (P_in_eff · In_avg + P_out · Out_avg) / 1e6 · k_peak · k_batch

    4. Runnable Accounting Code

    The implementation below hard-codes the formulas above so it’s easy to plug into a bill-attribution pipeline.

    1. from dataclasses import dataclass
    2. @dataclass(frozen=True)
    3. class PriceSpec:
    4. """Pricing parameters. All in yuan per million tokens (storage in yuan per million tokens per hour)."""
    5. p_in: float # input, cache miss
    6. p_out: float # output
    7. p_hit: float # input, cache hit
    8. p_write: float = 0.0 # cache write
    9. p_store: float = 0.0 # cache storage (per hour)
    10. k_peak: float = 1.0 # peak-hour multiplier
    11. k_off: float = 1.0 # off-peak multiplier
    12. k_batch: float = 1.0 # batch factor
    13. def effective_input_price(p: PriceSpec, hit_ratio: float) -> float:
    14. """Fold the cache hit rate into the input unit price."""
    15. h = min(max(hit_ratio, 0.0), 1.0)
    16. return h * p.p_hit + (1.0 - h) * p.p_in
    17. def request_cost(p: PriceSpec, in_tokens: int, out_tokens: int,
    18. cached_tokens: int = 0, write_tokens: int = 0,
    19. cache_hours: float = 0.0,
    20. is_peak: bool = False, batch: bool = False,
    21. cache_batch_stackable: bool = False) -> dict:
    22. """
    23. Return the cost breakdown of a single request.
    24. cache_batch_stackable: whether the platform allows stacking the batch discount with the cache discount.
    25. Defaults to False, i.e., when the two are mutually exclusive, compute via the
    26. cache path; in that case the caller must decide whether batch is cheaper
    27. (see cache_vs_batch_breakeven).
    28. """
    29. h = cached_tokens / in_tokens if in_tokens else 0.0
    30. p_in_eff = effective_input_price(p, h)
    31. k_time = p.k_peak if is_peak else p.k_off
    32. # When mutually exclusive: batch path is computed without cache, cache path without batch
    33. if batch and not cache_batch_stackable:
    34. k_batch, k_time_cache = p.k_batch, 1.0
    35. base = (in_tokens * p.p_in + out_tokens * p.p_out) / 1e6
    36. cached_base = (in_tokens * p_in_eff + out_tokens * p.p_out) / 1e6
    37. else:
    38. k_batch = p.k_batch if batch else 1.0
    39. k_time_cache = k_time
    40. base = cached_base = (in_tokens * p_in_eff + out_tokens * p.p_out) / 1e6
    41. token_cost = cached_base * k_time_cache
    42. batch_cost = base * k_time * k_batch
    43. store_cost = cached_tokens * cache_hours * p.p_store / 1e6
    44. write_cost = write_tokens * p.p_write / 1e6
    45. return {
    46. "hit_ratio": round(h, 4),
    47. "p_in_effective": round(p_in_eff, 6),
    48. "token_cost": round(token_cost, 8),
    49. "batch_alternative": round(batch_cost, 8),
    50. "write_cost": round(write_cost, 8),
    51. "store_cost": round(store_cost, 8),
    52. "total": round(token_cost + write_cost + store_cost, 8),
    53. "output_share": round(out_tokens * p.p_out / 1e6 * k_time_cache
    54. / max(token_cost, 1e-12), 4),
    55. }
    56. def cache_vs_batch_breakeven(k_batch: float, hit_discount: float) -> float:
    57. """When the batch and cache discounts are mutually exclusive, the minimum hit rate at which the cache path overtakes batch."""
    58. if hit_discount >= 1.0:
    59. return float("inf")
    60. return max(0.0, (1.0 - k_batch) / (1.0 - hit_discount))
    61. def cache_write_breakeven(p: PriceSpec, prefix_tokens: int) -> float:
    62. """How many requests, on average, a single cache write must be reused across before the cache pays off."""
    63. saving = (p.p_in - p.p_hit) * prefix_tokens / 1e6
    64. cost = p.p_write * prefix_tokens / 1e6
    65. return cost / saving if saving > 0 else float("inf")

    For the accompanying call-detail collection, it’s recommended to log one structured record per request, fixing all the inputs needed for billing:

    1. record = {
    2. "ts": "2026-09-03T10:12:00+08:00",
    3. "biz_line": "code-assist", # business line, for cost allocation
    4. "model_tier": "flagship", # model tier, for tiered-routing accounting
    5. "in_tokens": 8500,
    6. "cached_tokens": 8000,
    7. "write_tokens": 0,
    8. "out_tokens": 400,
    9. "is_peak": True, # aligned with the platform's time-period cutoff
    10. "batch": False,
    11. "cache_hours": 1.0,
    12. "price_snapshot": "2026-09", # price-snapshot version, so price changes are traceable
    13. }

    With this level of detail, the monthly bill can be attributed by business line, model tier, and cache-hit band, turning cost optimization from guesswork into reading a curve.

    5. Worked Examples: Finding the Critical Points with Illustrative Numbers

    The prices below are illustrative values chosen for easy hand calculation and don’t represent any platform’s actual pricing. Assume a flagship tier: input miss 5, output 20, cache hit 0.5, cache write 5, cache storage 0.02/hour; and a lightweight tier: input miss 1, output 3, cache hit 0.2.

    Scenario A: Long System Prompt + High-Frequency Short Requests

    Assume each request has 8,500 input tokens (of which 8,000 are a cacheable static prefix) and 400 output tokens.

    • Flagship + cache hit: hit rate h ≈ 94.1%, effective input price = 0.5 × 0.941 + 5 × 0.059 ≈ 0.77, single-request cost ≈ 0.0065 + 0.008 = 0.0145 yuan
    • Flagship + no cache: 8,500 × 5 / 1e6 + 0.008 = 0.0425 + 0.008 = 0.0505 yuan

    Caching saves about 71%, and the output term’s share of total cost rises to about 55%—which shows that once caching is in place, the next optimization focus naturally shifts from “compressing input” to “compressing output.”

    Scenario B: Batch or Cache?

    Batch and cache discounts are mutually exclusive. From the critical formula in Section 4, let k_batch = 0.5 and hit_discount = 0.5 / 5 = 0.1:

    1. h* = (1 - 0.5) / (1 - 0.1) ≈ 0.556

    That is, when the cache hit rate is above about 55.6%, going the cache route is cheaper than batch; below that, drop the cache and go straight to batch. Scenario A’s 94.1% is far above the threshold, so the conclusion is clear; but for a chain with an extremely unstable prefix and a hit rate of only 20%, continuing to maintain the cache mechanism is actually just paying for cache writes. This threshold is one of the most memorable conclusions here: it turns “should we cache” from a judgment call into a division.

    Scenario C: Amortizing the Cache Write

    By the amortization formula, with a prefix length of 8,000 tokens:

    1. N* = P_write / (P_in - P_hit) = 5 / (5 - 0.5) ≈ 1.11

    So a cache content of 8,000 tokens only needs to be reused by an average of two or more requests for the write to pay off. But if the workload is “a different set of domain material for every request,” the reuse count approaches 1, and the cache mechanism only adds write fees—in that case you should disable caching, or move the dynamic material to the prompt’s tail and cache only the truly stable system part.

    The storage-cost criterion is calculable too: with 24-hour residence and a storage unit price of 0.02, the storage overhead comes to 0.48 per million tokens, which takes savings from more than about 0.11 cache hits to cover. For short-lived high-frequency traffic, the storage term is usually negligible; for long-TTL + low-frequency traffic, it can become the dominant term, and you should actively shorten the TTL.

    Scenario D: Peak-Shifting and Batch Combined

    Suppose an offline task must process 100 million output tokens with no shared prefix (h ≈ 0). Peak non-batch cost = 2,000 yuan; if the whole batch is moved to off-peak (multiplier 0.5) and run through batch (factor 0.5), the cost is 500 yuan, a 75% reduction. This kind of “move the entire non-real-time workload off peak” is the single highest-ROI optimization, and its only precondition is that the task can be made asynchronous.

    Scenario E: Weighted Cost of Tiered Routing

    Suppose that among 1 million requests, 60% are simple tasks (can go to the lightweight tier) and 40% need the flagship tier. Using Scenario A’s per-request costs (flagship 0.0145, lightweight 0.0033):

    • All flagship: 14,500 yuan
    • All lightweight: 3,300 yuan
    • Tiered routing: 0.6 × 3,300 + 0.4 × 14,500 = 7,780 yuan, saving about 46% versus all-flagship

    The essence of tiered routing is treating “model capability” as a budget that can be consumed at different levels, decided by an upfront intent classifier. It requires confirming on an offline evaluation set that the lightweight tier’s quality loss is acceptable, then gradually opening up traffic share.

    6. A Practical Checklist for Cost Governance

    1. Establish a unified cost basis: all accounting scripts share one PriceSpec; forbid each business line from hardcoding its own unit prices; generate a new price-snapshot version number whenever pricing rules change.
    2. Make bills attributable: call records should always carry business line, tenant, model tier, time period, and batch flag to support showback/chargeback; records can be archived to object storage, with log services and monitoring/alerting systems handling aggregation and threshold alerts.
    3. Do a small-traffic shadow validation first: theoretical unit prices often diverge from real bills (time-attribution cutoff, rounding, discount mutual-exclusion judgment, whether retries and failed requests are billed). Before going live, run 1%–5% traffic in dual-track, compare predicted vs actual cost, calibrate the model, then ramp to full.
    4. Standardize cache design: arrange prompts as “static prefix → dynamic suffix” and make it a code-review checklist item; monitor the ratio of cache hit rate to cache write count, and auto-alert on chains below the threshold.
    5. Optimize the output side: set output length caps, use structured-output constraints, and trim redundant explanation; since output price is usually several times the input, this is where the most direct gains are.
    6. Load tiering and scheduling: real-time interaction goes through the online path, offline processing through batch, accounted separately; move deferrable tasks in batches to off-peak.
    7. Quota and degradation plans: predefine and rehearse peak-time rate limiting, queuing, and fallback to the lightweight tier, to avoid bill spikes from peak multipliers stacking with timeout retries.
    8. Periodically re-reconcile: price sheets and discount rules change, so do a monthly variance analysis of “predicted vs actual,” and when deviation exceeds the threshold, pinpoint the specific pricing item.

    7. Common Pitfalls Quick Reference

    • Comparing only input prices side by side, ignoring that output price is usually several times the input;
    • Assuming batch and cache discounts stack, when the rules often make them mutually exclusive;
    • Ignoring cache write and hourly storage, systematically overestimating cache benefits;
    • Embedding timestamps or random IDs in the prompt, so the prefix can never hit;
    • A time-attribution cutoff inconsistent with the platform, so monthly reconciliation never matches;
    • Not confirming whether reasoning content counts as output tokens, severely underestimating long-thinking task budgets;
    • Very short requests ignoring the minimum billing unit and rounding rules;
    • Pinning all traffic to the flagship tier, without intent classification or tiering.

    8. Conclusion

    The cost of a model API isn’t a unit-price problem—it’s a multiplicative problem jointly determined by token structure, cache hit rate, time-of-day distribution, and batch ratio. Decompose pricing into parameters, write cost as a formula, and map each optimization lever to a computable critical condition—cache hit rate against the batch threshold, cache write against reuse count, storage against TTL and traffic—and model selection and optimization turn from “which is cheaper” into “which is cheaper on my workload.” This method is vendor-independent: swap in a new price sheet, replace the parameters, and recompute. A more pragmatic engineering approach is to choose a platform with clearly disclosed billing rules and capability coverage that fits your scenario (for example, the model services offered by Baidu AI Cloud), paired with a self-built cost-observability system, and make decisions with data rather than price sheets.