1. Unpack the Bill: What Each Request Actually Pays For

AI coding tools, whether IDE plugins, command-line agents, or batch tasks in CI, are themselves stateless. The model is stateless too: it does not “remember” what happened in the previous turn. On every request, the tool must reassemble the system prompt, tool or function definitions, project-level context files, and the full conversation history into one complete prefix before sending it to the model.

This creates a counterintuitive but important fact: if a task runs for ten turns, the stable prefix is read ten times in full. If every token is billed at full price, cost grows roughly linearly with the number of turns, and much of the spend goes toward rereading old content rather than generating new code.

To reduce this waste, mainstream inference services offer prompt caching. The first time a prefix is read, it is written into the cache, usually at a price slightly above normal input. Later requests that hit the same prefix are billed at a substantial discount. The bill is therefore split into several components:

Billing item	Normalized unit price (ordinary input = 1.00)	Content source
Ordinary input (cache miss)	1.00	New content added in this turn, not covered by the cached prefix
Cache write (short TTL)	About 1.25	Prefix read and stored in cache for the first time
Cache write (long TTL)	About 2.00	Longer lifetime, higher premium
Cache read (hit)	About 0.10	Historical prefix that hits the cache
Output	About 5.00	Code generated by the model, explanations, and reasoning process

The table uses normalized ratios to make cross-provider comparison easier. Actual multipliers, minimum cacheable length, and TTL tiers vary significantly across providers. Before production use, always check the official price sheet of the platform being used.

Three details are easy to overlook:

1. Output is usually the most expensive single billing item, about five times the price of ordinary input. For “thinking” models, the reasoning process is also counted as output. Enabling deep thinking means generating content at the most expensive unit price.
2. Cache writes are more expensive than full-price input, at about 1.25x. They are not a “money-saving switch”; they are a prepayment that is amortized by later cache hits.
3. Cache matching is based on the longest common prefix. Any change anywhere in the prefix, including timestamps, random IDs, or a reordered file list, causes everything after that point to fall back to ordinary input or even be written into the cache again.

These three points explain the most counterintuitive conclusion of this article: in small tasks that finish within a few minutes, the biggest cost often comes from writing the cache, not from model output.

2. Experiment Design: Turning “Feels Expensive” into Comparable Numbers

2.1 Fixed Task and Acceptance Criteria

The task selected is a typical, edge-case-heavy business defect: an e-commerce order amount aggregation occasionally fails to reconcile. The repository is a single-module Java project. The defective function is about 60 lines long and involves floating-point accumulation, proportional coupon allocation, rounding mode, and validation for negative or zero quantities.

The core error in the original code:

// Before fix: accumulate amounts with double and compare with ==
double total = 0.0;
for (OrderItem item : items) {
    total += item.getPrice() * item.getQuantity();   // unit price is double
}
if (total == expected) { /* ... */ }
// When allocating a coupon proportionally, round each line first, then sum
double discounted = total * (1 - couponRate);

The corrected version should use the smallest currency unit, specify the rounding mode explicitly, and concentrate rounding error into one line:

// After fix: use integer cents throughout; convert only at the boundary
long totalCents = items.stream()
        .filter(i => i.getQuantity() > 0 && i.getPriceCents() >= 0)
        .mapToLong(i => i.getPriceCents() * i.getQuantity())
        .sum();
// Proportional allocation: distribute first; give the remainder to the largest-amount line to keep the total invariant
long discountedTotal = BigDecimal.valueOf(totalCents)
        .multiply(BigDecimal.ONE.subtract(BigDecimal.valueOf(couponRate)))
        .setScale(0, RoundingMode.HALF_UP)
        .longValueExact();

The acceptance method was identical for all six configurations: 10 hidden unit tests covering edge cases such as cent-level precision, large-amount overflow, stacked coupons, empty lists, negative quantities, and rounding after currency conversion. A configuration was successful only if all 10 tests passed. Three metrics were also recorded: total tokens, token counts for the four billing components, and an estimated cost based on published unit prices.

2.2 Measurement Caveats

If these rules are not followed carefully, later comparisons become meaningless:

• When command-line tools use non-interactive mode with structured JSON output, the cost field in the usage data is calculated locally using published unit prices. When signed in with a subscription plan, this number does not represent the actual amount deducted; it only represents “how much it would cost under pay-as-you-go billing.”
• How subscription quotas are converted into tokens is generally not disclosed. Therefore, this article can only say which setting consumes fewer tokens, not how much subscription quota was saved.
• Caches have TTLs. Running the same configuration again immediately will produce many cache hits, but running it after a long pause may rewrite everything. Therefore, the measurement must use a cold-start policy: wait for the cache to expire or start a new session before each measurement.
• Output tokens include the reasoning process. When comparing outputs, failing to distinguish reasoning tokens from final-output tokens can lead to misjudging configuration differences.

3. Six Configurations and Measured Results

ID	Configuration	Cost index (most expensive = 100)	Hidden tests	Main cost characteristic
A	Flagship model + full repository context + deep thinking	100	10/10	High input and output; large cache-write share
B	Flagship model + full repository context + deep thinking disabled	74	10/10	Output drops most visibly
C	Flagship model + targeted retrieved context + deep thinking disabled	43	10/10	Input volume drops; cache writes drop accordingly
D	Mid-tier model + targeted context + deep thinking disabled	27	10/10	Lower unit price; same input structure
E	Mid-tier model + targeted context + trimmed tool set + session reuse	9	10/10	Cache hit rate improves significantly
F	Lightweight model for localization → mid-tier model for repair → session reuse + merged turns	3	10/10	Clear division of labor; fewest turns and repeated prefixes

Converted into absolute amounts, the estimated cost for the same task dropped from about $0.69 to about $0.02, a difference of more than 30x, while all configurations passed 10/10. It should be emphasized that this is a single-task, single cold-start sample. The magnitude is only for reference; actual numbers vary with model pricing, repository size, and number of turns.

The most informative result is the change in cost structure. In configuration A, cache writes accounted for about 40% to 50% of total cost, while output accounted for about 25% to 30%. As the configuration moved toward F, ordinary input and cache writes both shrank, while output became a relatively larger share. In other words, the cost-reduction order is: first eliminate “rereading,” then eliminate “expensive writing,” and only optimize output last.

4. Why Cache Writes Become the Main Cost in Small Tasks

Use a simple model. Assume the stable prefix for each request is N = 25,000 tokens, and the task requires 1 + k requests in total.

• Without caching: cost = (k + 1) * N * 1.00
• With short-TTL caching: cost = N * 1.25 + k * N * 0.10

For caching to be cheaper, we need:

1.25 + 0.1k < k + 1

This gives k > 0.28. In practice, once there is at least one cache hit, caching is worthwhile. So why do cache writes still dominate small tasks?

Because the write happens when the denominator is small. Take k = 3:

• Without cache: 4 * 25,000 * 1.00 = 100,000 cost units
• With cache: 31,250 cache write + 7,500 for three cache hits = 38,750, where cache writes account for 81%

The fewer the turns, the harder it is to amortize the 1.25N prepayment. Worse, cache invalidation can happen. If the request prefix in the second turn differs from the first, for example because the system prompt inserts “current time 14:03:22,” the tool list order is randomized, a new session ID is generated each turn, or dynamic log tails are truncated, the matching point may move earlier or disappear. That turn then degrades into another full cache write. If this happens two or three times in one task, cache-write cost can easily double, while the user’s subjective experience is: “I did nothing extra, why is it more expensive?”

This leads to a hard rule: prefix stability = cache hit rate = cost. Tool definitions, project context files, and system prompt order should be fixed. Dynamic information such as timestamps, random IDs, and volatile logs should be placed at the end of the prefix or inside user messages, not allowed to pollute the stable prefix.

5. Where Each Configuration Saves and What It Costs

Deep thinking toggle (A → B, about 26% savings). Reasoning chains are billed at the most expensive output price, but single-file, clearly scoped bug fixes usually do not require long-chain reasoning. The judgment rule is simple: does the task require cross-module trade-offs or solution design? Locating “which line calculated the wrong value” does not. Reviewing architectural choices does.

Context feeding method (B → C, about 42% savings). Replacing “put the entire repository into context” with “symbol-based retrieval + on-demand reading” often reduces input volume by an order of magnitude. This provides the largest benefit and almost no quality loss, because the model would not actually read every line of the extra code. The cost is that retrieval must be accurate. It is recommended to maintain a context description file in the project root, documenting module boundaries, build and test commands, coding conventions, and prohibited practices, so the tool wastes less time exploring.

Model tier (C → D, about 37% savings). Bug repair is a patterned task, and a mid-tier model is usually sufficient. But do not overapply this rule. When understanding an unfamiliar large codebase or debugging across services, switching to a lightweight model can create rework costs far above the savings. Rework is the most expensive cost-reduction method.

Trimmed tool set and session reuse (D → E, about 67% savings). Tool definitions are always in the prefix. Whether 30 tools or 6 tools are mounted, every turn pays for them. Keeping only the read, search, and execution tools needed for the current task is pure net benefit. Session reuse directly improves cache hit rate. Continuing the same task within one session is much cheaper than repeatedly opening new sessions and restating the background.

Task orchestration (E → F, about 67% savings). First use a lightweight model for localization, such as reading the stack trace and narrowing down to the file and function, then let a more capable model handle only that function. Pricing “reading code” and “writing the fix” separately puts cost where it matters. Also merge turns. Instead of “change a little, run tests, change again” five times, it is better to provide the complete modification and test commands in one pass.

6. Practical Cost-Reduction Checklist

P0 (zero risk, do first)

• Enable prompt caching and keep the prefix stable: fix system prompt order and tool order; put dynamic variables later.
• Disable the long TTL tier for sessions that do not need it. The long TTL tier, such as the 1-hour tier, has about a 2x premium and should only be used when a long continuous conversation is expected.
• Mount only the tools needed for the current task.
• Maintain a context description file in the project root to reduce exploratory reading.

P1 (low risk, requires measurement)

• Disable deep thinking by default, and enable it explicitly by task type.
• Replace full-repository feeding with symbol-based retrieval and on-demand reading.
• Split complex tasks into three stages: localization with a lightweight model, repair with a mid-tier model, and review with a stronger model.
• Continue the same task within the same session to avoid repeatedly restating background information.

P2 (requires governance mechanisms)

• Set per-task token and cost limits for CI batch jobs. Fail immediately and alert when exceeded.
• Output token counts and estimated cost as build artifacts, and include them in trend dashboards.
• Review prompt template changes: any commit that changes the prefix should explain why.
• Establish a regression baseline. After each configuration change, rerun the same hidden tests. If cost drops but pass rate falls, roll back.

7. Minimal Implementation for Cost Observability

There is no need to introduce a heavyweight platform first. Structured output is enough to start collecting data:

# Non-interactive mode + structured output, saved as JSON
ai-cli -p "Fix the order amount aggregation precision defect, Issue #1024" \
       --output-format json > run-$(date +%s).json
# Extract usage fields. Field names depend on the tool's output.
jq '{
      input:      .usage.input_tokens,
      cache_write:.usage.cache_write_tokens,
      cache_read: .usage.cache_read_tokens,
      output:     .usage.output_tokens,
      cost:       .usage.total_cost_usd
    }' run-*.json

With this breakdown, you can answer useful questions: Is my cost input-heavy or output-heavy? What is my cache hit rate? Which type of task is subsidizing which other type of task? Looking only at one total amount tells you that something is expensive, but not where the expense comes from.

8. Important Boundaries

Subscription billing and pay-as-you-go billing mean different things by “saving money.” Under a subscription, no additional per-token charge is incurred. Cost reduction appears as slower quota consumption and hitting the session usage limit later. Under pay-as-you-go billing, the bill drops directly. Subscription users should focus more on token efficiency; pay-as-you-go users should also pay attention to unit-price selection.

Estimated amounts are not invoices. Local estimates are calculated using published unit prices. The actual conversion relationship with subscription quotas is not public, so the two cannot be converted into each other.

Do not trade acceptance quality for cost. All six configurations in this article passed the same 10 hidden tests. That is the premise of the conclusion. Cost reduction without a fixed acceptance baseline merely transfers cost into rework.

Task scale determines the optimal strategy. The conclusion that “cache writes dominate” applies to small tasks that finish in a few minutes and have few turns. In large refactors lasting hours and involving hundreds of turns, savings from cache hits can far exceed cache-write costs, making cache benefits the dominant factor. Measure your own task distribution first, then choose a strategy.