Cutting AI Coding Agent Costs: Six Configurations for the Same JVM Bug Fix, Compared

1. Where the Money in a Coding Agent Actually Goes

A coding agent in CLI form has no memory of its own. Every time it makes a request, it resends the system prompt, the tool definitions, the project convention files, and the entire conversation and code context accumulated so far to the model. When it “remembers” which file it just changed, that is only because the text is still sitting in the context.

To reduce the cost of this repeated sending, cloud LLM providers commonly offer prompt caching. A prefix is written to the cache when it is read for the first time, at a price higher than normal input. Later requests that hit the same prefix are billed at a very low cache-read price. The cache matches prefixes exactly, and cached content has a limited lifetime, commonly offered in minutes-level and hours-level tiers. Any change partway through invalidates everything after it.

A single task’s bill can therefore be broken down into four buckets:

Bucket	Meaning	Relative price (normal input = 1×)
New input	New content that does not hit the cache	1.0×
Cache write (short TTL)	Content read in and stored in the cache for the first time	≈ 1.25×
Cache write (long TTL)	Same, but with a longer lifetime	≈ 2.0×
Cache read	Historical content that hits the cache	≈ 0.05×
Output	Text generated by the model, including its thinking	≈ 5.0×

The table above shows the relative multipliers of a common cloud pricing structure, to illustrate how the buckets compare in cost. For actual unit prices, minimum cacheable length, and TTL tiers, refer to the official price list of the platform you use.

This table explains two things. First, output is the most expensive, so “making the model write less filler” is effective. Second, a cache write costs more than new input, so repeated cache invalidation and rewriting is a hidden waste that is easy to overlook.

2. How the Test Was Designed for a Fair Comparison

There is plenty of cost-saving advice online, but few people put the cost of the same task under several settings side by side. To make the numbers comparable, the following variables were fixed:

• Task: Fixing a bug in a JVM server-side project (an order-amount calculation module). The bug is a real behavioral deviation involving amount accumulation and rounding boundaries. It can be spotted by reading the code, but it cannot be fixed by guessing.
• Acceptance: 10 hidden unit tests, run once before the fix and once after. Only groups that pass all of them count as valid samples.
• Environment: An LTS release of the JDK, a mainstream dependency build tool (offline repository mode), and a mainstream unit testing framework.
• Execution: A single non-interactive run with structured JSON output, file-edit permissions accepted automatically. A tool whitelist allows only six capabilities: running build/test commands, reading files, writing files, editing, searching by name, and searching by content.
• Sampling: Each setting is run 3 times, and the median is taken. Local cache directories are cleared before each run so that results from different groups do not contaminate one another.

A note on units: The dollar amounts in this article come from the local estimate field in the agent’s structured output, which is calculated from its built-in price list. They reflect the order of magnitude of what the task would cost under pay-as-you-go billing. They are not the actual bill a subscriber pays. Subscribers are not charged extra because of this, but they will hit their usage limits sooner. Vendors have not made public how they convert subscription quotas into tokens. This article can therefore only compare which group consumes fewer tokens. It cannot convert that into “how much quota was saved.”

3. Results Across the Six Settings

#	Key setting	New input	Cache write	Cache read	Output	Estimated cost	Relative to baseline
1	Default baseline (minutes-level TTL, full thinking, one session to the end)	8.0k	90.0k	240.0k	8.0k	≈ 0.69	1.00
2	Switch to hours-level TTL + suppress redundant thinking	6.0k	22.0k	380.0k	3.5k	≈ 0.35	0.51
3	Split the session into locate / fix / verify phases, each carrying only a summary of conclusions	9.0k	12.0k	300.0k	2.5k	≈ 0.24	0.35
4	Slim down the context: compress project convention files, search before reading files, minimize the tool whitelist	4.0k	9.0k	250.0k	2.0k	≈ 0.18	0.26
5	Delegate retrieval: a lightweight model handles code location; the main model only makes judgments and edits code	2.0k	6.0k	120.0k	1.5k	≈ 0.12	0.17
6	All of the above combined + budget gate + output only the minimal patch	0.5k	1.5k	20.0k	0.3k	≈ 0.02	0.03

(Units: thousand tokens. Dollar amounts are the median of 3 runs. Group 6 ranged from 0.02 to 0.06 in a single run and is highly sensitive to whether the bug is fixed on the first try.)

A few key takeaways:

In the baseline group, 65% of the money went to cache writes. This is the most counterintuitive finding. In a small task, the output is only 8k tokens. Yet the context prefix changed repeatedly during the task, and gaps for thinking exceeded the minutes-level TTL. As a result, 90k tokens of cache were rewritten. In other words, you are mainly paying not for “the model doing the work,” but for “the model repeatedly re-reading its own files.”

A longer TTL gives the best return of any single change. The write price rises to 2×, but the number of rewrites drops from more than ten to two or three. As a result, the total cost is cut in half. The test is simple: in a given task, do the intervals between requests often exceed the minutes-level TTL? If there is a build, a test run, or a human review in between, they will.

A longer session does not mean a smarter model. After splitting, each segment has a shorter context and a more stable cache prefix, and the model’s attention is more focused. As a result, it takes fewer rounds to locate the bug.

4. A Practical Cost-Reduction Checklist

1. Context governance: locate first, then read files

The most expensive context is “paste in a whole file.” By default, an agent tends to read entire files, whereas people usually grep first and then jump to the relevant code. Explicitly tell it to use name- or content-based search to get line numbers first, and then read only the relevant snippets:

Location rules: first use content search to pinpoint the suspicious
methods and line numbers, then read only the matching snippets and
their direct callers. Read no more than 200 lines from a single file.
Do not read every source file in a module when there is no clear lead.

Project convention files (architecture constraints, code style, build commands) should be written as short, structured lists, not long documents. They are resent on every turn, so the difference between 200 lines and 20 lines is multiplied by the number of requests.

2. Cache-friendly orchestration: stable prefix, volatile content last

Because the cache matches prefixes, the order and stability of the context affect the bill just as much as its size:

Stable zone (change as little as possible): system prompt → tool definitions → project conventions → directory structure
Semi-stable zone: confirmed key code snippets
Volatile zone (changes every turn): current instructions, latest test output, temporary exploration results

In practice, this means: do not frequently add or remove tools from the whitelist or rewrite the system prompt, because doing so invalidates the entire prefix. Put each turn’s temporary exploration results at the end so they do not pollute the prefix.

3. Use long TTL for long tasks and short TTL for short interactions

Do not always choose the most expensive tier. Use a short TTL for one-off Q&A, and a long TTL for long tasks that span builds and tests. If the platform lets you switch TTL during a session, switch to the long one when you enter the “edit code + verify repeatedly” phase.

4. Split the session by phase and pass on only the conclusions

A three-phase approach is recommended:

1. Location phase: Answer only “where is the bug and why.” Do not write code.
2. Fix phase: Start a new session with only a summary of the previous phase’s conclusions (file + method + root cause, three to five lines), then change the code.
3. Verification phase: Start a new session with only the patch and the test output.

Each phase has a short, stable prefix, so cache hit rates are high. Splitting the session also prevents “a wrong conclusion from repeatedly contaminating later reasoning across a long history.”

5. Control the output: let the tests do the accepting, not the text

Explicitly require “output only the patch and a one-line note; do not repeat the code or explain steps already completed.” Set the thinking budget by task type: keep some thinking for localization tasks, and reduce it for code-editing tasks. Do not turn thinking off across the board, because the output cost you save is often eaten up by extra rounds of rework.

6. Tier the models: lightweight for retrieval, flagship for judgment

Work that involves a lot of volume but shallow judgment, such as code search, log summarization, formatting, and sorting unit test results, can be delegated to a lightweight model (typically around one-tenth the price of a flagship model). Reserve the flagship model for conclusions about where the bug is, patch design, and code review. In the tests, this change alone brought the cost down from 0.18 to 0.12.

7. Budget gates and instrumentation: understand the four buckets before optimizing

Any optimization done without per-bucket statistics is guesswork. We recommend logging the four buckets for every request at the wrapper layer and setting a hard cap:

#!/usr/bin/env bash
# Illustration: replace <agent> with the coding agent command you use
BUDGET=0.30          # Estimated cost cap for this task
: > usage.jsonl
run_agent() {
  local out cost
  out=$(<agent> --print --output-format json "$@")
  cost=$(printf '%s' "$out" | jq -r '.cost_usd // 0')
  printf '%s' "$out" | jq -c '{
      input:      .usage.input_tokens,
      cache_write:.usage.cache_creation_input_tokens,
      cache_read: .usage.cache_read_input_tokens,
      output:     .usage.output_tokens,
      cost_usd:   .cost_usd
  }' >> usage.jsonl
  if awk -v c="$cost" -v b="$BUDGET" 'BEGIN{exit !(c>b)}'; then
    echo "Budget exceeded; stopping further rounds" >&2; exit 2
  fi
  printf '%s' "$out" | jq -r '.result'
}

When running agents in batch in CI, add two more hard limits: a maximum number of rounds, and stop if there is no progress (exit and report if two consecutive rounds show no improvement in the tests). This prevents the budget from being drained by an endless loop.

5. When Optimization Is Not Worth It

• Subscription users: You are not charged extra, so the optimization goal is “hit usage limits less often.” In this case, prioritize cutting cache writes and the number of rounds rather than worrying about the output price.
• One-off script-like small tasks: The time spent on configuration may exceed the money saved. Just run it; do not tune it.
• Architecture tasks that require deep reasoning: Forcing down the thinking budget and context often leads to more rework rounds, and the total cost goes up instead.
• When acceptance is weakened: No cost-saving measure is worth skipping the regression run of the hidden tests. The bottom line for saving money is “still passing all the tests.”

6. Common Misconceptions

1. “Cache reads are cheap, so a longer context doesn’t matter.” Reading is cheap; writing is expensive. Once the prefix of a long context is invalidated, the rewrite cost multiplies.
2. “Turning off thinking always saves money.” Output has the highest unit price, but less thinking often leads to failed localization and multiple retries, so measure it.
3. “A longer context means the model understands more.” Beyond a certain length, attention is diluted and localization accuracy drops. Meanwhile, you pay for that history on every turn.
4. “The cost the tool reports is the bill.” It is an estimate calculated locally from the built-in price list. It works well for comparing settings side by side, but not for financial reconciliation.

The bottom line: In a coding agent’s cost structure, the most expensive thing is not the model’s thinking but repeatedly moving context around. Manage the context well, keep the cache prefix stable, split sessions by phase, and hand retrieval work to a lightweight model. With these changes, the cost of the same Java bug fix can drop from about 0.69 to around 0.02, while the acceptance results stay the same.