Engineering Observability for Context Consumption in AI Coding

A Total Bill Does Not Tell You What to Cut

Heavy use of an AI coding assistant often looks like this: the conversation is not long, yet the context is already near its limit, or the same kind of task burns a very different number of tokens from one run to the next. If all you can see is “this session used N tokens,” optimization is guesswork. Was the system instruction too long? Were mounted capability descriptions bloated? Did one tool call dump a large result back into the context?

That is the same problem as an electricity bill that shows only total usage. You can tell it is expensive, but you cannot decide which line to change first. Engineering work starts with attribution, then trimming. Observation itself does not reduce consumption. It turns invisible occupancy into a list you can rank.

Two costs need to stay separate. Upfront occupancy is already in context before the user asks a real question: system instructions, capability lists, tool parameter descriptions, and role settings. Incremental occupancy comes from conversation turns, tool results, code snippets, error logs, and multi-round revisions. Many people watch only the second class and miss that the first class often sets the ceiling on the remaining budget.

The Fixed Tenants in a Context Window

Treat a coding session’s context as a finite room, not an infinite scratchpad. The room usually holds these fixed contents at the same time:

• Task and constraint instructions: coding standards, output format, safety boundaries, and repository conventions.
• Capability descriptions: names of callable skills, when they apply, and what triggers them.
• Tool contracts: parameters, return fields, and error semantics for each external capability.
• Agent personas: if agents can run subtasks on their own, their goals, permissions, and handoff formats all take space.
• History: files already read, summaries of commands already run, and long outputs that were never folded.

The more you mount, the higher the upfront occupancy. That is not a linear “more features are better” relationship. Every extra description is reread on each inference. Vague, repeated, or overly broad descriptions also trigger unnecessary tool calls, turning an upfront cost into multi-turn incremental cost.

So the first optimization question is not “can we add another capability?” It is “must this capability stay resident for this session?”

Attribute by Four Dimensions, Not by Unplugging at Random

You do not need a particular product’s command names to split consumption. Engineering-wise, it is enough to aggregate by source category. Four dimensions explain most of what you see in practice.

1. Skill packs: load on demand, but they stay once loaded

A skill pack wraps instructions, checklists, and a set of tool usages into a reusable flow. Its value is less repeated prompting. The risk is that once loaded, the prose stays in context even when the user does not need that skill on this turn.

A skill that should stay resident needs all three traits: high frequency, a short description, and a low cost if it is triggered by mistake. Low-frequency, heavy, strongly domain-specific skills should load late, after routing is clear, and should be removed from later turns or folded into a very short summary when the task ends.

Treat the description as code. Drop marketing adjectives. Keep only trigger conditions, inputs and outputs, and invariants. If a skill note cannot say, in a few dozen words, when to use it and what to do when not to use it, it usually does not belong in the default context.

2. Sub-agents: every dispatch carries a calling-card cost

Sub-agents fit subtasks with a clear boundary, such as read-only retrieval, test execution, or drafting documentation. Each agent at least carries a goal, a permission scope, and a return format. As the number of agents grows, those upfront notes stack into a fixed tax.

The quieter cost is the handoff. The main session copies a task description to the agent, and the agent copies a long result back. If the return is a whole file or a whole log instead of a conclusion plus a location, the context fills with evidence rather than decisions.

Controls can be concrete:

• Limit the paths, commands, and data the agent can see, so it returns less material just to “find its own context.”
• Require a return shape: conclusion, where the key evidence is, and open questions. Do not paste full text by default.
• If the main session can finish the job with one tool call, do not wrap it in another agent.
• Parallel agents should share one short task brief, not each write a long prompt of their own.

3. External tools: descriptions spend budget; calls spend turns

Exposing databases, APIs, filesystems, or issue trackers to the model through a common tool connection is one of the highest-leverage parts of AI coding, and one of the easiest ways for cost to creep up. The cost has two parts. The tool list and parameter notes occupy space on every turn. After a real call, the response body enters history.

A tool description should answer three things: what it can do, what it cannot do, and how failure is expressed. Do not put internal implementation, compatibility history, or a full example catalog into the default description. Put examples in on-demand docs. Keep one minimal example in the default text.

On the return side, set a result budget: cap list queries by row count and fields; aggregate logs before sampling; give file path, symbol, and diff first, then expand a snippet only if needed. Treat a raw dump as the exception path, not the default path.

4. Plugins and extensions: convenience widens both the attack surface and the context surface

Plugins package skills, tools, and UI entry points. The usual problem is not that a plugin “does something bad.” It is that several plugins declare similar capabilities, the model cannot tell which to use, and descriptions, probe calls, and failed retries pile up.

Govern with an allowlist, not “mount whatever is installed.” Configure the extension set by repository or task type. Keep one of each overlapping capability. Record, for each extension, whether recent sessions actually called it. Extensions with long-term zero calls should leave the default set. Unused dependencies do not belong in a production image just because they might be useful later. The same rule applies here.

The Observation Loop: See, Rank, Then Change One Thing

Without a breakdown, people change the prompt, switch the model, and turn plugins off at the same time, then cannot tell which change worked. A steadier order is:

1. Sample: for typical tasks—read code and change one place, add tests, look up an API, fix a build—record per-source occupancy for a session each, not only the peak run.
2. Split fixed and variable: fixed is the instructions and tool list already present when the session starts; variable is user messages, tool results, and model replies.
3. Rank: sort by occupancy × frequency of use × how safely it can be removed. Large descriptions that are rarely called should leave the default context first.
4. Change one variable: remove or shorten one source at a time, then rerun the same task set. Changing several things at once invalidates the conclusion.
5. Set thresholds: for example, block new default extensions once upfront occupancy passes some share of the budget, and truncate or summarize any single tool return over a threshold. Set thresholds from your own task mix. Do not copy someone else’s numbers.

A recording shape can stay simple. It is not a platform API. It is how a team books one session internally:

session_id
task_type
budget_limit
fixed:
  instructions
  skills
  agent_profiles
  tool_schemas
  extensions
variable:
  user_messages
  tool_results
  model_replies
decisions:
  loaded_but_unused
  truncated_results
  removed_next_round

“Loaded but unused” is the column with the most action in it. It shows which notes are only sitting there.

Mistakes That Are Easy to Make While Optimizing

Compress only the prompt the user sees, and leave tool descriptions alone. The user prompt is often a small slice of incremental cost. Tool contracts and skill notes are rent paid on every turn.

Write a longer system instruction to forbid waste. Spending extra tokens to tell the model “do not waste tokens” often loses. Put constraints in parameter checks and return truncation, not in another paragraph of prose.

Turn the summary into a second copy of the original. If the summary keeps every field and every log line, it is not a summary. A summary should support the next decision: which symbols to change, which check to run, and what information is still missing.

Never fold the session. Long tasks should settle in stages: write confirmed conclusions into a short state and drop failed output from exploration. Otherwise the second half of inference is reading noise from the first half.

Treat one successful run as the cost model. Interactively fixing one function has a completely different cost shape from batch-editing many modules and rerunning tests. Look at fixed cost and the amplification factor by task type.

How This Connects to Existing Engineering Systems

Token attribution should not live only in the chat window. It is the same class of problem as other engineering observability: a total with no dimensions cannot alert, and it cannot support a review.

Send the breakdown into a general logging and metrics system. Aggregate by project, task type, and extension set, not by how a single chat felt. Signals worth watching include a rising share of upfront occupancy, a high percentile on one tool’s response body, an extension whose load count is far above its call count, and repeated spend from failed retries. Alerts only need to follow those trends. They do not need to bind to a vendor’s proprietary field names.

If the session runs in a container or elastic compute environment, read token cost next to task duration and retry count. A strategy that saves tokens can add round trips and make total latency worse. A bit more structured context can sometimes avoid a full rework cycle. The target is the total cost of finishing the task, not the minimum tokens on a single turn.

Tighten permissions at the same time. The data an external tool can read sets how much sensitive content can appear in a return body. Least privilege is both a security control and a context control: a large table the tool cannot read will not be stuffed whole into the session.

A Team Agreement You Can Actually Check

Turn the principles above into agreements a team can review. That is more stable than chasing one interactive command:

• Declare the default extension set per repository. In review, look at the length of new descriptions and overlap with capabilities you already have.
• Version skill and tool descriptions. A change should say what was removed, not only what was added.
• Sub-agents must declare a return cap and content types they must not send back.
• Tool results pass a field allowlist or a length truncation before they enter the session.
• Each week, rerun a fixed task set for upfront occupancy and remove “loaded but unused” items from the default set.
• In reviews, look at success rate, rework rounds, and the token breakdown together. Do not reward only “a shorter prompt.”

Once those agreements exist, context consumption stops being folklore. You still cannot predict every token before the first line is written. You can know where the bulk is, which item is idling, and what to change next. That is what observability means for AI coding: give the bill line items first, then give optimization an order.