Coding Agent Context Engineering: How the Message Pipeline Builds, Injects, and Caches a Request

1. Starting Point: What the Model Actually Receives in One Request

Coding assistants differ from ordinary chat assistants in one essential way: they are not answering questions, they are running a persistent loop that repeatedly observes, thinks, calls tools, and observes again. Every iteration reassembles the complete context from scratch and sends it to the model.

That context usually has three parts:

Part	Purpose	Typical share (empirical)
Identity and environment (System)	Tells the model who it is, what environment it works in, and what rules to follow	~30%
Message list (Messages)	Carries “what has already happened”: user input, model replies, tool calls and their results	~60%
Tool definitions (Tools)	Declares which capabilities are callable and what the parameters look like, as structured schemas	~10%

The message list is both the largest part and the fastest-changing one: every turn appends to it. More importantly, beyond the text the user typed, it contains a great deal of information the user never sees but the model does.

So the design of the message list has to answer three engineering questions at once:

1. Semantic correctness: internally you need to express far more than the roles the model protocol supports, so a mapping layer is mandatory;
2. Information completeness: what must be injected, what must not, and how injected content can be recognized by the model as coming from a different source;
3. Cost control: context only grows monotonically, so how do you reduce token and cache cost without damaging the model’s understanding.

2. Three Layers: Message → Content Part → Cache Boundary

The first design principle: do not carry a message in a single flat string. Mature implementations split a message into an ordered array of content parts, each with a clear type and responsibility:

• Text parts: the user’s own words, model output, synthesized notes;
• Image parts: screenshots, diagrams, error screens;
• Tool call parts: which tool the model decided to call, with what arguments, and under which call id;
• Tool result parts: execution output linked back by call id (stdout, file contents, retrieved snippets, error messages);
• Reminder parts: context supplements injected by the system rather than typed by the user.

Once messages are split into parts, three capabilities become possible: per-part token budgeting and degradation (an oversized image becomes a text placeholder), per-part cleanup and truncation (old tool results are replaced wholesale with placeholders), and per-part cache boundaries (a breakpoint placed after a specific part). None of these can be implemented cleanly on a flat string.

The internal data structure looks roughly like this (illustrative pseudocode):

type InternalRole = 'user' | 'assistant' | 'tool' | 'system' | 'meta';
type Part =
  | { kind: 'text';       text: string }
  | { kind: 'image';      ref: string; mime: string }
  | { kind: 'toolCall';   id: string; name: string; input: unknown }
  | { kind: 'toolResult'; id: string; output: string; isError?: boolean }
  | { kind: 'reminder';   tag: string; text: string };
interface InternalMessage {
  id: string;
  role: InternalRole;
  parts: Part[];
  persisted: boolean;   // whether this is written to conversation history
  cacheable: boolean;   // whether it may participate in cache hits
  createdAt: number;
}

Note the persisted and cacheable flags: they separate “conversation facts” from “one-off context supplements”, and they are the foundation for every cost and caching decision that follows.

3. Five to Two: Five Internal Message Types, Two Protocol Roles

Mainstream model protocols are minimal about messages: typically only user and assistant roles plus a set of content parts. A coding agent needs to express much more internally:

Internal type	What it carries	Maps to protocol role
user	Real user input (including expanded attachment references)	user
assistant	Model output: prose, reasoning, tool call intent	assistant
tool	Tool execution result (success / failure / rejected by the user)	user (carried as a tool result part, aligned by call id)
system	System reminders, environment info, permission and mode descriptions	user (assembled per request)
meta	Compaction summaries, subtask write-backs, recovery records	assistant (synthesized reply, keeping the turn structure well-formed)

The mapping is not optional, and the reasons are practical:

• A tool result is an observation of the external world; semantically it belongs on the input side, not in things the model itself said;
• System reminders were not said by the user, and if they are mixed into user text, the model tends to treat the injected content as a user instruction;
• A post-compaction summary must occupy an assistant reply slot in the protocol, otherwise the turn structure is invalid.

3.1 The Seven Actions of the Sanitization Pipeline

Mapping is only the first step. Turning an “internal message list” into a “valid request body” is the job of a sanitization pipeline:

1. Drop empty messages: filter out whitespace-only messages with no valid parts, avoiding empty-content errors at the protocol layer;
2. Merge adjacent same-role messages: collapse consecutive messages of the same role into one multi-part message, reducing protocol-level redundancy;
3. Balance calls: every tool call part must have a matching tool result part with the same id; unclosed calls get a placeholder result (“call did not complete / was cancelled”), otherwise the request is rejected outright — this is the most frequent protocol error in practice;
4. Handle reasoning parts: the model’s intermediate reasoning is usually sent back only for the current request; earlier turns are stripped or trimmed on demand, saving tokens and preventing stale reasoning from misleading later decisions;
5. Degrade oversized media: image parts over budget are replaced with text placeholders;
6. Validate head, tail, and ordering: the first message must be a user message, and a tool result part must immediately follow its call;
7. Budget truncation: drop by priority, not by position — protect the task goal, the most recent turns, and all unclosed tool calls.

Illustrative code:

function toApiMessages(msgs: InternalMessage[], budget: TokenBudget) {
  const out = [];
  for (const m of normalize(compact(msgs))) {      // filter → merge → balance
    const parts = m.parts
      .filter(p => budget.allow(p))
      .map(toApiPart)
      .filter(Boolean);
    if (parts.length === 0) continue;              // empty messages never enter the request
    out.push({ role: mapRole(m.role), content: parts });
  }
  assertWellFormed(out);                            // balance and ordering checks
  return out;
}

In one sentence: the five internal types exist for expressiveness, the two protocol roles exist for compatibility, the sanitization pipeline is the translation layer between them, and balancing validation is the most overlooked and most failure-prone link in that pipeline.

4. Hidden Injections: Who Else Ends Up in the Message List Besides the Keyboard

The user sees a chat box; the model sees a thickened context. Common injection sources fall into six categories:

Injection category	Typical content	Persisted?	Change frequency
Identity and environment	Working directory, OS, shell, model name, permission mode	Usually not	Every turn
Project convention files	Context/instruction files in the project root and nested directories (coding conventions, build commands, directory layout)	Partly	First load + on match
Editor context	Currently open file, selected snippet, editor diagnostics	Usually not	As the user works
Implicit user input	Expanded file/directory references, expanded command-style directives, long pasted text	Persisted	Every turn
System reminders	Todo state, plan-mode notices, warnings that a file changed externally, long-output notices	Usually not	On demand
Tool results	File reads, content search, command execution, subtask write-backs	Persisted	Every tool call

Two design points here are routinely underestimated.

First, injected content must be distinguishable. Everything not typed by the user should be wrapped in a container with a source tag that tells the model plainly: “this is environment info / tool output / a system reminder, not your user giving an instruction.” Once injected text is indistinguishable from the user’s own words, you get both instruction confusion and a manipulation risk — text read from file contents or command output is fundamentally untrusted data and must be labeled as data, not as instructions.

Second, “persist or not” is a cost decision. Writing environment info into persistent history keeps it present for the whole conversation (it costs tokens, but helps cache stability); assembling it dynamically on each request saves tokens but breaks prefix stability. The engineering rule of thumb is change frequency: one-off, current-turn-only content (selected code, external-modification warnings) suits non-persistence; long-lived project conventions suit persistence or should be baked into the system setup.

5. Caching Strategy: Why Old Tool Results Can Be Cleaned Up Without Breaking

5.1 The Basic Constraint of Prefix Caching

Model services commonly offer context caching: the server matches the longest common prefix of the request, skips recomputation for the matched portion, and computes only the new part. Writing the cache the first time also costs something. Three hard rules follow:

• Append only: the cache hits fully, marginal cost is lowest;
• Change any character in the middle of the prefix: everything after the change point is invalidated;
• Change the tail: the invalidation range is smallest.

So the goal of a caching strategy is not “send fewer tokens” but make changes happen as late as possible and keep the prefix byte-for-byte stable.

5.2 Three Ways to Clean Up Old Tool Results

Approach one: fixed-length placeholder replacement. Replace an early tool result wholesale with a short fixed string (e.g., “This tool output has been cleared; call the tool again if you need it”) rather than deleting the message. Three benefits: the message structure is unchanged and the protocol stays valid; what gets replaced is earlier history, so the invalidation range is small and predictable; and the call/result balancing relationship is preserved.

Approach two: sliding-window retention. Keep the raw text of only the most recent N tool results and clear earlier ones per approach one. Choose N against task characteristics: file-read results usually need a longer retention window, while search and command output can be cleared sooner.

Approach three: externalize and re-read on demand. Very long outputs (whole files, build logs, test reports) do not enter the context directly; they land in a temp file or object storage, and the message keeps only a summary plus a handle (path, object id, line count, how to read it again). The model explicitly reads it back when it needs details. This decouples “context length” from “information availability” and is the most effective way to keep the message list from ballooning.

5.3 Where to Place Cache Breakpoints

In protocols that support cache markers, breakpoints can be placed on content parts. Empirically:

• Put one at the tail of the system setup and one at the tail of the tool definitions (these two are the most stable and hit most often);
• Put one every K turns in the history region, so the whole history is not recomputed because of one small change;
• Put the final breakpoint just before the current turn, covering the part that is already stable.

Each breakpoint costs one cache write, so more breakpoints is not better; you balance hit rate against write cost.

5.4 Compaction: One Large Invalidation for a Long-Term Smaller Context

When the message list approaches the budget ceiling, the usual move is a full compaction: summarize history into a structured summary (goal, completed items, key conclusions, outstanding todos, important file paths) and replace the raw history with it. This invalidates the cache entirely once, in exchange for every subsequent turn running on a small context. So timing matters:

• Run it at a natural breakpoint (a task finished, the user switched goals), not in the middle of a tool-calling round;
• Set the trigger threshold at a high-water mark of the budget (e.g., 80%) instead of waiting for overflow;
• The summary must explicitly preserve the task goal and unclosed items — losing the goal during compaction is the most typical failure mode of long-session applications.

5.5 One Counter-Intuitive Detail

Move volatile information out of the prefix. If timestamps, random session ids, or live status appear in the system setup or early messages, the prefix differs on every request and the cache never hits again. They belong at the end of the message list, or outside the prose entirely, passed as parameters.

6. One Full Turn

Stringing the pieces together, a single loop looks like this:

async function runTurn(input: UserInput) {
  const userMsg = materialize(input);        // expand references, commands, attachments
  history.push(userMsg);                      // persist
  injectContextOnce();                        // project conventions, environment info (first time / on change)
  while (true) {
    const reminders = collectReminders();     // editor selection, external edits, todo state
    const request = buildRequest(history, reminders, budget);
    const reply = await model.stream(request); // text + tool calls
    history.push(reply);
    if (reply.toolCalls.length === 0) break;
    for (const call of reply.toolCalls) {
      const result = await execute(call);      // constrained by permissions and mode
      history.push(toToolResultMessage(call.id, result));
    }
    trimOldToolOutputs(history);               // fixed-length placeholders / externalization
    maybeCompact(history);                     // compact at the high-water mark
  }
}

Three easy mistakes: when to collect reminders (before building the request, not re-injected on every round), when to persist tool results (execute first, then persist, so no unclosed call is left behind), and the order of cleanup and compaction (clean first, then decide whether compaction is still needed, to avoid invalidating the cache twice).

7. Practical Checklist

1. Model messages as arrays of parts, never as concatenated strings;
2. Balance tool calls and results, and assert it at the pipeline exit;
3. Tag all non-user content with its source, and treat external data as untrusted;
4. Distinguish “persisted” from “one-off injection”, and decide by change frequency;
5. Put volatile information at the tail or outside the prose to protect prefix stability;
6. Clean old tool results with fixed-length placeholders + sliding-window retention + externalizing long output;
7. Place cache breakpoints sparsely: one each for the system and tool regions, and history breakpoints at turn intervals;
8. Compaction at natural breakpoints, with the goal and unclosed items always in the summary;
9. Truncate by priority, not by position: protect the goal, recent turns, and unclosed calls;
10. Build observability: per-turn message structure summaries, token counts, cache hit ratio, cleanup and compaction counts, average tool result length per turn. These metrics plug directly into logging and alerting, and they are the only objective basis for judging whether your context engineering is working.

8. Summary

The capability ceiling of a coding agent depends less on the model itself than on the quality of the context you feed it. The three-layer design around the message list — a part-based data structure, type mapping with a sanitization pipeline, and cache-aware cleanup and externalization — determines whether it gets smarter the more it sees, or gets more confused and more expensive the longer it runs.

The one-line takeaway: the goal of the message pipeline is a stable prefix, recognizable injections, and reclaimable growth.