Designing a Code Agent Message System: Type Mapping and Context Injection Behind Conversations

1. Introduction: How a Single Input Expands into a Large Context

When using an AI coding agent, the user experience often appears simple: enter a sentence, and the agent starts working. From an engineering perspective, however, the model receives much more than that sentence on every request.

A typical interaction sends a context composed of three major parts:

• System prompt: Defines the model’s identity, behavioral rules, and environment information. It often accounts for roughly 30% of the context.
• Messages: Carries conversation history, user inputs, model responses, and tool results. This is usually the largest section, at around 60%, and changes most frequently because every interaction appends new messages.
• Tool schemas: Declares the tools available to the model and their parameter definitions, accounting for approximately 10%.

Among these components, Messages has the highest information density and the greatest engineering complexity. It contains more than what the user types. It also carries tool outputs, static environment context, synchronized IDE state, expanded attachments, and other hidden information.

Understanding message design is therefore essential to understanding how a Code Agent works.

2. Mapping Five Types to Two Roles: Why a Normalization Pipeline Is Necessary

2.1 The Source of the Type Gap

A typical design challenge is that an agent’s internal model needs to represent rich business semantics. It may define five or more message types, including:

• Plain text messages
• Messages with attachments
• Tool-call messages
• Tool-result messages
• System reminder messages

By contrast, mainstream large language model APIs often accept only two roles: user and assistant. Some protocols also support system, but system instructions are generally handled separately from the main message array.

This means a message normalization pipeline must exist between the internal data model and the API request. Its job is not merely to convert types. It also needs to perform the following operations:

1. Role consolidation: Internal types such as system reminders and environment injections are commonly represented as user messages because the API may not allow system-level instructions to be inserted in the middle of a message stream.
2. Semantic labeling: To help the model distinguish genuine user input from injected environment information, the pipeline adds explicit labels to injected content. For example, it may wrap an injected block in XML-style tags to indicate that the content describes environment state rather than something the user directly said.
3. Structural validation: The pipeline verifies that the message sequence satisfies API constraints. For example, a tool_use block and its corresponding tool_result block must be correctly paired and appear in a valid order. Otherwise, the request may be rejected.

2.2 Why Not Use Only Two Types Internally?

Using the API’s two roles directly inside the agent would create a serious loss of semantic detail. User input, file snapshots, code selected in an IDE, and the output of a scheduled task all have different meanings. If they are represented by the same internal type, later filtering, caching, auditing, and compression become much harder to implement.

Rich internal types combined with a constrained external representation are a common architectural pattern: express semantics internally and adapt them at the boundary. The approach is similar to how a compiler transforms a rich front-end AST into target code for a back end.

3. Content Parts: Fine-Grained Units Within a Message

The content field of a message is not necessarily a single string. It is often an array of content parts, which is a core design feature of modern message protocols.

A part may be:

• A text block: Ordinary textual content.
• An image block: A screenshot, design mockup, or other multimodal input.
• A tool-call block on the assistant side: A tool invocation initiated by the model, including the tool name and arguments.
• A tool-result block on the user side: The output returned after a tool executes, paired with the corresponding call block.

Using an array instead of one large text string provides three important benefits:

1. Multimodal composition: A single message can contain both an explanation and a screenshot, allowing the model to associate the text with the image.
2. Precise localization and replacement: Context compaction and history trimming can operate on individual parts instead of deleting entire messages.
3. Pairing validation: Tool calls and their results can be paired at the block level. This granularity is important for the cache-safe deletion strategy discussed later.

A simplified example looks like this:

{
  "role": "user",
  "content": [
    { "type": "text", "text": "<system-reminder>Current date: 2025-01-15</system-reminder>" },
    { "type": "text", "text": "Help me fix this error" },
    { "type": "image", "source": { "type": "base64", "media_type": "image/png" } }
  ]
}

4. Hidden Injection: Information That Does Not Come from the Keyboard

One of the most overlooked aspects of Messages, and one of the clearest examples of careful engineering, is hidden context injection. These injections generally come from two sources.

4.1 User Context Injection

This information is generated when a session is created or before each request. It commonly includes:

• Project-level instruction files: Project convention or guidance files are read at the beginning of a session and injected into the context so the model can follow team-specific rules automatically.
• Environment snapshots: Metadata such as the current working directory, operating environment, and current date and time.
• Git state: The current branch and a summary of recent commits, helping the model understand the codebase’s recent evolution.

These injections typically use dedicated markers, such as <system-reminder>, to identify their source and prevent the model from treating them as direct user instructions.

4.2 Attachments and IDE State

• Expanded file references: When a user enters something such as @src/main.ts, the agent can expand the file content, or a required line range, before the message enters the processing pipeline. The model can then use the content without making another tool call, saving an interaction round.
• Selected code in the IDE: The currently selected code, cursor position, and active file path can be synchronized into the message stream. This is a key reason conversational programming can understand the developer’s current working context.
• Tool results: The output of every tool call—such as file contents, command execution results, or search matches—is appended to Messages as a tool_result block and becomes the basis for the model’s next decision.

The design principles for hidden injection can be summarized as follows:

Give the model the information it needs, clearly label its source, control its size, and preserve cache efficiency.

5. Cache-Friendly Context Management: Soft-Deleting Old Tool Results

5.1 Why Deletion Can Break the Cache

Large language model inference services commonly support prompt caching. When the message prefix in a new request is identical to the prefix from a previous request, the cached portion can be reused at very low cost, significantly reducing latency and expense.

Because caching generally matches prefixes, changing any content near the beginning of the message sequence can invalidate all cached content that follows it.

Agent workflows, however, frequently need to modify history. Large tool results—such as a file read that returns tens of thousands of lines—should eventually be removed after they have served their purpose. Otherwise, the context can grow rapidly and eventually exceed the model’s context window.

Directly deleting or rewriting historical messages can cause the entire cache to become invalid.

5.2 The In-Place Replacement Strategy

An elegant solution is in-place replacement. Instead of deleting an old message or changing the message structure and order, the agent replaces the old tool result with a very short placeholder, such as:

[File contents were read earlier and are omitted here]

The number of messages, their types, their roles, and the order of their content blocks all remain unchanged. Only the string inside the target block becomes shorter. As a result, the cache can continue matching the stable message structure. Combined with an incremental update mechanism based on block-level edit records—conceptually similar to a cache-edits strategy—previously cached content can be preserved, while only the section after the actual change needs to be recomputed.

This approach has three important advantages:

1. Structural stability: Deletion can break message validity, including the required pairing between tool_use and tool_result. Replacement keeps the structure valid.
2. Prefix stability: The change is confined to the target block instead of modifying the message sequence, maximizing the chance of a cache hit.
3. Semantic continuity: The placeholder tells the model that content previously existed, what it represented, and why it has been omitted. The reasoning chain is not completely broken, and the model can call the tool again if the information is needed.

For agents that support long conversations and multiple rounds of tool calls, cache-aware context management is a central technique for optimizing both cost and user experience. In production, message-service monitoring, alerts, and logs can be used to track cache hit rates and context-size trends, allowing the replacement threshold to be tuned over time.

6. Conclusion

A production-grade Code Agent message system must answer three questions:

• How should information be represented? Use rich internal types, then normalize them into the smaller set of roles supported by the model API.
• How should context be injected? Add environment information, file references, and IDE state as clearly labeled content parts so the model can understand the developer’s working context.
• How should the context evolve? Use cache-friendly techniques such as in-place replacement to control history growth while balancing information completeness and operational cost.

Messages are the communication pipeline between an agent and a model. They carry more than a conversation: they concentrate much of the agent’s engineering context capability in a single structure.

Understanding this roughly 60% portion of the model context reveals much of what drives Code Agent behavior. A later discussion can further explore tool schema design and tool-call orchestration.