Code Agent Architecture Explained: A Harness Engineering Perspective
A practical breakdown of how code agents are built, this article argues that an agent's real competitiveness comes less from the underlying model and more from the "harness" around it—loop control, context management, tool interfaces, permission boundaries, and evaluation. It walks through the five-layer architecture, the agent loop, context engineering, tool design, permissions and security, error recovery, model routing and cost control, evaluation and observability, and a maturity model from prototype to production, giving teams a concrete framework for building and improving their own coding agents.
Introduction: What a Source Leak Revealed
Not long ago, the client source code of a well-known intelligent coding tool was accidentally published, triggering wide discussion among developers. To be clear, what leaked was the client program, not the underlying large language model weights. Outside observers could only infer information about the model indirectly, from client-side traces such as system prompts and tool definitions. For many engineering teams, the first instinct was not to watch the spectacle but to reverse-engineer the design: how is a code agent actually organized from an engineering standpoint?
Treating that question as a form of code archaeology leads to a fairly plain conclusion: much of a code agent’s competitiveness comes from the system “outside the model.” That system includes loop control, context management, tool interfaces, permission boundaries, and evaluation mechanisms. The industry has given it a fitting name: harness engineering.
The word “harness” comes from the automotive world, where it refers to bundling, wrapping, and securing scattered wires into an orderly system. A harness provides no power itself, yet it determines whether power can be delivered safely and reliably. Applied to AI applications, the model is the engine that generates tokens, and the harness is the vehicle’s entire electrical system that constrains and delivers the model’s capability.
The term resonates because it names a long-overlooked fact: the same model wrapped in different harnesses can perform very differently. A model with no tools, no feedback, and no boundaries is just a chat box; add a file system, an execution environment, a loop, and constraint rules, and it becomes an engineering assistant capable of completing tasks independently. To understand a code agent, then, is essentially to understand its harness.
1. First, the Boundaries: What Exactly Is a Code Agent?
Before dissecting the implementation, it helps to separate three easily confused concepts.
- Code LLM: Understands intent, generates code, and makes decisions. It can only take text in and produce text out, and knows nothing about a real project.
- Coding assistant (completion form): Predicts and completes the next snippet of code based on the current file context. Interactions are small in scope and latency-sensitive, and usually require no complex loop.
- Code agent (agent form): Can autonomously plan steps, call tools to read and write files, run commands, observe results, and keep correcting itself until the task is done or it actively asks for help. Its goal is to “complete a task,” not merely “complete a snippet of code.”
Put differently, a code agent embeds the model in a “perceive–decide–act–observe” loop. The model sets the ceiling of its capability; the harness determines its actual performance.
2. Overall Architecture: The Five Parts of a Code Agent
An engineered code agent is typically built from five layers, from bottom to top:
- Model layer: Provides reasoning and code-generation capability. It can route between a “fast model” and a “deep-reasoning model” according to task complexity.
- Agent loop layer: Drives the model to repeatedly “think–call a tool–observe the result” until it produces a final answer. It handles termination decisions and budget control.
- Context layer: Manages what the model “sees” on each request—system instructions, project conventions, retrieved code, conversation history, and the compression and trimming of all of it.
- Tool layer: Packages capabilities such as file read/write, content search, and command execution into interfaces the model can call, and normalizes their inputs and outputs.
- Support layer: Includes permissions and sandboxing, model routing and caching, logging and tracing, and an evaluation system. These do not face the user directly, yet they determine whether the system can scale and run sustainably.
These five layers are connected by a series of “contracts.” The contract between the model and tools is the tool descriptions and parameter schemas; the contract between the loop and the model is a structured message protocol; the contract between the support layer and the others is log fields and metric definitions. Most harness engineering work is designing, maintaining, and continually refining these contracts.
3. The Core Engine: How the Agent Loop Runs
The agent loop is the “heart” of the whole system. A minimal working loop is often only a few dozen lines of pseudocode, yet it contains all the key elements:
Initialize the message list messages:- system instructions (role, behavioral rules, safety boundaries)- project context (tech stack, directory structure, summary of convention files)- the user's task descriptionAttach a tools parameter: name, description, and parameter schema for each toolfor turn in 1..MAX_TURNS:call LLM(messages, tools)if the model returns text directly:output the final answer, and finishif the model returns a tool call:for each call:permission check -> execute (timeout, sandbox) -> truncate result -> append a tool messageif no explicit requirement was given:append a prompt to prevent long silences
Several design points hide in this loop—easily overlooked, yet they directly shape the experience:
- Termination conditions must be explicit. At minimum they should cover four cases: the model proactively gives a conclusion, the task is judged complete, the maximum number of turns is reached, and cumulative tokens or elapsed time exceed the budget. Missing any one of them leads to either a “money-burning spin” or a loop that “never finishes.”
- Tool results must be fed back promptly. An execution result the model never sees is as good as not executed. When writing results back, truncate them—for example, keep the head and tail and elide the middle, with a note such as “N lines total, X lines omitted.” Otherwise a single wide-ranging log or search can blow up the context.
- The loop needs a sense of rhythm. Typically, it first spends a few turns scouting quickly (listing directories, reading key files), then makes changes in a focused burst, and finally verifies everything together. This matches the habit of a human engineer: understand the codebase before making changes.
- The right to stop must be returned to the model. For tasks that are highly ambiguous, high-risk, or beyond its ability, the system should allow the model to proactively say “I need more information” or “I suggest confirming the approach first,” rather than pushing ahead blindly.
4. Context Engineering: The Agent’s Workbench
If the loop is the engine, context is the agent’s workbench. What sits on the workbench, and how it is arranged, directly determines whether the agent is efficient or chaotic.
4.1 Context is a budget that must be budgeted carefully
Every request sends the system instructions, project conventions, retrieved snippets, and conversation history to the model together, while the context window and cost are both limited. A common allocation approach in practice: system instructions and tool definitions take a fixed share of overhead; project conventions and retrieval results are injected on demand; and conversation history grows with each turn, so it must be actively compressed.
4.2 Context should be organized in layers
A relatively clean layering is:
- Stable layer: role definitions, behavioral rules, safety rules—constant throughout the session;
- Project layer: tech stack, directory structure, build and test commands, coding conventions—usually drawn from convention files at the repository root;
- Task layer: the current goal, relevant file contents, retrieved snippets;
- Ephemeral layer: tool returns, error messages, intermediate conclusions.
The point of layering is that different layers have different update frequencies and eviction policies: the stable layer can be reused via caching, while the ephemeral layer can be discarded as soon as it is used.
4.3 Externalize context to the file system
Rather than cramming everything into the conversation, let the agent write plans, progress, and conclusions into files. For example, use a task-list file to record to-dos, checking off each item as it is completed and appending the key conclusion. This approach brings at least three benefits:
- progress is not lost when the context is compressed or reset;
- the user can inspect and intervene at any time;
- multiple sessions can connect naturally.
Essentially, this migrates “memory” from the volatile conversation window to a persistent medium that humans can also read and write.
4.4 Automatic compression: what to do when context fills up
When the history approaches the budget ceiling, a summarization-and-compression step can be triggered: fold earlier conversation into structured key points—such as “goal, completed, remaining, key files, verified conclusions, rejected approaches”—and keep the original text of the most recent few turns. Illustrative pseudocode:
if current token estimate > budget x 80%:summary = LLM(old conversation + compression instruction)keep = raw messages of the last k turnspinned = files and constraints the user explicitly specifiedmessages = [system instructions, summary, pinned content, kept messages]
When compressing, pay special attention to a list of “information that must not be lost”: requirements the user stated explicitly, approaches already confirmed as passing or failing, and the original text of key errors. Once this information is blurred by a summary, the agent is prone to repeating mistakes.
4.5 Retrieval should serve the task
Faced with a large repository, stuffing all the code into the context is neither realistic nor necessary. A common approach is to first retrieve relevant symbols, files, and call chains based on the task description, then expand them on demand. Retrieval results should carry as much “locatable information” as possible—file path, symbol name, and line number—so the model can jump precisely afterward.
5. Tool Design: The Agent’s Hands and Eyes
The model itself cannot see files or run commands; all external interaction depends on tools. The quality of tool design is often the boundary of the agent’s capability.
5.1 Keep the tool set small and precise
Four core tools are common:
- Read file: supports a specified range and returns content with line numbers;
- Precise edit: replaces based on a unique string, rather than rewriting the whole file;
- Content search: searches the repository by regex or symbol name;
- Command execution: runs build, test, lint, and similar commands.
Beyond these, there is a class of “workflow” tools used to maintain a task list, letting the model explicitly break a long task into checkable steps. The value of such tools lies not in execution but in turning an implicit plan into observable state.
5.2 Precise editing is more reliable than rewriting whole files
Having the model output an entire file wastes tokens and easily loses existing content or introduces unrelated changes. An editing approach based on “uniquely matched strings” limits the model’s output to a small diff, and pairing it with feedback such as “report an error and offer candidate locations when no match is found” can significantly reduce the error rate.
5.3 Tool descriptions are prompts
This is a point many teams overlook: which tool the model chooses to call, and what parameters it passes, depends entirely on the tool descriptions and parameter explanations. A vague description (“process a file”) and a precise one (“read a specified line range of the file at a given path, defaulting to the first 200 lines”) can produce behavior that differs by an order of magnitude.
Tool descriptions should therefore be written as carefully as API documentation: state the purpose, when to use it, when not to use it, the return format, and common errors. Parameter schemas should specify types and value constraints, and defaults should be stated clearly in the documentation.
5.4 Output should be structured and truncatable
Tool output goes directly into the context, so the format must be tidy: paths in a consistent format, line-number prefixes aligned, and error messages that include an error code and a suggested action. There must also be a built-in truncation strategy that clearly states in the result “whether it was truncated and how much,” so the model knows the information is incomplete.
6. Permissions and Security: Brakes for Capability
The stronger the capability, the more important the boundaries. An agent that can read and write any file and run any command is nearly unacceptable in an enterprise environment.
6.1 Grade by risk, rather than blocking everything
Operations can be divided into risk levels by their scope of impact:
| Operation type | Examples | Default policy |
|---|---|---|
| Read-only | reading files, searching, viewing logs | allow directly |
| Local write | creating or modifying files within the working directory | allow, keep a change record |
| Command execution | running tests, builds, installing dependencies | allow within the directory; confirm for commands outside it or destructive ones |
| High-risk | deleting data, changing permissions, publishing externally, accessing production | require explicit confirmation or forbid |
The purpose of grading is to keep “low-risk, high-frequency” operations smooth, concentrating human attention on the actions that genuinely carry risk.
6.2 Double validation of paths and commands
- Path validation: restrict all reads and writes to allowed root directories, and normalize inputs that contain relative-path traversal before judging them;
- Command validation: escape external input spliced into commands, prefer parameterized calls over string concatenation, and block obvious destructive patterns;
- Least privilege: run the execution environment under a restricted account, in a container or isolated sandbox when necessary, with network access limited.
6.3 Changes must be traceable
Every file modification should leave a record: before-and-after content, the source of the operation, and a timestamp. This both supports “undoing this change” and makes it quick to locate which step introduced a defect when something goes wrong. In particular, sensitive fields such as credentials and keys in logs must be redacted.
7. Error Recovery: The Real Dividing Line
Whether an agent can write code tests the model; whether it can fix broken code tests the harness. In practice, most task failures are not because the model “can’t do it,” but because error information was not effectively conveyed, or because the loop lacks a convergence mechanism.
Several proven practices:
- Hand the model the full original error text. Compile errors, failed test assertions, and stack traces should be preserved as close to the original text and location as possible, rather than being converted into a single “execution failed.”
- Verify before delivering. After any code change, automatically run the build, tests, or static checks and feed the result into the next turn. This is equivalent to giving the agent an objective “grader.”
- Limit the convergence of retries. When the same command fails repeatedly, reduce the probability of retrying it and instead require the model to change its approach: search more, split the task, or ask the user.
- Distinguish “technical failure” from “wrong direction.” The former (tool timeouts, missing environment) can be retried; the latter (misunderstanding the requirement) must stop for clarification, or retrying will only waste the budget.
8. Model Routing and Cost Control
Different stages have very different demands on model capability. Using an expensive model for “design decisions and complex refactoring” and a fast model for “search, summarization, and simple edits” is a highly cost-efficient strategy. Illustrative routing rules might read:
if turn <= 2 or tool type is search/summary:use the fast modelelse if the task involves cross-file refactoring or is marked complex:use the deep-reasoning modelelse:use the default model
Combined with two general optimizations, costs can drop further: first, enable prefix caching for stable system instructions and project conventions; second, merge tool calls that can run in parallel into a single request. In addition, setting a token and time budget for each task is the last gate against a “single task spinning out of control.”
9. Evaluation and Observability: Making Optimization Evidence-Based
Iterating on harness engineering cannot rely on gut feeling. A sustainably optimizable agent system should have at least two things.
First, traceable execution records. Every turn should leave behind: an input summary, the model chosen, tool calls and their result status, token consumption, elapsed time, and the termination reason. With these fields, one can answer questions such as “why did the task fail,” “which step spent the cost,” and “which tool has the highest error rate.”
Second, a repeatable evaluation set. Accumulate a batch of tasks from real usage scenarios, each with clear acceptance criteria—for example, “all tests pass,” “the build succeeds,” “static checks raise no new warnings.” After each adjustment to the system instructions, tool descriptions, or loop parameters, run a regression on the same batch and track a few core metrics:
- task success rate;
- average number of interaction turns;
- tool-call failure rate;
- average token consumption and average elapsed time per task.
Notably, changes to tool descriptions should also be included in the regression scope. A single change in wording can significantly alter how often a certain tool gets called.
10. Maturity Levels: From Prototype to Production
Combining the analysis above, the engineering maturity of a code agent can be roughly divided into four stages, useful for teams to self-assess and plan their evolution path:
| Stage | Key characteristics | Typical shortcoming |
|---|---|---|
| L1 Prototype | single- or multi-turn conversation, no tools or very few | cannot touch a real project |
| L2 Usable | file read/write, search, and execution, with a basic loop and budget | context easily overflows, weak error recovery |
| L3 Reliable | layered context and auto-compression, tiered permissions, test-feedback loop | cost and latency lack fine-grained governance |
| L4 Scaled | model routing and caching, full tracing and regression evaluation, traceable changes | requires ongoing investment to maintain evaluation assets |
Most teams actually sit between L2 and L3. The key to moving from L2 to L3 is not switching to a stronger model, but completing three things: context compression, tiered permissions, and test feedback. Moving from L3 to L4 depends more on building up data and evaluation systems.
Conclusion: Competitiveness Comes from the System, Not a Single Prompt
Back to the original question: how is a code agent implemented? The answer can be condensed into one sentence: the model sets the ceiling, the harness sets the floor, and production cares about the floor.
Concretely: the loop turns a single generation into a convergent workflow; context ensures the model always “sees the right information”; tools translate intent onto real files and commands; permissions make the capability deliverable; and evaluation keeps optimization from regressing.
This also explains why “harness engineering” resonates so widely. Rather than hoping some model version will suddenly become omnipotent, it is better to repair the reins, the track, and the guardrails one by one. The capability of the base model matters too, of course. As cloud platforms such as Baidu AI Cloud continue to upgrade their model services, developers can spend more energy on these “outside-the-model” system designs when building their own coding agents.
The test for whether a code agent is truly mature is not complicated: run several dozen tasks in a row on a real project, and see whether it can complete them reliably, self-correct after errors, keep the process observable, and keep costs predictable. A system that passes all four checks has truly installed its harness properly.
