Agent Harness Engineering: From a 30-Line Loop to a Layered Architecture
Agent harness engineering treats the model itself as the agent and the surrounding scaffold as the real engineering work. This article explains the minimal agentic loop that stays unchanged at the core, why more than a dozen layers get stacked on top of it, and how the key layers—tool design, context engineering, permissions and safety, and sub-agents—work together. It closes with production concerns such as observability, persistence, and resilient infrastructure, plus six design principles you can apply when building your own agent.
1. A Counterintuitive Starting Point
When it comes to building AI coding agents, a long-standing misconception treats the work as writing large amounts of conditional branching and flow orchestration: using if-else statements to break a task into fixed steps and then plugging a large language model into one of the nodes. Engineers following this approach pour their effort into “control flow,” and the model ends up as a passively driven tool.
Better practice offers the opposite answer: the core of an agent is the model itself. The model is already capable of reasoning, decision-making, and planning. Your job is not to “develop” that intelligence in code, but to build an environment in which it can actually do work—which tools it may use, which files it can see, where the boundaries of its actions lie, and how it coordinates with others. This layer wrapping around the model is commonly called the Harness (the scaffolding or carrier framework).
A fitting analogy is autonomous driving: the model is the car, and the Harness is the driving environment. You don’t need to steer by hand; you install the “in-vehicle brain,” plan the route, load the high-definition map, and set the rules of the road—then the car drives itself. The better the Harness is polished, the more complex and reliable the tasks the model can complete on its own.
This perspective leads to a key engineering conclusion: the ceiling of an agent’s capability is largely determined not by the model alone, but by “model + Harness” together. The same model with a crude Harness can only handle simple question answering, while with a well-built Harness it can autonomously carry out multi-step software engineering tasks.
2. The Core Loop: An Unchanging Kernel
No matter how many mechanisms the Harness eventually accumulates, it starts from an extremely simple loop, known in the industry as the Agentic Loop—essentially an engineering implementation of the ReAct (reason-act) pattern. It has only four actions:
- Send the current context (system prompt + user request + message history) to the model.
- The model returns text, or returns one or more “tool call” requests.
- If there are no tool calls, the task is done—output the result.
- If there are tool calls, execute the tools, append the results back to the context, and return to step 1.
In pseudocode, this kernel is usually only twenty or thirty lines:
def agent_loop(messages):while True:response = model.generate(messages) # Reason: the model decidesif not response.tool_calls:return response.text # No tool calls: converged, exitfor call in response.tool_calls: # Act: execute the toolsresult = execute_tool(call.name, call.arguments)messages.append(tool_result(call.id, result))# Results are written back to the context; next round begins
This loop matters because it embodies the idea of “letting the model drive the control flow”: the direction of execution is decided dynamically by the model based on what it observes at each step, rather than being hard-coded in advance by engineers. The dozen or so layers that follow are all essentially reinforcements built around this unchanging kernel—making it better at using tools, more economical with context, safer, and better at collaboration—while the loop itself never changes a single line.
3. Why Layered: Evolving from 30 Lines to a Dozen Layers
A minimal loop that runs soon exposes problems: the model may call tools repeatedly and fall into an infinite loop; the context keeps growing longer and overflows the window; dangerous commands get executed directly and cause damage; the task becomes too complex to converge within a single round.
Each class of problem corresponds to a layer of mechanisms to stack on top. The common layering looks roughly like this:
- Role and system prompt layer: defines the agent’s identity, behavioral rules, output specifications, and tone. This is the master charter for all behavior.
- Tool definition layer: declares the name, purpose, parameter structure (Schema), and constraints of available tools. The quality of tool design directly determines whether the model can “apply its effort correctly.”
- Context engineering layer: manages the limited context window, including history compression, summarization, retrieval, and pruning.
- Permission and safety layer: sets up approvals, sandboxes, and boundaries for tool execution to prevent destructive operations.
- Task planning layer: lets the agent maintain a to-do list, decompose subgoals, and track progress.
- Sub-agent and delegation layer: assigns large tasks to clones with independent contexts to isolate complexity.
- Memory and persistence layer: saves key information and state across sessions.
- Error recovery layer: handles retries and fallbacks for tool failures, timeouts, and exceptions.
- Observability layer: records every call, its latency, and its result to make troubleshooting easier.
- Extension mechanism layer: plugs in custom logic through hooks, extension points, and similar facilities without modifying the kernel.
The key point: these layers are not independent stacks piled on top of each other, but coordinated reinforcement around the same kernel.
4. A Closer Look at Several Key Layers
4.1 Tool Design: Capability Is the Boundary
Tools are the only channel through which the model acts on the real world. Good tool design follows a few principles:
- Clear semantics: names and descriptions should make it obvious at a glance “when to use it and what it does.”
- Moderate granularity: too fine increases the number of call rounds; too coarse makes precise control difficult for the model.
- Structured returns: let the model parse results reliably instead of facing a long block of natural language.
- Idempotent and rollback-friendly: where possible, repeated execution should not produce side effects.
The model doesn’t read your source code; it understands your intent only through tool descriptions. The quality of tool documentation is therefore equivalent to the agent’s “actual capability boundary.”
4.2 Context Engineering: Trade-offs in Memory
The context window is the agent’s scarcest resource. As the loop progresses, messages keep accumulating and the window is quickly exhausted. Context engineering is about “showing the model the most relevant information within a limited window”:
- Compression and summarization: fold lengthy history into key points.
- On-demand retrieval: rather than stuffing in all the material at once, fetch it when needed.
- Structured pruning: keep observations relevant to decisions first, and discard redundant output.
An often-overlooked fact: the more information you feed the model, the more easily it “gets lost.” Providing information with restraint usually improves performance more than piling it on.
4.3 Permissions and Safety: Controllable Boundaries
If you let the model execute commands autonomously, you must set boundaries for it. Common practices include:
- Approval gates: high-risk operations (such as deletion or external writes) require human confirmation.
- Sandboxed isolation: run commands in a restricted environment, limiting the scope of file and network access.
- Least privilege: grant only the minimum permissions necessary to complete the task.
Safety mechanisms should not block normal tasks; they should strike a balance between “capability” and “control.”
4.4 Sub-Agents and Collaboration
For large tasks, a single context is easily overwhelmed by details. The sub-agent mechanism lets a main agent delegate a subtask to a clone with an independent context, which reports back only its result when finished. The benefit is context isolation—the massive intermediate work generated inside the clone doesn’t pollute the main loop, so the main agent stays focused on the big picture.
5. Engineering Practice and Operational Support
As a Harness moves from prototype to production, engineering and operations capabilities are equally indispensable:
- Observability: fully recording every model call, tool execution, latency, and token consumption is a prerequisite for troubleshooting and tuning.
- Persistence: session state and task progress need reliable storage so work can resume after an interruption.
- Elastic infrastructure: on a cloud platform, you can rely on general-purpose capabilities such as object storage, message queues, container platforms, log services, and monitoring and alerting to host agent execution, task scheduling, and notifications—for example, using standardized cloud products from a platform like Baidu AI Cloud to handle deployment and operations rather than building all the underlying infrastructure yourself.
These capabilities don’t change the kernel logic, but they determine whether the Harness can run stably and scale over the long term.
6. Summary of Design Principles
Reviewing the whole picture, Harness engineering can be distilled into a few principles:
- Let the model drive the control flow: let the model decide the flow instead of hard-coding steps with if-else.
- Stable kernel, layered mechanisms: keep the core loop simple, and add capability through layers.
- Tools are the capability boundary: tool design determines what the agent can do and how well it does it.
- Context is a scarce resource: provide information with restraint and actively manage memory.
- Safety by design, up front: design boundaries at the same time as you grant capabilities.
- Observable and extensible: make every step traceable and let capabilities keep evolving.
Understanding the Harness means understanding a counterintuitive division of labor: give the “intelligence” to the model and the “environment” to engineering. When that environment is polished solidly enough, the model can truly go from a component that chats to an engineering partner that works independently.
