# How AI Coding Agent Harnesses Are Architected: Design Patterns and Implementation Guide

1. Start With the Definition: A Harness Is Not a Model

In English, “harness” originally refers to the set of equipment that puts a horse to work. In AI, harness engineering means building a full set of rules, frameworks, and runtime infrastructure around a model: constraining its behavioral boundaries, orchestrating its execution cadence, and managing its memory and state, so that it can complete tasks the model “cannot finish on its own.”

Understanding that evolution is the prerequisite for understanding a harness:

• Prompt stage: a single prompt drives one-turn or simple multi-turn interactions. The capability ceiling is whatever the model can reason through in the moment.
• Skills stage: domain knowledge, operating rules, and tool usage are captured as loadable assets injected into context on demand, solving the “knowledge isn’t in the model’s head” problem.
• Harness stage: attention shifts from “what the model says” to “how the system runs.” How context is compressed, when tools run concurrently, how failures are retried, how permissions are gated, where state lives — these engineering decisions determine the end-user experience.

A common misdiagnosis is to attribute all the gap between agents to the base model. In reality, at equal model capability, the difference in harness quality alone is enough to produce a several-orders-of-magnitude gap in task completion rates. The architecture is worth studying openly precisely because it converts a hard-to-reproduce “feel” advantage into an engineering structure that can be reviewed item by item.

2. From One Agent to an Agent Cluster

These coding agents are usually described as a command-line program, but if you break down their implementation you will find they are, at their core, a deeply optimized harness with clear internal role separation:

Role	Responsibility	Design Point
Main query engine	Handles user input, calls the model, executes tools, maintains the main loop	The sole authoritative state holder
Derived sub-agents	Handle side-path work such as context compression and summary rebuilding	Isolated context, so the main line is not polluted
Background task agents	Take on time-consuming commands, long builds, and continuous monitoring	Asynchronous decoupling; does not block interaction
Parallel collaboration module	Multiple agents split up independent subtasks	Task-decomposability check + result merging

Beyond that, mature implementations usually include a few cross-cutting capability domains: a tool system (each tool ships with its own permission declaration and parameter validation), a command system (a large number of slash commands as composable shortcut entries), a permission system (per-tool authorization and auditing), a bridging module (local sessions and remote sessions interoperate with each other), and a plugin and skill system (third-party extensions).

Key insight: these modules are not a pile of features. They are organized around a single main line: keep the main loop minimal and push all complexity down into pluggable components.

3. Six Core Mechanisms

3.1 AsyncGenerator as the Backbone: Turning “the Stream” Into a Control Structure

The traditional approach is to write the model call as a function that returns a complete response, then parse tool requests out of it. That leads to two problems: on the interaction side, you cannot render while the model is still generating; on the control side, you cannot insert decision points into the generation process.

A better approach is to make every layer of the main loop an async generator that yields events upward one by one:

// Illustrative: async generators as the skeleton of the main loop
async function* runAgentLoop(input: string, state: State): AsyncGenerator<Event> {
  const messages = state.history;
  messages.push(userMessage(input));
  while (true) {
    const stream = model.stream(messages, state.tools); // returns an AsyncGenerator
    const toolCalls: ToolCall[] = [];
    for await (const chunk of stream) {
      yield { type: "stream", delta: chunk };          // forwarded to the render layer immediately
      if (chunk.kind === "tool_call_start") toolCalls.push(chunk.call);
    }
    if (toolCalls.length === 0) {
      yield { type: "done", message: messages.at(-1) };
      return;                                           // normal exit
    }
    // Dispatch the tools, write the results back to the message array, continue to the next round
    yield* executeTools(toolCalls, state);
    await maybeCompactContext(state);                   // context maintenance point
  }
}

This structure delivers three direct benefits: the rendering layer is decoupled from the inference layer; any point in the chain can be turned into a compression, audit, or hook insertion point; and yield forms natural backpressure, automatically slowing the pull rate when the consumer is slow.

3.2 The Agentic Loop: while(true) and State Management

The loop itself has only four steps — call the model → parse tool requests → execute them → write the results back. But its correctness depends entirely on the state machine design:

• A single owner of state: message history, todo list, modified files, current working directory, and so on all converge into one state object, avoiding inconsistency across multiple copies.
• Explicit termination conditions: normal termination (the model stops requesting tools), budget termination (turn count, token count, or duration exceeded), permission termination (user refusal), and error termination (unrecoverable failure). In engineering terms, a loop without explicit termination conditions is equivalent to a time bomb.
• Resumability: if state is serializable, a session can be interrupted, migrated, and resumed. This is the prerequisite for bridging local and remote sessions.
• Reasoning and tool calls mixed in one turn: when a model turn contains both text output and tool calls, emitting the text first and then executing the tools keeps the order of the narration the user sees consistent with the order of the actions.

3.3 Context Management: Compression as a Layered Strategy

The context window is the agent’s hardest resource constraint. Compression should not be as crude as “do a summary once.” It should be a tiered pipeline with thresholds triggered in sequence, where earlier tiers are cheaper and later ones cost more:

• L0 Structural optimization: put the stable prefix (system prompt, tool definitions) at the front to take advantage of prefix caching and lower cost; put volatile content at the end so a single change does not invalidate the entire cache.
• L1 Single-result truncation: when a single tool output exceeds the threshold, keep only the head and tail plus a summary placeholder, and note that “the full content has been written to a file and can be read on demand.” This one step usually eliminates most of the waste.
• L2 Micro-compression (lightweight in-place replacement): as context slowly approaches the threshold, replace the earliest few large tool results, whose value has already decayed, with one-line placeholder summaries. The conversation text stays as is.
• L3 Session-level rebuild: near the hard limit, produce a structured summary of the whole history (goal, what’s done, what’s not done, key files and decisions), then replace the raw history with that summary and keep running.
• L4 Side-channel derived compression: hand compression to an independently derived sub-agent. The benefit is that tokens spent on the summarization process itself never enter the main context, and compression quality can be optimized separately with a dedicated prompt.
• L5 External memory: persist long-term information in the file system or a retrieval store, and keep only indices plus on-demand retrieval hooks in context, decoupling “memory capacity” from the “context window.”

Trigger timing matters just as much. Compression should happen at natural tool-call boundaries, not in the middle of a streaming model output — otherwise the integrity of the message sequence is broken and subsequent request parameters become invalid.

3.4 Tool Execution Preemption: Start Before the Model Finishes Speaking

A single model turn often contains multiple tool requests, and a fair number of them have no dependencies on each other (reading several files, running several independent queries). Serial execution stacks network and I/O latency linearly.

The core of the optimization is dependency analysis plus concurrent scheduling:

1. During the streaming parse phase, start identifying tool names and arguments; queue requests as soon as their arguments are complete, without waiting for the whole turn to end.
2. Read and query tools are classified as concurrency-safe; write tools default to serial unless you can prove their scopes do not overlap.
3. Put an upper bound on concurrency and pair it with timeouts, so a burst does not exhaust system resources.
4. Tier the failure strategy: retryable errors (rate limiting, network jitter) get retried with exponential backoff; deterministic errors (invalid arguments, nonexistent paths) return a clear error message immediately, so the model can fix them itself instead of spinning on empty retries.
5. Keep the results written back in their original order, so the message sequence the model sees matches the order of the requests it issued.

3.5 Hook Architecture: Pulling Cross-Cutting Concerns Out of the Main Loop

The customization that users and teams actually need is usually strongly tied to business: unified formatting, blocking dangerous commands, running checks automatically before a commit, reporting logs, tracking cost. If all of that is hard-coded into the main loop, the code will rot fast.

The hook idea is to expose events at key lifecycle points and let external scripts register handlers:

Point	Typical Use
Session start/end	Environment initialization, statistics roll-up
Before a tool call	Argument rewriting, blocking dangerous operations, human confirmation
After a tool call	Output cleaning, automatic formatting, result archiving
Before context compression	Export a snapshot, back up history
After the model responds	Sensitive-information detection, writing logs to disk

The design point is clear failure semantics. A blocking hook must be able to return “denied + reason,” and that reason must be fed back to the model so it can adjust its strategy on the next turn; an observation hook should fail silently and never affect the main flow.

3.6 Permission System: Least Privilege, Per Tool

Autonomy and safety are an inherent tradeoff. A workable approach is to model permission as a policy decision on the “tool × parameter” dimensions, rather than as a global switch:

decision = f(tool name, argument fingerprint, target path, session trust level, prior grants)

• Read operations are allowed by default; writes are restricted to within the working directory; high-risk actions such as writing outside the directory, deletion, and outbound network calls require explicit confirmation.
• Memoize authorizations: equivalent operations the user has already confirmed in the same session are not asked about again, reducing the interruption frequency.
• Audit the whole way: for every tool call, record the time, an argument summary, the decision, and the basis for that decision, so it can be traced after the fact.

4. Let Runtime Data Drive Engineering Decisions

Whether an architecture is good cannot be judged subjectively. One thing worth learning from these implementations is that they treat engineering metrics as first-class citizens, collect them continuously, and use them to set optimization priorities.

The minimum recommended metric set:

• Task success rate: bucketed by task type, to locate weak spots.
• Average turns and token consumption: reflects harness efficiency; at equal success rate, lower is better.
• Compression trigger frequency and the share of each tier: verifies whether the compression strategy is working and whether it is over-compressing.
• Tool error distribution: distinguishes “the model used the wrong tool” from “the tool implementation has a defect” — fix the first with prompts and tool descriptions, the second in code.
• Human intervention rate: the share of tasks that require the user to take over; maps directly to product experience.
• First-token latency and gains from tool concurrency: evaluates interaction responsiveness and scheduling strategy.

With this data, improvement moves from “something feels off somewhere” to “which metric, at which layer, is worst,” which gives iteration a verifiable closed loop.

5. Design Patterns Worth Borrowing Directly

1. Minimal main loop plus pluggable components: push all complexity down and avoid giant functions.
2. Async generators through the whole chain: unify streaming and control flow, with natural support for mid-flight decisions.
3. A single authoritative, serializable state: the foundation for resuming, migrating, and bridging sessions.
4. Layered context compression: lightweight first, triggered at boundaries, executed via a side-channel sub-agent.
5. Tool concurrency with dependency analysis: reads concurrent, writes serial, failures tiered.
6. Lifecycle hooks: cross-cutting concerns externalized, customization with zero intrusion.
7. Least privilege per tool plus full audit: build security into the call chain rather than checking it after the fact.
8. Multi-agent role separation: the main agent stays focused and heavy work is outsourced to sub-agents.
9. Metric-driven iteration: use data to prioritize and experiments to verify gains.

6. How to Adopt This in Your Own Project

A pragmatic sequence:

• Step 1: Wrap raw calls into a pluggable loop. First extract three abstractions: a model interface, a tool registry, and a state object — even if you start with only 3 tools and 1 termination condition.
• Step 2: Add observability. Record per-turn input/output summaries, tokens, and elapsed time. Without measurement you cannot tell whether later changes are good or bad.
• Step 3: Implement L1 and L2 compression. These two give the best return on investment; usually do single-result truncation first, then micro-compression.
• Step 4: Introduce the hook and permission layers. Make security policy configurable and auditable, clearing the path for team-wide rollout.
• Step 5: Add sub-agents and parallel collaboration as needed. Note that the precondition is that the task is decomposable: if subtasks share a lot of intermediate state, the context-synchronization cost of parallelism will outweigh the gains.

When evaluating, always run regression tests against a fixed task set, hold model version and prompt variables constant, and compare only the harness changes — otherwise the conclusions are not credible.

7. Things to Keep in Mind

• Reference the source code; don’t copy it. The architectural patterns are general engineering knowledge, but the specific code involves licensing and compliance. In commercial projects, implement from your own understanding.
• Complexity is not free. Every abstraction layer adds debugging difficulty, and in an early-stage project over-design is more dangerous than under-design.
• A harness cannot make up for missing model capability. It amplifies what already exists; it does not create capability that isn’t there. When the task itself is beyond the model’s boundary, the right move is to narrow the task granularity or introduce deterministic tools, not to stack prompt tricks.
• Observability before optimization. Before any compression, concurrency, or multi-agent rework, make sure you can see exactly what the system did on each turn.

8. Conclusion

The biggest value of making this kind of architecture public is not that it provides reusable code. It is that it turns a fuzzy question into one you can discuss: a large part of the performance gap between agents comes from harness engineering quality. Prompts decide what the model “knows,” skills decide what it “can do,” and the harness decides whether it can “finish the job reliably.”

For an engineering team, what is worth investing in is not chasing one particular implementation. It is abstracting the patterns above into capabilities of your own platform: a unified loop skeleton, pluggable context strategies, fine-grained permissions and hooks, and an honest measurement system. Once these four are in place, swapping the base model, extending the tool set, and supporting multi-agent collaboration are all configuration-level adjustments.