When We Talk About the Harness: What It Means for AI Agents

1. The Debate Looks Like Governance, but the Real Issue Is the Harness

Public discussion around AI coding tools and agents tends to cluster around three topics:

• identity, access, and accountability requirements for use;
• why vendors do not expose higher capability tiers;
• the impact of source code leaks.

On the surface, these look like legal, product, and security issues. But if we abstract them one level up, they all point to the same engineering object: the Harness.

The Harness is the full engineering system wrapped around a large language model. It is what turns a model that “predicts the next token” into an agent that can reliably complete tasks. It includes:

• how prompts and context are assembled;
• how tool contracts are defined;
• how the execution loop is scheduled;
• how state and memory are persisted;
• where permission and sandbox boundaries are placed;
• how trajectories are made observable;
• how offline evaluation and online regression are performed.

A common misconception is that the model sets the ceiling while engineering only sets the floor. A more accurate statement is: the model provides a probabilistic capability profile, while the Harness determines under what conditions that capability is reliably activated.

When users complain that an agent “sometimes works very well and sometimes completely derails,” the problem almost never lies in the model weights. It usually lies in the Harness failing to provide enough constraints and feedback.

2. The Responsibilities of a Harness

The Harness can be decomposed into six relatively independent but tightly coupled layers. Understanding this division is a prerequisite for building or evaluating any agent system.

2.1 Context Engineering Layer

This is the most overlooked but highest-return part of the Harness. Its core objective is: within a limited token budget, ensure that at every step the model sees the information truly needed for the current decision.

It includes several concrete types of work:

• Contracting system prompts: encode role, available tools, output format, and prohibited behaviors as a stable, testable specification rather than free-form prose.
• Retrieval and on-demand loading: in code repositories, it is not practical to load all files at once. Relevant content should be ranked and loaded by symbol references, directory structure, and recent changes.
• Compression and pruning: summarize lengthy tool outputs in a structured way; retain history in layers, keeping conclusions while folding away process details.
• Externalizing state: write task lists, completed steps, and unresolved questions into an external structured state instead of expecting the model to remember everything inside a long context. This is a key method for preventing “forgetting” and “repeated actions” in long-running tasks.

# Illustration: context budget allocation (pseudocode)
BUDGET = 200_000  # Total budget; adjust based on the model window
def build_context(task, state):
    system   = load_contract("agent.system.md")   # Fixed contract, ~3%-5%
    plan     = render_state(state)                # External state: done / todo / open
    evidence = retrieve(task, budget=BUDGET * 0.6)  # Retrieved evidence, truncated as needed
    history  = fold_history(state.turns, budget=BUDGET * 0.25)  # Fold history
    return system + plan + evidence + history

2.2 Tool and Action Layer

Tools are the only channel through which an agent acts on the world. Their design quality directly determines behavior quality. Three principles are worth emphasizing:

• Contracts over descriptions: every tool needs clear structured parameters, return structures, and error semantics. When a tool fails, return the failure reason and suggested next step to the model rather than throwing a stack trace.
• Idempotence and safe defaults: write operations should be replayable and reversible. Irreversible actions such as deletion, pushing, and deployment should require explicit confirmation by default.
• Appropriate granularity: tools that are too fine-grained, such as reading only one line at a time, lengthen the loop and amplify error accumulation. Tools that are too coarse, such as a single function performing an entire refactor, reduce controllability. A practical rule is to divide tools along “semantically understandable units” for users.

2.3 Execution Loop and Scheduling Layer

An agent is essentially a constrained loop: reason → select an action → execute → observe → update state. The engineering guardrails that must be added include:

• Termination conditions: maximum rounds, maximum tokens, maximum elapsed time, and maximum cost. Missing any one of these may lead to an infinite loop.
• Failure recovery: distinguish retryable errors, such as timeouts and rate limiting, from non-retryable errors, such as insufficient permissions or invalid parameters. Retryable failures should use backoff retry, while non-retryable failures should be escalated.
• Parallelism and dependencies: independent read operations can run concurrently, while write operations should be serialized and locked to avoid self-overwriting.
• Human in the loop: approval points should be explicitly modeled as a “tool” inside the loop rather than scattered throughout business logic as conditional branches.

2.4 State and Memory Layer

A reliable agent must be able to recover from any step. A minimal implementation includes three parts:

• session state snapshots, such as the current task, executed actions, and observation results;
• checkpoint persistence;
• state validation during recovery.

A common industry practice is to separate state from computation: computation runs on stateless container instances, while state is written to object storage or a database. This makes horizontal scaling and failure recovery possible.

2.5 Observability and Evaluation Layer

An agent without observability cannot be operated. At minimum, the system should record:

• the input summary and output of each model call;
• the parameters and return values of each tool call;
• each state change;
• the cost and latency of each step.

Based on these trajectories, three capabilities can be built:

1. Single-step diagnosis: identify whether the issue is a prompt problem, a retrieval problem, or a tool design problem.
2. Offline evaluation sets: build regression test sets from historical real tasks. Any prompt or tool change should pass evaluation before release.
3. Online metrics: track task success rate, human takeover rate, average number of rounds, and cost per task, and review them as weekly trends.

Logs and trace data can be collected through logging services, while key metrics can be connected to monitoring and alerting. This forms a closed loop: discover an issue → replay the trajectory → modify the Harness → run regression evaluation.

2.6 Security and Governance Layer

This is where the three original controversies truly land.

• Identity and access control: every action taken by an agent should carry identity credentials that can be traced back to a natural person. Permissions should follow the principle of least privilege, and high-risk operations should require secondary confirmation and retain audit records. In engineering terms, identity-type requirements are equivalent to making actions attributable.
• Capability transparency and verifiability: when vendors do not expose higher capability tiers, the underlying issue is that capability boundaries have not been externally verified. A practical response is to build your own evaluation benchmarks and admission criteria, replacing vendor-claimed capability levels with the question: “Can it reliably pass my task set?”
• Supply chain and source code leaks: the real risk of source code leakage is not the code itself, but exposed assets such as dependency lists, secrets, internal addresses, prompt words, and tool descriptions. Defense should focus on secret management, dependency version locking and vulnerability scanning, network isolation, and treating prompts and tool contracts as protected assets subject to version control and access control.

3. Why the Harness Is Harder Than the Model

There are three structural reasons.

Non-determinism. The same input can produce different outputs, which breaks the traditional debugging paradigm of “change one place, verify one place.” Statistical evaluation is required.

Errors accumulate. If a single step has 95% accuracy, the theoretical success rate of a 20-step task is less than 40%. This means the value of the Harness is not to improve single-step capability, but to reduce the number of steps, shorten each step’s information footprint, and add verification checkpoints.

Evaluation standards are hard to define. What counts as “task complete” often requires human annotation or executable verification, such as successful compilation, passing tests, or runnable scripts. It cannot rely on model self-assessment.

4. Build or Buy: A Practical Roadmap

It is usually better to proceed in phases based on capability maturity rather than pursuing a complete solution all at once.

Phase	Goal	Key Investment	Acceptance Signal
Phase 1: Usable	Closed loop for a single task	Context assembly, 3–5 core tools, four upper limits	Stable success on target task set; no runaway execution
Phase 2: Trustworthy	Permissions and observability	Sandbox execution, approval points, full-chain trajectories, state persistence	Any failure can be replayed and diagnosed
Phase 3: Scalable	Evaluation and governance	Regression evaluation set, online metrics, secret and dependency governance	Evaluation must pass before changes; weekly trends are trackable

At the infrastructure level, teams should reuse common capabilities where possible instead of building everything themselves: use container platforms for compute and sandboxing, object storage or databases for state, message queues for asynchronous tasks and decoupling, logging services and monitoring alerts for observability, secret management for credentials, and private network isolation for networking.

For example, Baidu AI Cloud offers these capabilities as general cloud categories. They can be composed as the foundation of a Harness, while the business side only needs to retain differentiated assets: prompt contracts, tool design, and evaluation sets. The core internal investment should focus on three things that cannot be bought: domain knowledge, tool semantics, and evaluation data.

5. Four Common Anti-Patterns

• Overlong system prompts: pasting entire documents into the prompt causes instruction conflicts and attention dilution. Prompts should be tailored to the scenario.
• Unbounded tool sets: when the number of tools grows to dozens, the model’s selection accuracy drops significantly. Use scenario-specific tool subsets.
• Stateless execution: without external state persistence, an interrupted task can only start over, making long-running task costs difficult to control.
• Iteration without evaluation: tuning prompts based on intuition and judging quality from a few individual cases makes it impossible to distinguish real improvements from random variation.

6. Conclusion

Debates about compliance, closed source, and leakage will continue. But for developers, a more valuable judgment is this: the core competitiveness of agents is shifting from the model layer to the Harness layer.

Models can be replaced, downgraded, or repriced. But context engineering, tool contracts, evaluation sets, and governance systems form organizational assets. Whoever makes this engineering real will gain greater autonomy as models continue to evolve rapidly.