AI Agent Infrastructure Barriers: After Harness Engineering

1. The starting point: an agent is not just a longer prompt

In early development, many teams understood agents as a combination of “large language model + tools + prompts.” Once the model could understand a task, call tools, and return results, the basic loop seemed complete. However, when agents move from demos into real business systems, complexity quickly shifts to the runtime environment, state management, task scheduling, access control, error recovery, and result acceptance.

What determines whether an agent can run reliably is often not whether a single prompt is clever enough or whether a single tool is rich enough. It is whether the entire system can maintain consistency and recoverability over long, multi-step, asynchronous, and high-concurrency workloads.

Therefore, the focus of agent engineering is shifting from “how to build a model that can use tools” to “how to build a manageable, observable, and recoverable agent runtime system.” This also explains why many teams make rapid progress in prototypes but encounter unavoidable infrastructure barriers once they move into production.

2. What does Harness engineering actually solve?

Harness engineering can be viewed as the adaptation layer between model capabilities and real tasks. It is not only responsible for writing prompts or registering tools. It organizes models, context, tools, execution environments, state, permissions, and acceptance mechanisms into a runnable task system.

A complete Harness layer usually has to handle the following problems:

• Task decomposition: Breaking complex goals into executable steps.
• Tool orchestration: Deciding which tools to call, when to call them, and what boundaries apply.
• Context management: Maintaining short-term session state, long-term memory, and intermediate task results.
• Execution environment control: Providing isolated environments for code execution, file operations, and command execution.
• Error recovery: Handling timeouts, failures, disconnections, repeated executions, and inconsistent state.
• Permission control: Restricting the resources an agent can access.
• Result validation: Determining whether a task is complete, whether retries are needed, or whether human intervention is required.

In other words, Harness engineering addresses the question: “Can the model perform tasks reliably?” It is neither the model itself nor merely an application interface. It is the middle layer that turns agent capability into productivity.

Many teams initially assume that Harness engineering is the hardest part because it involves many details: tool definitions, prompts, flow control, context compression, retry strategies, and more. But as project scale grows, what often blocks teams is not Harness engineering itself, but whether the underlying infrastructure it depends on is stable enough.

3. The infrastructure wall: real difficulties exposed after prototypes

When an agent only completes a simple Q&A interaction or a single tool call, many underlying problems may not appear immediately. But once tasks enter production environments, these problems tend to emerge all at once.

1. Stability of sandboxes and execution environments

Agents often need to execute code, read and write files, run scripts, call command-line tools, or access external data. If the execution environment is unstable, tasks will be interrupted.

Common problems include:

• A broken sandbox connection prevents the task from continuing;
• Containers or virtual machines exit unexpectedly and intermediate state is lost;
• Network access policies are either too loose or too strict, causing tool calls to fail;
• Resource limits are unclear, leading to execution timeouts or memory overflows;
• Long-running tasks cannot be recovered and must be rerun from the beginning.

The root cause of these problems is usually not insufficient model capability, but a lack of engineering safeguards in the execution environment. Production-grade agents must treat sandboxes as standard infrastructure, not temporary processes.

A basic improvement direction is to decouple task execution environments from business logic, and to manage task startup, resource allocation, state checking, exception cleanup, and result return through standardized sandbox lifecycle management.

An abstract configuration example can be expressed as follows:

agent_task:
  task_id: "task_001"
  sandbox:
    isolation_mode: "container_or_microvm"
    network_policy: "allowlist"
    cpu_limit: "1_core"
    memory_limit: "2_gb"
    timeout: "30_min"
  recovery:
    checkpoint_enabled: true
    retry_policy: "exponential_backoff"
    dead_letter_enabled: true

The key point here is not the specific parameter values, but that the task execution environment must be explicitly defined, monitored, and recoverable.

2. Continuity of memory and state

Agent tasks often need to preserve information across steps. For example, an agent may first read a file, then generate a patch, run tests, and continue fixing issues based on test results. If intermediate state is lost, the agent may repeat actions, drift from the intended context, or break logical continuity.

“Memory loss” is a common production problem. It may appear as:

• Task state cannot be recovered after a session ends;
• Context is over-compressed during long-running execution;
• Tool execution results are not persisted in time;
• Key information is missed across multiple interaction rounds;
• After a task restart, the system cannot determine which step was completed.

Therefore, an agent system should not rely only on process memory. It should establish an external state layer. Short-term state can store the current session and tool-call results, while long-term state can store project knowledge, historical tasks, acceptance results, and user preferences.

A typical state structure may include:

state:
  short_term:
    storage_type: "session_state"
    content: "current_dialogue_and_tool_results"
  long_term:
    storage_type: "knowledge_or_memory_store"
    content: "project_context_and_historical_patterns"
  checkpoint:
    storage_type: "durable_object_storage"
    content: "task_progress_and_intermediate_artifacts"

The value of this design is that even if an execution node restarts, the agent can continue from the latest checkpoint.

3. Asynchronous tasks and long-running workflows

Many agent tasks cannot be completed in a single request. They may include code generation, test execution, human confirmation, data queries, document retrieval, and external approvals. If every step is designed as a synchronous request, the system can easily block, time out, or fail.

Asynchronous processing is an important part of agent infrastructure. Common capabilities include:

• Task queues: breaking long tasks into schedulable units.
• Event notifications: triggering the next step after a tool call or human confirmation completes.
• Timeout control: preventing tasks from waiting indefinitely.
• Retry mechanisms: automatically retrying recoverable errors.
• Idempotency control: preventing duplicate execution from polluting data.
• Dead-letter handling: isolating repeatedly failed tasks for manual review.

An illustrative flow is as follows:

submit_task
  -> enqueue_async_job
  -> execute_agent_step
  -> call_tool_or_model
  -> persist_state
  -> wait_for_event_or_completion
  -> validate_result
  -> notify_or_continue

For production-grade agents, asynchronous capability is not optional. It is a foundational condition for ensuring the stability of long-running tasks.

4. Permissions, security, and auditing

Once an agent is connected to real business systems, it faces permission boundary questions. Which files can it read? Can it modify production data? Can it access the external network? Can it call high-privilege tools? These questions must be governed before the Harness layer is implemented.

Permission governance should include at least four layers:

• Least privilege: Granting only the minimum resources required to complete the task.
• Policy validation: Checking parameters, targets, and behaviors before a tool call is made.
• Audit logs: Recording models, tools, parameters, results, and operators.
• Sensitive data isolation: Preventing the agent from directly accessing unauthorized data.

Without a permission system, the risk of an agent can escalate from “incorrect output” to “incorrect execution.”

5. Acceptance loops and result evaluation

The fact that an agent has finished a task does not mean the task succeeded. Many teams overlook the acceptance stage, causing agent output to look reasonable while failing to meet actual business requirements.

An acceptance mechanism should include:

• Automated testing: checking whether code or artifacts pass predefined tests.
• Rule validation: checking whether formats, fields, paths, and permissions comply with requirements.
• Regression checks: confirming that the task has not broken existing functionality.
• Human review: introducing manual confirmation for high-risk tasks.
• Failure attribution: distinguishing model errors, tool errors, environment issues, or prompt issues.

Only by building a complete acceptance loop can agents move from “seemingly capable” to “reliably usable.”

4. Harness and models are becoming highly paired

For a period of time, a popular view in the industry was that teams should first build a general-purpose Harness and then swap in different models as needed. This approach worked when model capabilities were relatively close and tasks were simple. However, as model capability differences widen, this approach encounters clear bottlenecks.

Models can differ significantly in the following areas:

• Long-context utilization efficiency;
• Tool-calling stability;
• Multi-step reasoning capability;
• Code generation and repair capability;
• Instruction-following strength;
• Output format consistency;
• Latency and cost characteristics.

As a result, Harness engineering is shifting from a “general adapter” to an engineering system that is deeply paired with model capabilities. This does not mean locking in one specific model. It means establishing adaptation strategies for different models.

For example:

• For models with strong long-context capability, a more complete context injection strategy can be used.
• For models with strong tool-calling capability, flow constraints can be reduced to allow more autonomous decision-making.
• For models with strong reasoning capability but higher cost, use them only at critical nodes.
• For fast-response models, assign them to multi-step execution and routine repair tasks.
• For models with stable output formats, let them directly generate structured results.
• For models with weaker format stability, add validation and retry mechanisms.

This means Harness engineering is no longer simple prompt assembly. It is a combination of “model capability profiling + task strategy + tool constraints + acceptance mechanisms.”

An illustrative configuration may look like this:

model_policy:
  primary_model:
    role: "planning_and_reasoning"
    context_strategy: "full_history"
    tool_permission: "high"
  execution_model:
    role: "code_generation_and_testing"
    context_strategy: "compressed_task_state"
    tool_permission: "medium"
  validator_model:
    role: "result_checking"
    context_strategy: "diff_and_criteria_only"
    tool_permission: "read_only"

This design shows that the relationship between Harness and models is not “one general framework adapting to all models,” but “different models taking on different roles in different tasks.”

5. Platformization: pushing file systems, skills, and memory down to the platform layer

When many teams build agents, they repeatedly build the same underlying capabilities: file management, tool registration, memory storage, sandbox scheduling, log tracing, task queues, and permission control. These efforts are not meaningless, but if every team starts from scratch, the cost becomes very high.

Therefore, an obvious industry trend is emerging: common agent infrastructure is being pushed down to the platform layer. Platforms can provide standardized file systems, skill libraries, memory services, task queues, execution sandboxes, and runtime observability, allowing application teams to focus on business goals and acceptance rules.

Platformization usually brings several types of benefits:

• Reducing duplicated engineering costs: Teams do not need to rewrite sandboxes, queues, and state management every time.
• Improving stability: Platforms can uniformly handle timeouts, retries, disconnections, and failure recovery.
• Enhancing observability: Task logs, tool calls, and model outputs can be traced in a unified way.
• Improving security: Permissions, auditing, and isolation policies can be managed centrally at the platform layer.
• Accelerating iteration speed: Business teams can validate agent tasks faster.

On cloud platforms such as Baidu AI Cloud, teams can compose general infrastructure capabilities for compute, storage, messaging, logging, and monitoring that are needed to run agents, rather than building each capability individually. Common general-purpose capabilities for agent systems include:

• Object storage: saving task artifacts, files, and checkpoints.
• Message queues: carrying asynchronous tasks and event notifications.
• Container platforms: running isolated sandboxes and tool services.
• Logging services: recording model, tool, and task execution traces.
• Monitoring and alerting: detecting timeouts, rising failure rates, and resource anomalies.
• Key management: protecting access credentials and sensitive configuration.

This platform-oriented approach is not intended to replace Harness engineering. Instead, it sinks the most basic and repetitive parts of Harness engineering into standard capabilities, allowing developers to focus on higher-value scenario design and task optimization.

6. Multi-agent orchestration: from single-point agents to composable systems

When tasks become complex enough, a single agent may not be able to handle planning, execution, review, repair, and acceptance at the same time. At that point, multi-agent orchestration becomes an important approach.

Multi-agent systems are not simply about increasing the number of agents. They split tasks into different roles, with each role focused on a specific objective.

Common role divisions include:

1. Separation of execution agents and advisory agents

Execution agents are responsible for actually calling tools, modifying files, and running commands. Advisory agents only provide recommendations and do not directly perform high-risk operations. This separation can reduce the risk of misoperation.

2. Confrontation between generation agents and review agents

Generation agents produce content, code, or proposals. Review agents check format, logic, security, and business rules. The two form a confrontational relationship that can help improve output quality.

3. Exploration agents for finding problems

In scenarios such as code review, defect localization, and log analysis, multiple exploration agents can be introduced to inspect the same target from different angles. This pattern is similar to “swarm search” and is suitable for discovering problems that a single agent may miss.

However, multi-agent systems do not mean that higher complexity is always better. They introduce new engineering problems:

• Unclear role responsibilities can lead to duplicated work.
• Inconsistent shared state can cause conflicts.
• Non-unified message protocols can lead to communication failures.
• Uncontrollable costs can waste resources.
• Unclear acceptance standards can lead to a situation where “multi-agent systems are busy but results do not converge.”

Therefore, multi-agent orchestration must be designed together with state management, message buses, permission control, and acceptance mechanisms.

7. Will Harness engineering disappear in a year?

In the short term, Harness engineering will not disappear, but its form will change.

Early-stage developers often need to hand-write large amounts of scaffolding code: registering tools, assembling prompts, managing context, handling retries, maintaining sandboxes, writing logs, and implementing acceptance logic. In the future, these tasks will be gradually absorbed by platformized capabilities. Developers will no longer need to repeatedly implement underlying frameworks for every agent. Instead, they will build systems through configuration, components, and policies.

In other words, Harness engineering will not completely disappear, but it will undergo three changes:

1. From hand-written scaffolding to platform configuration

Many capabilities that originally required coding will become standard configuration items in platforms. Examples include sandbox specifications, memory policies, tool permissions, asynchronous queues, and acceptance rules.

2. From prompt stacking to policy-based orchestration

Prompts will remain important, but they will no longer be the only method. Developers will control system behavior through task strategies, model division of labor, tool constraints, and acceptance standards.

3. From single-agent implementation to multi-component collaboration

Future agent systems will look more like software systems rather than a single prompt. Models, tools, sandboxes, memory, queues, logs, and validators will together form a complete runtime chain.

Therefore, a more accurate statement than “Harness engineering will not be needed in a year” is: developers will not need to repeatedly build Harness engineering from scratch, but they will still need to understand and configure Harness capabilities.

8. Practical recommendations for teams

For teams preparing to move from prototype to production, the following practices may be useful.

1. Define acceptance criteria before designing the agent

Do not first ask, “Which model should we use?” Instead, first ask, “What is the standard for completing the task?” The clearer the acceptance criteria, the easier it is to engineer the agent.

2. Externalize state

Do not keep all critical information in memory or in a single session. Task state, tool results, intermediate artifacts, and checkpoints should be persisted.

3. Standardize sandboxes

All code execution, file modifications, and command execution should be performed in isolated environments with timeout, resource limits, and exception recovery capabilities.

4. Make long-running tasks asynchronous

For multi-step, long-duration, or human-confirmation tasks, use queues, events, and state machines for scheduling.

5. Build a model adaptation layer

Do not directly bind business logic to a specific model invocation method. Use an adaptation layer to manage model selection, context strategies, tool permissions, and failure degradation.

6. Build an evaluation set

Production-grade agents must have a repeatable evaluation set. The evaluation set should cover common tasks, boundary tasks, failure scenarios, and security scenarios.

7. Introduce multi-agent systems cautiously

Multi-agent systems are suitable for complex tasks, but not for all scenarios. Start with a single-agent closed loop, then gradually introduce review, exploration, and repair roles.

8. Pay attention to cost and observability

Agent systems can generate large amounts of model calls, tool calls, and log data. Teams need to plan cost budgets, monitoring metrics, and alert strategies in advance.

9. Conclusion: infrastructure barriers are the long-term threshold

Competition in agent applications is shifting from “whose model is stronger” to “who can reliably turn model capability into business results.” Prompts, tool definitions, and task flows are certainly important, but they are only surface-level capabilities. What truly determines whether an agent can scale is whether infrastructure such as sandboxes, state, asynchronous processing, permissions, observability, and acceptance is mature.

Harness engineering will not disappear. It will evolve from scattered, manual, and repetitive scaffolding work into platformized, componentized, and reusable infrastructure capabilities. For developers and enterprises, whoever completes this layer earlier will be able to move agents from demos to production faster and continuously accumulate reusable engineering assets.