Long-Running AI Agents: Evaluation and Engineering Implementation Guide

1. Why Long-Running Agents Cannot Be Evaluated Only by Demo Outcomes

In recent years, long-running agents have become a frequent topic in the adoption of large model applications. Unlike a single question-and-answer session, a one-time summary, or a code completion, a long-running agent usually needs to span hours, days, or even longer periods. It must continuously maintain task goals, break down execution steps, call tools, check results, fix errors, and finally deliver an output that can be reviewed, accepted, and reproduced.

Common examples in public discussions include building a bootable operating system prototype from an empty directory, with continuous coding, testing, review, and integration; or reproducing a mathematical proof in a formalized form, turning an existing proof into a machine-checkable reasoning chain and significantly shortening completion time. These cases can show that long-running agents already have some engineering capability, but they cannot be directly equated with production readiness.

The reason is that the value of a long-running agent is not that it can start or that it can run to completion, but that it can deliver stable results under complex constraints. An operating system prototype that can boot does not necessarily have long-term stability, compatibility, performance, or security boundaries. A formalized proof that passes machine checking does not necessarily mean the model can independently propose new proofs or discover new theorems. Therefore, evaluating long-running agents requires moving from demo narratives to engineering acceptance.

2. Key Capability Breakdown: Continuous Decomposition, Execution, and Integration

Long-cycle tasks are complex because they are not static goals. They are dynamic processes. During execution, new subproblems appear. The results of subproblems can change the priority of the original plan. Tool outputs, test results, and human feedback can also continuously affect subsequent actions.

1. Task Decomposition Is Not One-Time Planning

Many agents generate a task plan at the beginning. The harder part is continuous decomposition during execution. For example, building an operating system prototype is not just a single task of writing code. It includes kernel structure, driver interfaces, disk access, keyboard input, network communication, file persistence, window systems, login flows, installers, and built-in feature modules. Each type of work needs clear boundaries, interfaces, test methods, and failure recovery strategies.

Long-running agents need three layers of decomposition:

• Goal decomposition: Turn business goals into executable engineering tasks.
• Risk decomposition: Identify high-risk, high-uncertainty parts that require human confirmation.
• Verification decomposition: Define acceptance methods for each task, not only execution methods.

If there is only execution decomposition but no verification decomposition, the agent can easily produce results that look completed but cannot pass overall acceptance.

2. Multi-Agent Collaboration Requires Unified Context

In complex tasks, a single agent often cannot simultaneously handle planning, coding, testing, review, recording, and integration. Therefore, multi-agent collaboration has become a common approach. Typical roles include:

• Planning agent: Turns goals into a task tree.
• Execution agent: Handles concrete actions such as coding, testing, analysis, and writing.
• Review agent: Checks correctness, consistency, and potential risks.
• Recording agent: Maintains decision logs, change records, and context summaries.
• Integration agent: Merges results, resolves conflicts, and generates the final deliverable.

The key to multi-agent collaboration is not a large number of agents, but context consistency. If sub-agents understand requirements differently, duplicate work, interface conflicts, inconsistent style, or goal drift can easily occur. In engineering practice, systems usually need a shared task bus, unified specification documents, versioned context, checkpoint mechanisms, and conflict resolution strategies.

3. Tool Use and Sandbox Execution Determine Controllability

Long-running agents usually call tools such as code execution, test runs, file read/write operations, log queries, static checks, and dependency installation. Without tools, an agent can only generate suggestions. With tools, an agent can modify the environment and verify results.

However, tool capabilities must be combined with sandboxes and permission controls. A sustainably executable agent system needs at least the following:

• Isolated runtime environments to avoid affecting production systems.
• Restrictions on network access and file access scope.
• Records of every tool call and returned result.
• Human approval for dangerous operations.
• Budgets and timeouts for resource consumption.

Otherwise, a long-running agent may change from an efficiency tool into an uncontrollable source of changes.

3. From “Executable” to “Acceptable”: Core Delivery Standards for Long Tasks

The most common risk of long-running agents is treating “execution finished” as “task completed.” A truly deliverable result must go through clear acceptance. The clearer the acceptance criteria, the more controllable the agent becomes.

1. Acceptance System for Code-Related Tasks

For tasks such as code generation, refactoring, and system prototypes, acceptance cannot rely only on “it runs.” At minimum, it should include:

• Static checks: Syntax, types, style, and dependency legitimacy.
• Unit tests: Core functions and module behavior are correct.
• Integration tests: Interfaces between modules are consistent.
• End-to-end tests: Major business processes can be completed.
• Performance baseline: Critical paths meet minimum performance requirements.
• Compatibility tests: Target environments are reproducible.
• Long-term stability observation: Continuous running reveals whether memory leaks, state contamination, or abnormal recovery failures occur.

For an operating system prototype, being able to boot is only basic validation. Real engineering also requires validating driver stability, file system consistency, process isolation, exception recovery, and installer reliability. Without such acceptance, demo results cannot be turned into production assets.

2. Acceptance Boundaries for Formalized Mathematical Proofs

The value of formalized proofs is to turn reasoning into a machine-checkable structure. The basic process usually includes defining objects, declaring assumptions, introducing rules, reasoning step by step, and having a checker verify each step.

For formalized tasks, acceptance should focus on:

• Whether definitions match the objects being verified.
• Whether assumptions are complete and whether any premises are missing.
• Whether the rule library is trustworthy.
• Whether each proof step can be accepted by the checker.
• Whether there are skipped steps, circular references, or undeclared lemmas.

It is important to note that reproducing an existing formalized proof and independently proposing a new proof are two different capabilities. The former emphasizes engineering, parallelization, and a closed verification loop. The latter still involves research innovation, problem formulation, and proof discovery. These two capabilities should not be confused during evaluation.

3. Acceptance Challenges for Unstructured Tasks

In addition to code and mathematics, long-running agents may also handle report writing, requirements analysis, knowledge organization, configuration generation, and process design. These tasks are difficult to fully formalize for acceptance, so layered criteria are needed:

• Structural completeness: Whether necessary sections, fields, or steps are included.
• Factual consistency: Whether key information is supported by sources.
• Logical consistency: Whether conclusions conflict with each other.
• Executability: Whether the output can be turned into subsequent actions.
• Human spot checks: Domain experts sample and confirm quality.

For unstructured tasks, it is better to use an acceptance mechanism of “machine pre-screening + human spot checks + risk grading” rather than fully relying on model self-evaluation.

4. Safety and Compliance: Long-Running Agents Cannot Be Judged Only by Capability Ceiling

Long-running agents often have stronger execution capabilities, so safety boundaries are more important than in short tasks. Whether a model or platform can complete a certain type of task depends not only on capability, but also on protection strategies, access eligibility, data permissions, and organizational compliance requirements.

1. High-Risk Tasks Need Separate Evaluation

In scenarios such as cybersecurity, vulnerability analysis, biological research, and sensitive data processing, an agent may be unable to directly execute certain tasks because of policy restrictions, or it may be routed to a more conservative processing path. These restrictions are not simply missing capabilities. They are risk control designs.

During evaluation, it is necessary to distinguish:

• Whether the task type belongs to a high-risk category.
• Whether blocking rules apply globally or are configured by scenario.
• Whether there is an alternative path after blocking.
• Whether the alternative path introduces human review.
• Whether human review becomes the main bottleneck.

For enterprise users, security and compliance are not optional additions. They are prerequisites for selection.

2. Access Eligibility and Product Form Must Be Recorded Separately

The same underlying capability may be provided in different service forms. Different forms may differ in protection level, open scope, audit requirements, and organizational access eligibility. Therefore, selection cannot record only model capability. It also needs to record:

• Whether the current organization is eligible to use the capability.
• Which protection strategies are enabled for that service form.
• Which tasks require additional approval.
• Whether there are regional, industry, or data-level restrictions.
• Whether audit logs meet internal requirements.

If a capability has a high benchmark score but the current organization cannot legally and compliantly invoke it, that score has little practical business meaning.

3. Auditing and Traceability Are the Foundation of Long-Running Agent Systems

Long-running agents need to record a complete execution trace, including:

• Task goals and initial requirements.
• Each task decomposition result.
• Agent roles and division of labor.
• Tool call parameters and returned results.
• Human approval records.
• Artifact versions and change history.
• Failure reasons and recovery actions.

Without an audit trail, it is difficult for a long-running agent system to enter core business processes. Once an error occurs, the team cannot locate the source of the problem, nor can it prove that the delivery process was compliant.

5. Cost-Effectiveness: Single-Call Cost Is Not Enough

The cost structure of long-running agents is much more complex than short question-and-answer tasks. Many teams evaluate only model call prices and ignore engineering costs. Real cost-effectiveness should be measured from two dimensions: total cost of ownership and effective delivery value.

1. Cost Components

The cost of a long-running agent includes at least:

• Model call cost.
• Time cost caused by inference waiting.
• Tool execution cost, including sandbox, testing, compilation, and scanning.
• Storage cost, including code, logs, checkpoints, and artifact archiving.
• Human review cost.
• Failure retry cost.
• System integration cost.
• Monitoring, alerting, and operations cost.
• Security and compliance cost.

Among these, failure retries and human review are often the most underestimated. A long task may consume many calls and tool resources, but only tasks that finally pass acceptance count as effective delivery.

2. Effective Delivery Cost Formula

An estimation can use the following approach:

Unit effective delivery cost = Total cost / Number of valid tasks that pass acceptance

The total cost should include:

Total cost = Model call cost + Tool execution cost + Storage cost + Human review cost + Operations cost + Failure retry cost

The number of valid tasks should be counted according to acceptance criteria, not according to the number of tasks that finished running.

3. Recommended Key Metrics

Enterprises can establish the following metric system:

• First-pass rate: The proportion of tasks that pass acceptance without human rework.
• Human intervention rate: The proportion of tasks that require human approval, repair, or additional explanation.
• Average repair time: The average time from discovering a problem to completing a repair.
• Task cancellation rate: The proportion of tasks terminated due to risk, cost, or quality issues.
• Rework rate: The proportion of deliverables returned for rework.
• Single-task cycle time: The total duration from task creation to final acceptance.
• Cost variance rate: The deviation between actual cost and budgeted cost.

These metrics better reflect real productivity than model scores alone.

6. Reference Architecture for Engineering Deployment

To truly put long-running agents into use, teams should build a layered architecture rather than exposing models directly to business systems.

1. Control Plane

The control plane is responsible for task orchestration, budget control, permission approval, and risk grading. Core capabilities include:

• Task creation and decomposition.
• Subtask priority ordering.
• Budget and timeout control.
• Approval workflow for high-risk tasks.
• Entry points for manual takeover.

2. Execution Plane

The execution plane is responsible for actually calling agents, tools, and sandbox environments. Core capabilities include:

• Isolation of code execution environments.
• File system isolation.
• Network access control.
• Testing and static checks.
• Tool call auditing.

3. Data Plane

The data plane is responsible for saving task status, context, artifacts, and logs. Core capabilities include:

• Task status persistence.
• Checkpoint saving.
• Artifact version management.
• Context summaries.
• Execution trace archiving.

4. Governance Plane

The governance plane is responsible for monitoring, alerting, auditing, and cost analysis. Core capabilities include:

• Abnormal task alerting.
• Cost overrun warnings.
• High-risk behavior blocking.
• Call chain tracing.
• Periodic quality review.

If a team builds such a system on Baidu AI Cloud, it can use general cloud-native capabilities for implementation: use a container platform to host agent services and sandbox tasks, use a message queue to implement asynchronous task distribution and decoupling, use a logging service to record task traces and tool calls, use monitoring and alerting to control anomalies and budgets, and use object storage to archive code artifacts, test reports, and audit records. This architecture helps turn long-running agents from single-point demonstrations into sustainable engineering systems.

7. Example Task Orchestration Pseudocode

The following pseudocode shows a basic control flow for a long-running agent. The focus is on planning, execution, verification, repair, checking, and human approval:

from dataclasses import dataclass
@dataclass
class Task:
    task_id: str
    goal: str
    acceptance: str
    risk_level: str
def run_long_task(goal: str, acceptance: str, risk_level: str):
    plan = create_plan(goal)
    budget = load_budget()
    while not plan.is_finished():
        if budget.is_exceeded():
            pause_for_manual_review("Budget exceeded")
        task = plan.next_task()
        if task.risk_level == "high":
            approval = request_human_approval(task)
            if not approval:
                plan.mark_blocked(task)
                continue
        result = execute_in_sandbox(task)
        if not verify(result, task.acceptance):
            result = repair(task, result)
        if verify(result, task.acceptance):
            plan.mark_done(task, result)
        else:
            escalate_to_human(task, result)
    final_result = integrate(plan)
    if not final_acceptance_check(final_result, acceptance):
        return request_manual_review(final_result)
    return deliver(final_result)

This pseudocode has three key points:

1. Every task has a clear acceptance criterion.
2. Execution results must pass verification before moving to the next step.
3. High-risk situations, exceptions, and budget overruns must pause the process and enter a human workflow.

8. Common Failure Modes and Mitigations

1. Goal Drift

During long tasks, agents may gradually drift away from the original goal because of context compression or changes in subtasks. Mitigations include maintaining a goal card, regularly checking against the original requirements, and performing consistency review during the integration stage.

2. Context Fragmentation

In multi-agent collaboration, if context is not shared, interface inconsistencies or duplicate development can easily occur. Mitigations include unified task specifications, a shared change log, checkpoints, and versioned context.

3. Test Fabrication or Self-Certified Passing

Agents may generate tests that appear to pass but do not cover key scenarios. Mitigations include introducing an independent review agent, using fixed acceptance scripts, performing human spot checks, and checking coverage.

4. Tool Abuse

Agents may call tools too frequently, causing costs to rise, or repeatedly try in the wrong direction. Mitigations include tool call budgets, timeouts, frequency limits, and error circuit breakers.

5. Excessive Permissions

If an agent has too many permissions, it may delete files, modify configurations, or touch sensitive data. Mitigations include the principle of least privilege, sandbox isolation, approval for dangerous operations, and audit trails.

6. Optimizing Only for Success Cases

If a team evaluates models only with successful cases, it will ignore failure paths in real business. Mitigations include fault injection, boundary task sets, long-cycle regression testing, and adversarial acceptance.

9. Enterprise Adoption Recommendations: Start with Low-Risk Scenarios

For teams preparing to adopt long-running agents, it is not advisable to hand core business processes entirely to agents from the beginning. A more prudent path is to start with low-risk, high-certainty, acceptable scenarios.

Scenarios suitable for pilots include:

• Unit test generation and supplementation.
• Code comment and documentation generation.
• Log analysis and exception summaries.
• Configuration checks and consistency validation.
• Structured organization of requirements documents.
• Low-risk refactoring and module migration.
• Automatic generation of test reports.
• Knowledge base question answering and material aggregation.

These scenarios share common characteristics: acceptance criteria are relatively clear, the impact of failure is controllable, and human cost can be reduced through automated checks.

During the pilot stage, teams should establish four lists:

1. Task list: Define which tasks agents are allowed to execute.
2. Acceptance list: Define the passing criteria for each task.
3. Risk list: Define which tasks require human approval.
4. Cost list: Define budgets, timeouts, and alert thresholds.

After the pilot runs stably, teams can gradually expand to tasks with higher complexity and longer cycles.

10. Conclusion: The Real Threshold for Long-Running Agents Is Engineering

Progress in long-running agents has made task decomposition, parallel execution, tool use, and result integration practical. Public cases show that agents can complete complex prototype building over multiple days and can significantly improve execution efficiency in formalized verification tasks. However, these results must be understood within specific acceptance criteria.

For technical teams, evaluating long-running agents should not only ask whether they can do something. Teams should also ask:

• Can the agent continuously decompose tasks under a clear goal?
• Can it recover from failures, rather than only showing successful paths?
• Can it provide auditable, reproducible, and rollbackable deliverables?
• Can it run under safety and compliance constraints?
• Is its total cost of ownership lower than human or traditional automation approaches?

Only by combining capability, acceptance, safety, cost, and engineering architecture can long-running agents move from demonstrations to production. The real competitiveness is not a single impressive demo, but whether the system can stably, controllably, and sustainably complete enterprise-level tasks.