Evaluating Long-Task AI Agents: Delivery Capability and Full Cost

Long-Task Capability Is More Than “Can It Be Built?”

Evaluation of AI agents is shifting from single-turn Q&A and snippet generation toward whether an agent can keep advancing complex work across multiple stages. A typical long task requires breaking down a goal, making a plan, writing an implementation, running tests, fixing defects, and adjusting the approach based on feedback. Compared with one-shot generation, these tasks better reveal planning, tool use, state management, and error recovery.

Recent public experiments illustrate two representative outcomes. In one, agents starting from an empty directory coordinated many sub-agents to build a prototype operating system that could boot. In the other, multiple agents helped turn a mathematical proof into a formal expression that a computer can check step by step. These results suggest agents can take on longer chains of work. They do not, by themselves, show that a system is ready to ship, can replace a professional team, or is cheaper.

Any evaluation should start with a basic question: what standard defines “complete”? A program that boots, a proof that passes a checker, behavior that matches requirements, and long-term operational stability are different levels of acceptance. If those levels are not distinguished, useful experimental progress is easy to misread as mature production capability.

What Each Kind of Experiment Actually Shows

Building a Prototype System from a Blank Environment

The notable part of the prototype operating-system experiment was not merely generating source files. The goal was split into kernel, device drivers, networking, persistent storage, a graphical interface, a login flow, and an installation process, with different agents exploring and implementing those parts in parallel. The system eventually booted in a laptop environment, which shows that this style of collaboration can organize related subtasks into a runnable whole.

“It boots” covers only a limited verification scope. It does not by itself prove adequate security, stability, hardware compatibility, or operational readiness. Device drivers may support only a small set of hardware. A file system may corrupt data after an unexpected power loss. A network stack may lack sufficient handling of boundary inputs and concurrent scenarios. A visible interface does not mean access control, update mechanisms, and failure recovery are complete.

Results like this are better treated as evidence of end-to-end engineering organization than as a conclusion about production-grade quality. They show that agents can keep contributing code and test results across a long workflow, while engineers still need to confirm requirement boundaries, review architecture, design stress tests, and own the decision to ship.

Turning a Mathematical Proof into a Checkable Form

A formal proof differs from a natural-language proof because each reasoning step must be expressed as a strict logical object and checked by a verifier. When agents help reproduce an existing proof, they may save time in reading source material, constructing definitions, filling in intermediate derivations, and handling checker errors.

If several agents work in parallel, what shrinks is the elapsed time from problem decomposition to a passing verification. That does not mean compute or human effort falls by the same proportion. The task may still depend on existing proofs, mature formal libraries, clear definitions, and a domain expert’s explanation of the goal. Formal verification supplies a strict check on the reasoning process; it does not replace a judgment about whether the problem was modeled correctly.

Comparisons of elapsed time across experiments should also be cautious. Difficulty, starting materials, available tools, degree of parallelism, runtime environment, and success criteria all affect the result. When those conditions differ, judging capability by “how many hours it took” alone is rarely comparable.

Gains and Costs of Multi-Agent Collaboration

Splitting a complex task across multiple agents can, in principle, parallelize implementation, test writing, research, and defect analysis, and reduce serial waiting. Different agents can also inspect the same design from different angles and raise the chance of catching omissions. The approach is especially attractive when module boundaries and interfaces are clear.

Parallelism is not free. Subtasks must share context, agree on interfaces, merge results, and resolve conflicts. With weak planning, agents may duplicate work or produce contradictory implementations. A coordinator has to track task state, judge dependencies, and reassign work when a subtask fails. As the number of participants grows, communication and review overhead can rise as well.

Agent count is therefore not a capability metric. Results depend on whether the work can be split, how tightly modules are coupled, whether the collaboration is observable, and whether the final output is independently verified. For highly coupled work, frequently changing requirements, or tasks where errors are extremely costly, too much parallelism can increase integration risk.

An Evaluation Framework from Demo to Deliverable System

When assessing long-task automation, acceptance can be split into these layers:

1. Functional correctness: Do the target features work as specified, including normal and exceptional paths?
2. Quality and security: Were code review, dependency checks, permission verification, and security testing performed, and are critical risks explicitly handled?
3. Stability and compatibility: Can the result keep running in the target environment, and has it been checked for load, failure recovery, and compatibility?
4. Maintainability: Are structure, documentation, tests, and change history sufficient for later development, rather than only a one-time demo?
5. Process traceability: Are plans, subtasks, changes, test results, and human interventions recorded so that problems can be traced?
6. Resources and cost: Are model calls, compute, repeated attempts, human review, and rework counted as a complete investment?

These measures should match task risk. An internal prototype can accept a limited feature scope and manual repair. A user-facing core system needs stricter security, reliability, and operations acceptance. The same definition of “success” should not be applied to every setting.

Cost-Effectiveness Needs the Full Bill

Whether agents are economical cannot be judged only by model-call fees or by wall-clock time from start to finish. A more useful cost model is:

Total cost = automated run cost + human planning and review cost + rework cost + risk and maintenance cost

Run cost includes inference and supporting compute. Human cost covers clarifying requirements, decomposing tasks, checking results, and release review. Rework cost comes from incorrect implementations, repeated attempts, and failed integration. Risk and maintenance cost includes security flaws, long-term upkeep, and incident handling.

Evaluation should use repeated trials, not a single selected success. A set of real tasks should record completion rate, amount of human correction, defect count, delivery cycle, and total resource use, then compare those figures with the existing workflow. It should also separate “fully automatic,” “completed after human guidance,” and “a draft that people largely redid.” Otherwise the share of work attributed to automation will be overstated.

Uses That Fit Current Practice

At this stage, agents fit work with clear boundaries, verifiable results, and reversible failure—for example, drafting tests, organizing migration steps, implementing isolated prototype modules, or helping construct intermediate steps in a formal proof. Work that involves production privileges, sensitive data, critical security boundaries, or irreversible actions should keep human approval and least-privilege control.

A cautious workflow usually looks like this: a person states the goal and acceptance criteria; the agent breaks down the work and proposes a plan; every change and test result is recorded during execution; critical stages pass automated tests or independent review; and a responsible owner decides whether to deliver. Requirements that cannot be checked automatically should be marked for human review, not treated as already satisfied.

Conclusion: Verifiable Delivery, Not a Single Demo

Prototype systems and formal-proof cases show that agents can handle long tasks, coordinate multiple steps, and produce results that can be checked. They are reason enough for engineering teams to revisit where automation should stop. They are not enough, on elapsed time or a demo alone, to judge production maturity.

A useful judgment rests on explicit acceptance criteria, reproducible tests, a complete cost account, and risk tiering. For developers, the question is not whether an agent can finish everything on its own. It is which steps can be automated reliably, which must be reviewed by people, and whether automation actually reduces total work.