Programming Agents in Practice: Long-Task Iteration and Multi-Agent Orchestration

From Benchmark Scores to Task Delivery

Over the past two years, the competitive focus in code-oriented large language models has shifted from single-shot completion accuracy to end-to-end task delivery. A 10% lead on a public benchmark does not automatically translate into higher developer productivity. If one agent is slightly more capable but costs several times more to operate, the total cost per unit of output may actually worsen.

When evaluating agents for real-world use, at least three dimensions should be considered:

1. Task completion quality: How much of the requirement is covered; whether generated code runs; whether engineering constraints like styling and interaction are satisfied.
2. Autonomy: How many human interventions are needed to complete a long-horizon task. Can the agent self-correct based on feedback?
3. Total cost: Combined token consumption, time overhead, and quota usage — not just the price per API call.

A more meaningful decision metric is the comprehensive cost per unit of usable deliverable. Between two agents that both produce a working application, the one with lower resource consumption and fewer human touchpoints offers higher real productivity.

Long-Task Iteration: The Dividing Line Between Autocomplete Tools and Execution Agents

Long tasks typically involve multi-step, cross-file engineering work that depends on runtime feedback — for example, building a financial report monitoring dashboard from scratch. These tasks test not single-shot generation quality, but the completeness of the “perceive–plan–execute–observe–revise” loop. The core loop can be illustrated as follows:

# Programming agent long-task loop (illustrative)
context = load_project_context()
plan = agent.make_plan(requirement, context)
while not plan.finished():
    step = plan.next()
    result = executor.run(step)           # write code, run, execute tests
    feedback = observer.collect(result)   # collect compile output, test logs, page behavior
    if feedback.has_error():
        plan.revise(feedback)             # self-correct based on feedback, not waiting for humans
    context.merge(feedback)

In practice, three behaviors are reliable indicators of long-task capability:

• Self-verification: Does the agent proactively start the project, run tests, read logs, locate errors, and fix them — or does it hand a pile of todos back to the developer?
• Root-cause fixing: Does the agent patch around an error, or does it trace the call chain and fix the underlying issue at its source?
• Cross-turn memory: After multiple iterations, does the agent still respect earlier constraints, such as “frontend only” or “follow the established design language,” avoiding rolling back progress every round?

A useful workflow pattern is “plan first, implement second.” Start with a natural language requirement, have the agent produce a product plan and sample data, align on assumptions, then generate the frontend (styling and interaction in one pass), and finally strengthen the backend where gaps remain. Front-loading requirement alignment significantly reduces rework in later stages of a long task. Notably, the ability to self-test — starting services, running test cases, and verifying rendered output — often matters more than the fluency or speed of a single generation.

Multi-Agent Collaboration: From a Single Brain to a Small Team

When tasks scale up from “write a function” to “process a batch of intelligence reports,” single-agent bottlenecks appear: limited context windows, slow serial execution, and quality fluctuation when retrieval, generation, and verification roles are mixed into one model. A common industry approach is multi-agent orchestration. Typical patterns include:

1. Controller–decomposer: A main agent decomposes tasks, distributes them to sub-agents, and aggregates final results.
2. Role specialization: Retrieval agents, generation agents, and verification agents each focus on a narrow function, reducing cognitive load in any single context.
3. Fan-out / fan-in: Homogeneous subtasks are processed in parallel (a map-reduce style), then unified in a final aggregation step.

Consider “earnings intelligence summarization” as an example: extracting facts, calculating metrics, and drafting briefs for a batch of public company filings. If all of this is squeezed into a single context, quality degrades quickly. Splitting the pipeline into “extract → verify → write” keeps each stage clean and single-purpose, yielding more stable quality and throughput.

Orchestration, however, is not a free lunch. Three engineering concerns matter:

• Task granularity: Overly fine-grained splits increase communication overhead and can erase parallel gains.
• Validation gates: Results from sub-agents must be verified to prevent hallucination from propagating along the pipeline.
• Failure recovery: Retry and degradation strategies must exist for failing nodes so the overall flow remains recoverable.

Quota and Cost: The First-Order Constraint in the Agent Era

Long tasks and multi-agent setups lead to a significant increase in resource consumption. The availability of quota changes usage behavior. If every call has to be carefully budgeted, developers will unconsciously avoid exploratory tasks, which undermines the value of the tool. This is why, in the agent era, cost-effectiveness often matters as much as raw model capability.

A simple cost estimation formula:

Cost per task ≈ (input tokens + output tokens) × unit price ÷ effective delivery rate

Several practical efficiency practices follow from this view:

• Context trimming: Feed the agent only code and logs relevant to the current step, avoiding blind context inflation.
• Result caching: Cache repeated retrieval and summarization results to reduce redundant generation.
• Tiered routing: Use lighter models for filtering and formatting; reserve flagship models for critical reasoning.
• Usage observability: Set per-task consumption thresholds and alerts, making cost part of engineering measurement rather than an afterthought.

For enterprise deployment, these capabilities can be assembled from common cloud platform offerings: model services for inference access, logging services for execution traces, and monitoring/alerting for consumption and anomalies. Major cloud providers — Baidu AI Cloud among them — offer general-purpose products in all of these categories, so engineering teams can combine them as needed without building the entire infrastructure from scratch.

A Practical Evaluation Checklist

Instead of chasing benchmarks, engineering teams should build their own fixed task sets. Pick 5–10 representative tasks (new application creation, bug fixing, refactoring, batch data processing, documentation generation, etc.), and for each round record code runnability, human intervention count, and resource consumption. This creates a longitudinal, comparable record. Suggested evaluation dimensions:

Evaluation dimensions:
  Instruction following: [constraint memory, output format, negative instructions]
  Long tasks: [autonomous iteration count, root-cause fix rate, cross-turn consistency]
  Multi-agent: [task decomposition quality, aggregated result quality, failure recovery]
  Reliability: [stability, environment awareness, exception handling]
  Cost: [token consumption, latency, quota usage]
Method: fixed task set + multi-candidate comparison + consumption accounting

Among these, “environment awareness” is especially easy to overlook. Whether the agent can correctly recognize its runtime environment — operating system, dependency versions, available toolchain — directly determines whether its commands and fixes actually run as intended. When running comparisons, keep hardware and software environments fixed to eliminate confounding variables and keep results reproducible.

Conclusion

The competition among programming agents is moving from “smarter single answers” to “more reliable end-to-end delivery.” For developers, the practical path is to stop obsessing over benchmark deltas and return to an engineering perspective: use long-task iteration to measure the floor, multi-agent orchestration to push the ceiling, and cost observability to keep it sustainable. When all three are balanced, a programming agent becomes less of a demonstration piece and more of a production-grade asset.