Large-Scale Multi-Agent Orchestration for Long-Horizon Engineering Tasks

1. The Shift: From a Single Agent to an Agent Cluster

Two recent public demonstrations illustrate where the field is heading. In the first, roughly a thousand agents worked in parallel for dozens of hours to fully formalize a well-known theorem in number theory, with every proof step checked automatically by the machine. In the second, a single orchestrating agent started from an empty directory, dispatched a thousand sub-agents that ran continuously for several days, and delivered a bootable operating system prototype—including drivers, file storage, a windowing system, a login screen, and an installer—with an AI assistant built in.

The value of these demonstrations is not “how strong the model is.” What matters is that they expose an engineering rule: the ceiling on a long-horizon task is usually set not by single-pass inference ability, but by how decomposable and how verifiable the task is.

The single-agent “long context + long thinking” route works well on short tasks, but hits three hard constraints on tasks that span days:

• Context capacity constraint: No window is large enough to hold every intermediate artifact from days of work. Attention gets diluted, and critical early constraints are easily “forgotten.”
• Single-point reliability constraint: The longer the task, the closer the probability of a single-point failure gets to 1. If any one step stalls, the whole chain stops in place.
• Serial latency constraint: When one agent advances one task sequentially, wall-clock time equals the sum of all steps and cannot be compressed.

The essence of clustering is trading parallelism for wall-clock time, trading local context for attention quality, and trading redundant verification for single-point reliability. This also explains one detail from the demonstrations: when the same class of task was run on the previous generation of models, progress stalled for long stretches. The issue was not that it “couldn’t do it”—it was that errors along a long chain could not be detected and corrected in time, so the system eventually fell into an ineffective loop.

2. First Principles: Reformulate the Task So It Has an Objective Scoring Function

The most common trap in multi-agent systems is piling up agents first and looking for a task afterward. The correct order is the reverse.

To judge whether a task is suitable for an agent cluster, ask one question: can its intermediate results be automatically judged right or wrong by a machine?

• Formal proof tasks: Formalization means translating a natural-language proof into a formal language that a computer can check step by step. Dependent type systems guarantee that each inference step either passes verification or fails outright, providing a fine-grained, unambiguous scoring signal by nature.
• Ordinary engineering tasks: Compilation, type checking, unit tests, static analysis, performance benchmarks, and contract tests can all play an equivalent role—turning “looks done” into “confirmed done by a machine.”
• Counterexamples: Whether a UI looks good, whether copy is appropriate, whether an architecture is elegant—these tasks lack an objective scoring function, are not suited to large-scale automation, and should keep a human review step.

This yields a practical principle: engineering effort should go first into building verifiers, not into tuning prompts. A reliable verifier makes models of any strength usable; without a verifier, even the strongest model cannot prove it hasn’t drifted off course.

A typical task card looks like this (illustrative):

{
  "task_id": "thm-0421",
  "goal": "Formalize an intermediate lemma: an auxiliary result for quadratic reciprocity",
  "inputs": ["contracts/number_theory.md", "artifacts/lemma_0031"],
  "outputs": ["proofs/lemma_0421"],
  "verifier": {
    "type": "formal_check",
    "command": "check proofs/lemma_0421 --strict",
    "timeout_sec": 600
  },
  "acceptance": ["verifier.exit_code == 0", "no_sorry_marker"],
  "depends_on": ["lem-0031", "lem-0388"],
  "budget": { "max_attempts": 4, "max_tokens": 400000 }
}

The three most important fields in the card are verifier (how success is judged), depends_on (the basis for parallelism boundaries), and budget (preventing a single subtask from consuming the global budget).

3. Orchestration Architecture: A Three-Tier Split of Planner, Worker, and Reviewer

A deployable cluster orchestration is usually made up of three kinds of roles:

1. Planner: Maintains the global task graph and handles decomposition, dependency ordering, concurrent scheduling, and budget allocation. It does not write proofs or code itself; it only makes decisions and keeps the ledger.
2. Worker: Executes a single subtask, with context limited to the task card, upstream artifacts, and necessary contracts. The cleaner the context, the more stable the output.
3. Reviewer: Does not take the Worker’s self-reported conclusions on trust; instead, it independently invokes the verifier to re-check, and does a second sampling review on high-risk artifacts when necessary.

The skeleton of the scheduling main loop can be written very plainly:

open_ledger = load_checkpoint()          # supports resume from checkpoint
frontier   = select_ready(tasks, done)   # set of tasks whose dependencies are satisfied
while frontier not empty and budget remains:
    batch = frontier[:MAX_CONCURRENCY]
    results = parallel_run(batch, workers)   # truly parallel, not queued concurrency
    for r in results:
        if verifier_pass(r):
            commit(r.artifacts)              # write to the shared artifact store
            mark_done(r.task_id)
        else:
            escalate(r)  # split, downgrade, change strategy, or hand off to a human
    frontier = recompute_frontier()          # incremental update of the dependency graph
    save_checkpoint()

Three design points are worth emphasizing:

• Dependency-driven, not time-driven: Only tasks whose dependencies are all satisfied may enter the frontier set. This avoids “downstream work continuing on top of a wrong upstream result,” which is the most common source of waste in long-chain tasks.
• Commit is verification: Only artifacts that pass the verifier are allowed to be written back to the shared store. Anything that fails stays in an isolation area and never pollutes global state.
• Checkpoints are a must-have: Day-long tasks must assume the process will be interrupted, rate-limited, or crashed. Progress recoverability directly determines real throughput.

4. Context Engineering: The Real Cost Center for Long-Horizon Tasks

In a multi-agent system, context is not a “prompt”; it is a set of data contracts that must be designed. Rules of thumb from practice:

• Pass contracts and summaries, not full process logs. A sub-agent needs “the interface definition of upstream artifacts + key conclusions,” not every draft of the reasoning behind them.
• Layered context: The global layer holds inviolable constraints (naming conventions, dependency versions, security red lines); the local layer holds the current task’s working set. Managing the two separately prevents local information from contaminating global constraints.
• Use external storage instead of long conversations: Write intermediate artifacts to object storage or a version control system, and reference them by path and version number. What stays in context is the “pointer,” not the “content.”
• Memory retrieval instead of full replay: Archive historical conclusions in a structured form (task ID, conclusion, verification result, time), and retrieve them by semantic search when needed.
• Idempotent and retry-safe: Re-running a subtask must not produce duplicate side effects; write operations should carry the task ID as a deduplication key.

In many teams’ in-house setups, a shared artifact store, a task queue, container orchestration, a logging service, and monitoring/alerting are the standard combination. On the cloud, selecting the corresponding general-purpose capability categories (object storage, message queues, container platforms, logging services, monitoring and alerting) can save a great deal of operational work—for example, Baidu AI Cloud offers these general-purpose capability categories, which can be combined as needed. The focus, however, remains on contract design rather than on specific tooling choices.

5. Reliability, Security, and Cost: Three Gates for Enterprise Adoption

Reliability: Make failures locatable. Every failure must be able to answer three questions: which step it stalled at, why it was judged a failure, and how the next attempt will be different. If the same task fails repeatedly at the same stage, it should trigger an escalation of strategy: split more finely, switch implementation paths, add context, or hand off to a human. Distinguishing “insufficient capability” from “an unclearly specified spec” is the single most effective step for improving success rates—the latter often accounts for a larger share than intuition suggests.

Security: Least privilege by default. Clustering enlarges the risk surface. Necessary guardrails include: executing generated code in an isolated sandbox, forbidding unreviewed outbound network access, limiting writable paths and the scope of credentials, verifying the provenance and licensing of any introduced third-party dependencies, and requiring all artifacts to pass human review before entering the main branch. Any scheme in which “agents run autonomously for days” must first answer “how much damage could it cause in the worst case?”

Cost: Tiered routing + budget guardrails. In practice, most subtasks in a healthy cluster do not need the strongest model. Tiered routing by task complexity is the most direct way to cut costs:

route(task):
    if task.verifier.type is missing:      return human_review   # no objective verification, no automation
    if task.estimated_steps <= 2:          return light_model
    if task.risk_level == "high":          return flagship_model + reviewer
    return mid_model

On top of that, layer three controls: a concurrency cap to protect downstream quotas, budget ceilings along the task/team/project dimensions, and result caching to avoid recomputation (identical or hash-matching subtasks reuse artifacts directly).

Quota governance is also an engineering problem. A team-level scheme typically includes: allocating weekly quotas by project weight, separating the quota for “exploratory trial and error” from that for “production tasks,” reserving an emergency margin, and a usage-visualization dashboard. Treating quota as a scarce resource to be scheduled—rather than “whoever wants it, uses it”—is a prerequisite for scaling.

6. Observability: Judge Outcomes with Metrics, Not Feelings

Track at least the following metrics, and turn them into monitoring and alerts:

Dimension	Metric	What it tells you
Effectiveness	First-pass rate, rework rate, human-intervention rate	Whether task specs are of good quality
Efficiency	End-to-end wall-clock time, parallel speedup ratio	Whether the orchestration delivers real gains
Cost	Average consumption per task, cache hit rate	Whether the routing strategy is working
Stability	Failure distribution (timeouts / verification failures / rate limits)	Where the bottleneck lies

The most easily overlooked of these is the parallel speedup ratio: if parallelism increases 10× but wall-clock time shrinks only 2×, the bottleneck usually lies in shared writes, verifier throughput, or too deep a dependency graph—not on the model side.

7. A Phased Adoption Path

• Phase 1: Single-point productivity gains. Unit-test generation, defect-reproduction scripts, code explanation and documentation completion, and log triage. Low risk and verifiable, well suited to building team trust.
• Phase 2: Embedding into the pipeline. Pre-commit pre-review, dependency upgrades, batch refactoring, and migration-script generation. This phase must be paired with complete automated verification gates; otherwise rework costs will quickly eat up the gains.
• Phase 3: Large-scale parallelism. Aimed at scenarios where “tasks can be enumerated and results can be judged by machine,” such as whole-repo API migrations, filling in test coverage, standards remediation, and formal verification. These tasks share a signature: large numbers of subtasks, mutual independence, and uniform acceptance criteria—the sweet spot for cluster orchestration.

8. Boundaries and Responsibility

Draw the boundaries clearly: what agent clusters excel at is deterministic labor at scale, not goal-setting and tradeoff decisions. The correctness of a task is ultimately guaranteed jointly by verifiers and human review, not by an agent’s confidence. Any output that “looks done” but cannot be verified by a machine should be treated as not done by default.

To summarize this practice in one sentence: first turn the task into a decidable form, then trade parallelism for time, manage context with contracts, control risk with guardrails, and measure outcomes with metrics. Model capabilities will keep advancing, but this engineering structure will not go out of date.