Long-Horizon Execution and Multi-Agent Collaboration: LLM Engineering Practices
As large language models (LLMs) transition from simple Q&A tools to complex execution systems, developers face new challenges in managing long-horizon tasks. This article explores practical engineering approaches for long-horizon coding tasks, common multi-agent collaboration architectures, and essential engineering guarantees for long-running systems. It also covers the standardization of interfaces for heterogeneous agents to ensure maintainable collaboration.
1. From Q&A Tools to Execution Systems
Early typical usage of large language models (LLMs) involved single-turn Q&A or text generation. Users provided an input, the model returned a result, and the task was complete. As applications deepen, developer demands have evolved. They expect models to modify code across multiple files, run continuously for hours, invoke external tools, and adjust strategies based on feedback. These types of tasks are known as long-horizon tasks.
Long-horizon tasks introduce three new challenges. First is stability: the more steps involved, the higher the cumulative probability of a single-step error. Second is context: as task history continuously grows, compression and retrieval are required once it exceeds the model’s context window. Third is cost: each step consumes inference resources, and without budget constraints, expenses can quickly spiral out of control.
2. Core Capabilities for Long-Horizon Coding Tasks
Long-horizon coding is not about generating code in one pass; it is a loop: understanding requirements, planning, modifying code, running verifications, and fixing based on errors. The system must maintain state consistency at each stage.
This can be broken down into three foundational capabilities:
- Task Planning: Decomposing the goal into dependent subtasks and recording the completion status of each.
- Context Management: Compressing completed steps into summaries, writing key information to external storage, and retrieving it on demand.
- Self-Verification: Obtaining objective feedback through tests, compilation, and static analysis, rather than relying on the model’s judgment of its own output.
The following pseudo-code illustrates the basic structure of this closed loop.
def run_coding_task(goal, planner, executor, verifier, max_rounds=20):state = planner.make_plan(goal) # Generate a list of subtasksfor round_id in range(max_rounds):task = state.next_pending()if task is None:breakpatch = executor.apply(task) # Modify coderesult = verifier.check(patch) # Run tests and checksif result.passed:state.mark_done(task)else:state.attach_feedback(task, result.errors)return state.report()
The verification stage is critical to the entire loop. If verification only checks whether the model output “looks correct,” errors will be amplified in subsequent steps.
3. Common Architectures for Multi-Agent Collaboration
When a single Agent cannot complete a task within a reasonable timeframe, multiple Agents can be introduced to work in parallel or divide the labor. There are three common architectures.
| Pattern | Structure | Advantages | Main Risks |
|---|---|---|---|
| Master-Slave | A dispatcher assigns tasks; multiple executors return results | Clear responsibilities, easy to control | Dispatcher may become a bottleneck |
| Pipeline | Stages are connected in series; previous stage output is the next stage input | Easy to understand and debug | Low parallelism, errors propagate between stages |
| Shared Blackboard | Agents autonomously claim tasks around a shared state | Flexible, suitable for exploratory tasks | State conflicts, requires concurrency control |
In actual systems, the master-slave pattern is the most common. The following example uses a semaphore to limit concurrency, preventing sub-agents from exhausting compute resources simultaneously.
import asyncioasync def run_worker(sem, subtask, worker):async with sem: # Limit the number of simultaneously running sub-agentsreturn await worker.run(subtask, timeout=600)async def orchestrate(subtasks, worker, max_parallel=8):sem = asyncio.Semaphore(max_parallel)results = await asyncio.gather(*(run_worker(sem, t, worker) for t in subtasks),return_exceptions=True,)return merge_results(results) # Aggregate results and handle failures
More parallelism is not always better. Factors determining the upper limit include: dependencies between subtasks, write conflicts on the same file or resources, cost budgets per task, and the overhead of result aggregation and consistency checks. If the splitting granularity is too fine, coordination costs may exceed the benefits of parallelism.
4. Engineering Guarantees for Long-Running Processes
When Agents run continuously for hours or longer, the system must be recoverable, observable, and constrainable.
Checkpointing: Upon completing each subtask, write the state to persistent storage. If the process is interrupted, recover from the most recent checkpoint rather than starting from scratch.
Timeout and Heartbeat: Set a timeout for each subtask and report heartbeats periodically. Agents that are unresponsive for extended periods should be terminated and rescheduled.
Budget Limits: Set maximum call counts, token usage, and execution time for a single task. Pause execution and request human confirmation when limits are reached to prevent cost overrun.
Permissions and Confirmation: Operations like deleting files, deploying changes, or modifying permissions are high-risk. They should have human confirmation nodes and follow the principle of least privilege.
Observability: Record the input, output, execution time, and verification results of each step. Logs must be desensitized to prevent leaking keys and user data.
Traffic issues during peak periods also need separate consideration. When request volume exceeds system capacity, core functions should be protected through rate limiting, queuing, and degradation strategies, while clearly communicating the current status to users.
5. Interface Standardization for Heterogeneous Agents
In real-world projects, different Agents often use different models, tools, or runtime environments. Without unified interface conventions, collaboration becomes difficult to maintain. It is recommended to standardize three aspects:
- Message Format: All Agents use the same task and result structure.
- Capability Declaration: Each Agent declares its supported task types, input requirements, and resource needs.
- State Semantics: Uniformly define states like pending, running, done, failed, and their transition rules.
Below is an example of a generic task message structure.
{"task_id": "t-0042","type": "code_patch","input": {"repo_path": "/workspace/app", "target_files": ["src/service.py"]},"constraints": {}}
