A Practical Framework for Multi-Agent Collaboration on Long-Running Software Engineering Tasks

1. Why Long-Running Tasks Need Multiple Agents

Large language models are effective at well-defined, single-turn problems, such as implementing a function, completing unit tests, or analyzing an error log. The nature of the problem changes when the goal is to build a runnable system from scratch. Such work may continue for days, involve dozens or even hundreds of modules, and require intermediate state to be preserved. Errors must also be discovered and corrected in later stages. Because a single session has a limited context window, it is difficult for one agent to handle work spanning this scale on its own.

A multi-agent architecture divides a large project into smaller tasks. Different agents handle different responsibilities, while shared state and a unified verification process are used to consolidate the results. This resembles the division of labor in a human software team: requirements specialists define objectives, developers implement features, testers design test cases, and reviewers enforce quality standards.

More agents do not necessarily produce better results. Additional roles introduce communication overhead, inconsistent state, and higher costs. A multi-agent approach is most appropriate when a task can be clearly decomposed, the interfaces between subtasks are stable, and acceptance criteria can be checked automatically.

2. Core Roles and Responsibilities

A typical engineering-oriented multi-agent system includes four types of roles.

1. Planner: Reads the overall objective and produces a task list, dependency graph, and priorities. The planner does not write code directly. Its responsibility is to keep the project aligned with the intended direction.
2. Worker: Receives an individual task, implements the required functionality within an approved file scope, and submits a description of the changes.
3. Reviewer: Checks whether the changes comply with interface contracts, coding standards, and security requirements, then provides approval or revision feedback.
4. Verifier: A deterministic system composed of test suites, static analysis, and build scripts. Verification results—not a model’s subjective judgment—provide the final basis for acceptance.

Separating these roles creates checks and balances. If the worker is also responsible for validating its own output, it may be inclined to assume that the implementation is correct, allowing defects to remain hidden. Assigning the final decision to deterministic programs helps reduce this self-confirmation bias.

3. Task Decomposition

The quality of task decomposition determines the upper limit of execution quality. The following principles are useful:

• Define interfaces first: Specify function signatures, data structures, and error-handling conventions before assigning implementation tasks.
• Use an appropriate level of granularity: A task should generally involve no more than a few hundred lines of expected changes and should be independently testable.
• Make dependencies explicit: Record task ordering in the task list to prevent multiple workers from modifying the same file at the same time.
• Keep scopes bounded: Each task should specify the directories it is allowed to modify. Changes outside that scope should be rejected immediately.

The following example shows a generic task description format. Field names can be adapted to a team’s conventions.

task_id: storage-004
goal: Implement a caching layer for block device I/O
depends_on: [storage-001, storage-002]
allowed_paths: [src/storage/cache/]
acceptance:
  - All unit tests under tests/storage/cache pass
  - Concurrent reads and writes do not produce data races
  - Public interfaces match the design document
budget:
  max_attempts: 3
  max_tokens: 200000

Acceptance criteria should be machine-checkable. Statements such as “the code should be high quality” cannot be evaluated automatically. By contrast, requirements such as “test coverage must meet the specified threshold and static analysis must produce no warnings” can be executed directly.

4. Designing the Verification Loop

The reliability of a multi-agent system primarily comes from its verification loop. A worker’s output is only a candidate result; it should be merged into the mainline only after passing verification. The process can be represented by the following pseudocode:

while unfinished tasks remain in the task list:
    task = planner.select_next_executable_task(state)
    changes = worker.implement(task, relevant_context)
    result = verifier.run(test_suite, static_checks, build, changes)
    if result fails:
        feedback = reviewer.analyze(result.logs)
        if attempt_count is within the limit:
            add feedback to context
            return to the worker step
        else:
            mark task as blocked
            notify a human operator
    else:
        reviewer.review(changes)
        if review is approved:
            merge into an isolated branch
            state.update(task, completed)
    checkpoint.save(state)

Three details are especially important.

First, the number of failed attempts must be capped. Otherwise, the system may continue spending resources on the same error indefinitely.

Second, blocked tasks must be explicitly handed over to a human. They should not be silently skipped, because later tasks may otherwise be built on incorrect assumptions.

Third, every state transition should be persisted. This allows the system to resume from a checkpoint after an interruption.

5. State Management and Checkpoint Recovery

Long-running tasks may span multiple process restarts, network interruptions, and resource scheduling events. State management should meet three requirements:

• Readable: Store state in structured files that include the task list, task statuses, attempt history, and key decisions.
• Traceable: Associate every change with a commit whose message contains the task ID, making it easier to determine why a change was introduced.
• Recoverable: When starting from any checkpoint, the system should be able to determine which tasks are complete and which need to be retried.

A version control system is a common way to provide traceability. It is advisable to let each task run in its own branch and merge it only after verification succeeds. A failed branch will then not contaminate the mainline, and rollback requires only discarding the relevant branch.

6. Controlling Cost and Reasoning Effort

The cost of a multi-agent system mainly comes from the number of model calls and the amount of reasoning performed per call. Cost overruns are often caused not by a single expensive call, but by unlimited retries and unproductive exploration.

In practice, costs can be controlled in four ways:

1. Allocate resources according to difficulty: Use lower reasoning effort for simple boilerplate tasks, and allocate larger budgets to tasks involving concurrency, security, or complex algorithms.
2. Set budget limits: Define maximum call counts and token budgets for both individual tasks and the overall project. Pause execution and notify an operator when a limit is exceeded.
3. Reuse context: Stable materials such as interface specifications and module summaries can be cached and reused instead of being submitted in full on every call.
4. Track unit cost: Use the average cost per accepted task as a key metric rather than looking only at the price of an individual model call.

Higher reasoning effort does not necessarily produce better final results. It may improve the quality of an individual response, but it also increases latency and cost. Whether the additional investment is worthwhile should be evaluated using acceptance rates together with the cost of human rework.

7. Evaluation Metrics

A multi-agent workflow should be evaluated with measurable indicators. At minimum, consider the following five metrics:

• First-pass rate: The percentage of tasks that pass verification without a retry.
• Final completion rate: The percentage of tasks that eventually pass acceptance within the defined budget.
• Regression rate: The number of times newly merged changes cause existing tests to fail.
• Number of human interventions: The number of cases requiring human judgment or correction, indicating the level of automation.
• Unit cost: The average cost required to complete one valid task.

Evaluation should use a test set independent of the development process. Otherwise, the system may achieve inflated results on examples it has already encountered or optimized for. Industry reports have indicated that multi-agent systems can autonomously complete substantial coding work over extended periods, but such results generally depend on robust testing infrastructure and human acceptance. Actual performance should be based on independent, reproducible evaluations rather than being directly generalized to other projects.

8. Security and Governance Boundaries

Autonomous execution systems must operate within clearly defined boundaries.

• Least-privilege access: Workers should access only the directories and services required for their assigned tasks. They should not have direct access to production credentials.
• Environment isolation: Code should be compiled and tested in a sandbox, with network access restricted through an allowlist.
• Human approval: Irreversible operations, such as deleting data, changing permissions, or deploying releases, must require human approval.
• Secret management: Credentials should be stored in a dedicated secrets service and must not be written into source code, prompts, or logs.
• Auditability: Model inputs, outputs, and tool calls should all be recorded for later review.

The licensing provenance of generated code and third-party dependencies also requires review. Unverified code and dependencies should not be introduced into a project without appropriate checks.