1. Where the problem starts: three ceilings of a single-model coding assistant

Over the past two years, the shape of the coding agent has converged: a model handles reasoning and generation, an execution framework reads and writes files, calls the terminal, and runs tests, and a session layer takes care of context assembly plus approval confirmation. Early on, these three layers were usually coupled together — the model you picked effectively decided which execution framework you used and which context strategy you followed.

Once teams started using these assistants on real projects, three ceilings showed up quickly:

1. Model capability is heterogeneous — there is no “globally best model.” Long-context refactoring, recall of niche framework APIs, cross-file semantic reasoning, and test generation are won by different models. Pushing all work onto one model means giving up the others’ strengths on specific task types.
2. Execution environments are fragmented. A developer’s task is often half local repository, half remote build machine or staging environment. Most assistants can only “work on the machine they happen to run on,” and cross-environment collaboration relies on manual copy and paste.
3. Context and approval flows are not unified. Every tool brings its own context management and permission confirmation logic. Switching tools means re-explaining the project background from scratch, and audit records end up scattered.

It is worth noting that the industry is not converging on a single direction: some products focus on pulling more office and collaboration entry points into one interface, while others expand toward a general-purpose assistant. The “engineers for engineers” approach is a different path — it does not chase entry-point unification, but the depth of orchestration itself: which models I have, which execution frameworks I have, which subscription quotas I have, and whether I can compose them per task.

2. The first key abstraction: decouple model, harness, and session

For “composition” to be possible, the first step is to break the three layers into replaceable components.

• Model layer (LLM): concerned only with inference requests and responses, including the output of tool-call intent. It should not know what the file system looks like.
• Harness layer: turns the model’s tool calls into real actions — reading and writing files, executing commands, running tests, searching the codebase. Its interface should be narrow and stable: submit one tool call, return a structured result.
• Session layer: responsible for context assembly, history trimming, approval confirmation, and cost and usage accounting.

Decoupling buys you freedom in orchestration. You can have model A perform code retrieval on harness 1 while model B does the implementation on harness 2; their session records, approval policies, and usage statistics stay independent, yet are scheduled by a single control plane.

The most common mistake here is binding the harness to the model. Many implementations ship “a prompt and a tool set custom-built for one specific model,” so switching models means switching harnesses. The sound approach is to define a capability manifest for each harness — which tools it supports, whether it supports streaming, whether it supports parallel tool calls, what the per-turn timeout is — and let a router pick the “model + harness” combination based on the task type.

3. The core of the architecture: the control plane and the execution plane must be separated

An engineering detail that gets repeatedly underestimated: the orchestrator should not drive what it orchestrates by “typing commands into a terminal” or relying on special calls from a UI layer.

There are three reasons:

• Terminal output is not a structured protocol. Using regular expressions to parse terminal text to decide whether a task finished or what the error was will fall apart on any non-trivial project.
• The terminal assumes a single session on a single machine. You cannot keep three tasks for three different environments alive in one terminal.
• There is no way to audit or replay. Commands and output are mixed together in a scrolling buffer; afterwards you cannot reconstruct which step ran, by whom, and under which permissions.

The architecture should therefore introduce an independent bidirectional channel that connects the orchestrator to every execution endpoint. WebSocket is a natural fit because it supports bidirectional push natively: an execution endpoint can proactively report tool-call events, incremental output, progress, and approval requests instead of waiting for the orchestrator to poll.

An illustrative message envelope:

{
  "v": 1,
  "id": "0f7c...",
  "type": "tool.call.request",
  "session": "s-1024",
  "env": "remote-build-03",
  "role": "implementer",
  "payload": { "tool": "write_file", "args": { "path": "src/api/user.ts" } },
  "deadline_ms": 30000
}

The key points on the server side:

async def dispatch(conn, msg):
    env = registry.get(msg["env"])          # Environment registry: health status, capability tags
    if not env.healthy:
        await reply(conn, msg, "error", {"reason": "env_unavailable"})
        return
    if not policy.allow(msg["role"], msg["payload"]["tool"], env):
        await request_approval(conn, msg)   # Suspend for human confirmation instead of executing silently
        return
    result = await env.executor.run(msg["payload"], timeout=msg["deadline_ms"])
    await reply(conn, msg, "result", result)
    audit.append(msg, result)               # Single audit entry point

The execution endpoint stays “thin”: it receives an instruction, executes it in a controlled environment, and returns a structured result. The environment can be your own machine, a container, or a remote build machine — as long as it can hold a long-lived connection, has an isolated file system, and declares its capability tags.

4. Remote environment management and cross-environment task delegation

Remote execution is not a new idea. The real difficulty is that a task can cross environment boundaries. A pragmatic model for this is a task tree:

• The parent task starts in the primary environment and owns the overall goal and acceptance criteria.
• Child tasks are dispatched to another registered environment — for example, “run an end-to-end regression in staging” or “produce artifacts on the build machine.”
• A child task returns only conclusions and the necessary artifact references, not an entire working directory copied back.

State synchronization needs explicit boundaries, otherwise data transfer will eat up the remote advantage. Recommended rules:

• Code and context travel through version control and an index; do not copy whole directories.
• Large artifacts (images, test reports, log bundles) are written to object storage, and the task message carries only a reference URI.
• Intermediate state lives on a shared volume or a cache layer, and each delegation gets its own temporary directory so concurrent child tasks do not pollute each other.
• Environment lifecycle must be managed explicitly: registration, health probing, leases, and timeout reclamation. A remote environment without a lease will inevitably turn into an unclaimed zombie machine within weeks.

A workable delegation interface needs only four parameters: target environment, role, input contract, and output contract.

sub = orchestrator.delegate(
    env="remote-e2e-runner",
    role="tester",
    inputs={"ref": "feat/user-profile", "artifact": "oss://builds/8231/app.tar.gz"},
    expects={"exit_code": "int", "report_uri": "str"},
)

Inputs and outputs are both described as contracts rather than “send the directory over.” Whether the remote side is a container or a physical machine, the orchestrator does not need to know the difference.

5. Multi-agent collaboration: three topologies and minimal context

The value of multiple agents is not “more people means more power.” It is that after role separation, each agent’s context is cleaner. Three topologies cover most scenarios:

1. Serial pipeline: research → design → implement → review → test. The advantage is that each step has a clear input; the drawback is high end-to-end latency. It suits higher-risk changes that need a human checkpoint.
2. Fan-out / fan-in (parallel): slice the task by module or test case, dispatch it to multiple agents to run in parallel, then have an aggregator merge the conclusions. It suits regression testing, multi-approach investigation, and large-scale refactoring. The key constraint is that slices must not share implicit state.
3. Supervisor–worker: one orchestrating agent holds the goal and acceptance criteria, creates worker agents as needed, and decides when to converge. It suits problems with a clear goal but an uncertain path.

Whichever topology you pick, context passing must follow a minimization principle: pass only conclusions, constraints, and the necessary evidence references — never the full conversation history. The thing that most easily gets out of control in practice is “the review agent gets the implementer agent’s full logs”: context balloons, cost goes up, and the quality of the conclusions goes down. A more reliable approach is to let each role maintain its own structured artifacts — design document, diff, test report — and have downstream roles read them on demand.

Roles can be split quite finely, but it is better to fix four basic ones first and split further only when needed:

• Researcher: retrieval and fact gathering only; not allowed to modify files.
• Implementer: allowed to change code only within the designated branch and directory.
• Reviewer: read-only access plus the ability to submit comments.
• Verifier: runs builds and tests in an isolated environment.

Permissions bound to roles are the precondition for this system to be trusted at all.

6. Governance: permissions, audit, cost, and observability

An orchestration system has a much larger risk surface than an ordinary assistant, because it holds multiple credentials and multiple environment entry points at the same time. At minimum you need:

• Credential isolation: each execution endpoint holds its own keys, and the orchestrator holds only delegation credentials. Keys must not enter session context and must not be written to logs.
• Tool allowlists: declare the allowed tools and path scope per role and per environment. Write operations are limited to the workspace and temporary directories, with no implicit escalation to system level.
• Approval gates: destructive actions (deletion, publishing, migration, bulk rewrites) must suspend and wait for human confirmation, and the approval record must be bound to the task ID.
• Audit and replay: all tool calls and results go into a unified log, so a task can be replayed in chronological order.
• Cost attribution: measure call volume and duration along four dimensions — session, role, environment, model — otherwise you cannot answer the question “how much more expensive is multi-model orchestration than a single model, actually.”

When you deploy this in the cloud, general-purpose capabilities are enough to fill in the governance surface; no proprietary interfaces are needed. Object storage holds artifacts and reports, a message queue carries child-task dispatch and smooths out spikes, a container platform hosts the isolated execution environments, and logging plus monitoring and alerting cover task success rate, queue wait time, cost per task, and environment health. Only when these metrics are on a dashboard can you avoid losing control as orchestration complexity grows.

7. Common anti-patterns and an evolution path

Anti-patterns to watch for:

• The giant single prompt: cramming every role and every tool description into one prompt. It looks good early on, but as soon as tasks get complex you get role confusion and tool misuse.
• No environment isolation: all agents share one working directory and parallel tasks overwrite each other. One temporary directory per task is mandatory.
• Infinite retries: retrying automatically whenever verification fails, without distinguishing error types. Classify failures instead: handle environment problems, dependency problems, and logic problems differently, and send logic problems back to the implementer with evidence attached.
• Full-context feeding: every sub-agent gets the complete history, and both cost and latency degrade.
• Parsing text instead of using a protocol: deciding whether a task succeeded with regular expressions is guaranteed to distort over time.

A reasonably safe order of evolution has four steps. First, get a single model stable and reliable. Then introduce model routing by task type, so retrieval tasks, generation tasks, and review tasks each use the best-fitting model. Next, abstract a harness adapter layer so the same session can switch between different harnesses. Finally, introduce multi-environment delegation and role-based multi-agent collaboration. Each step should be validated with quantitative metrics — first-pass rate, average number of fix rounds, cost per task, and human review time.

8. Closing thoughts

When model supply becomes abundant enough, “knowing how to use a particular model” is no longer a competitive edge; “being able to orchestrate multiple models, multiple execution frameworks, and multiple execution environments per task” is. The essence of the problem is systems engineering: three-layer decoupling, separation of the control plane from the execution plane, a task tree with contract-based delegation, and role-based permissions with complete auditing.

For deep-use scenarios that require a lot of parallelism, this orchestration architecture turns what used to be serial manual collaboration into an automated flow that is measurable, replayable, and auditable. For lightweight users, doing environment isolation and approval gates well first and then adding roles gradually is usually more effective than starting with a complex topology.