Multi-Session Agent Scheduling Architecture: A Dispatch-Only Main Session and Non-Blocking Interruptions

1. The Bottleneck in Multi-Task Agents Is Scheduling, Not Compute

When an agent moves from a question-and-answer mode to long-running autonomous execution, a counterintuitive issue appears: the number of tasks that can be kept active at the same time is usually determined less by the model’s reasoning capability and more by two external resources: upstream model concurrency or quota and human review bandwidth.

A judgment often repeated in public discussions is that a person can usually follow only 3 to 5 active sessions at once. In this interaction model, the ceiling for a single task depends on model capability, but the ceiling for multiple tasks depends on human attention.

In engineering terms, this is very concrete. The AI produces output too quickly: it can deliver a plan within minutes and then wait for review. After review, it may ask a few clarifying questions, pulling the person back and forth in the session. When someone tries to run three or four sessions in parallel, attention becomes fragmented, and the benefit of a single session is offset.

A second pain point comes from mid-session interruptions. Once an agent is connected to instant-messaging tools, a typical scenario is that the user sends a message, realizes it was unclear, and immediately sends a follow-up, or casually asks an unrelated status question. At that moment, the agent is already executing the previous task. The result is usually one of two bad outcomes:

• Silent ignoring: The interruption is treated as follow-up input and queued. The user assumes it was not heard and may send it again.
• Preemptive interruption: The agent stops the current execution to answer the interruption. The prior reasoning and tool-call results already produced for the current task are discarded. Restarting often costs more than simply waiting for the agent to finish.

The third issue is semantic drift. Even when the system prompt clearly says, “You only reply; delegate the work to subagents,” many single-session agent implementations still do the work themselves. The reason is that prompt constraints and tool permissions compete inside the same context. As long as the main session holds tools for writing files or running commands, it has the ability to execute directly, and the model’s default tendency is to take the shortest path to task completion.

This is therefore not merely a “bad prompt” problem. It is a missing architectural layer.

2. Why Prompts Cannot Enforce “Dispatch Only”

This problem can be reduced to a measurable experiment design. Assume a high-noise message stream: each round contains about 300 group-chat messages and pipeline notifications, but only about 20 are actual tasks that need execution. Each task is designed to run for 5 rounds. The experiment observes whether tasks are dispatched, the completion time of the slowest 5% of tasks, meaning the p95 latency, and the number of cache-read tokens per round.

A publicly shared comparison looks roughly like this:

Design	p50 / p95 completion time, minutes	Cache-read tokens per round
Hierarchical scheduling, coordinator only dispatches	12.4 / 15.7	23.2M
Single session plus subagent tools, neutral prompt	20.4 / about 29	higher
Single session plus subagent tools, prompt forcing dispatch-first behavior	clearly lower, but tail latency remains	higher

Two conclusions are worth noting. These numbers come from a specific simulated workload and should not be treated as a universal benchmark, but the trend is reproducible.

First, the effect of prompts is discontinuous and has a ceiling. With a neutral instruction, the model dispatched no tasks in all 5 rounds and completed everything inside the main session while being repeatedly interrupted. After changing to an explicit dispatch-first instruction, the model dispatched tasks in 4 out of 5 rounds. In the remaining round, it judged the task to be “too small” and did it itself, which happened to create the longest tail. In other words, prompts can move the dispatch probability from 0 to about 0.8, but not to 1. Scheduling correctness requires 1.

Second, the gap appears mainly at p95, not at p50. The median completion time differs by about 8 minutes, while the tail differs by nearly double. This shows that the failure mode of a single-session architecture is not simply “slower overall.” It is occasional loss of control. Once a task is misjudged as something to “just do quickly,” it occupies the main session’s context and attention. Other tasks arriving during that period must wait. In real work, tail latency is what users actually feel.

The root causes can be summarized as three structural issues:

1. The decision maker and the executor share the same context. Deciding whether to dispatch and deciding how to execute share the same context window. Growth on either side squeezes the available space of the other.
2. Tasks lack an independent lifecycle. Tasks have no external state and exist only in the conversation history. When the history is compressed or flooded by new messages, tasks are effectively lost.
3. Interruption equals preemption. There is only one input channel. User messages and system events enter through the same gateway, so later messages can only preempt or queue. There is no concept of classifying input first and then routing it to different channels.

3. Target Architecture: Coordinator-Worker Hierarchy

Mapping the three root causes to design constraints gives a stable architecture:

• Separate decision-making from execution: the coordinator session does not hold execution tools. It only holds tools for dispatching, querying, and summarizing.
• Externalize task state: task state is stored in shared storage. The session context only keeps task IDs and summaries.
• Classify input before routing: the message gateway classifies input first, then decides whether it goes to an interruption channel, a new-task queue, or a worker context.

3.1 Roles and Responsibility Boundaries

• Coordinator: Does only four things: understand intent, judge task boundaries, dispatch tasks, and aggregate delivery. Its context contains only task metadata and progress summaries, so its prefix is extremely stable.
• Worker: Each task gets an independent instance with the full toolset and an independent context. After completing the task, it writes back artifacts and conclusions according to the agreed format, then is destroyed.
• Gateway: Handles all external input, including intent classification, deduplication, authentication, and rate limiting, then delivers the result to the correct queue.
• State store: The authoritative source for tasks, locks, and artifact indexes. It usually needs transactional or conditional-write capability.

3.2 Task Metadata Schema

Dispatching requires tasks to be describable and traceable. A minimal usable schema looks like this:

{
  "task_id": "t-20260826-0007",
  "title": "Add idempotency-key validation to the order service",
  "objective": "Validate request_id at the order-submission entry and return the first result for duplicate requests",
  "constraints": [
    "Do not change the public interface signature",
    "Add unit tests covering concurrent duplicate submissions"
  ],
  "acceptance": [
    "Unit tests pass",
    "Regression suite has no new failures"
  ],
  "inputs": ["repo://svc-order@main", "design://docs/idempotency-v2"],
  "parent_task": "t-20260826-0001",
  "priority": 7,
  "isolation": "worktree",
  "timeout_sec": 3600,
  "max_retries": 1
}

Two fields are especially important. acceptance allows the coordinator to determine whether a task is complete without reading the code. inputs explicitly declares dependencies and is the basis for deciding whether tasks can run in parallel. Tasks that write to the same input resource cannot run concurrently.

3.3 Task State Machine

Once task state is externalized, every step in the main session becomes an operation on the state machine rather than an operation on context:

RECEIVED --> CLASSIFIED --> QUEUED --> DISPATCHED --> RUNNING
                                      |             |
                                      |             +--> BLOCKED (needs human confirmation)
                                      |             +--> FAILED --> RETRY / DEAD
                                      +-------------------------> CANCELED
                                                |
                                                +--> DELIVERED (aggregated by coordinator) --> CLOSED

The BLOCKED state is especially important. It represents cases that require human decision-making. It turns human-in-the-loop involvement from an “interruption” into “handling an explicit state.”

4. Non-Blocking Interruptions: From Preemption to Classified Routing

4.1 Three Types of Input, Three Channels

User interruptions are not one single thing. At minimum, they should be divided into three categories:

Type	Example	Handling
Supplement or correction to the current task	“Also add logging while you’re at it”	Merge into a mutable field of the current task. If execution has already started, create a patch task under the same parent task.
New task	“Also check the cache hit rate”	Enqueue it. The coordinator decides whether it can run in parallel and dispatches it.
Query or chitchat	“Where is it now?”	Side-channel response. It reads only the state store and does not touch the execution context.

4.2 Routing Policy

input_routing:
  ack_policy: immediate        # acknowledge first, then process, to avoid duplicate user sends
  classifier:
    model: small-fast
    labels: [amend, new_task, query, chitchat]
    fallback: new_task         # conservatively enqueue if classification fails; do not guess
  channels:
    amend: current_task_patch_queue
    new_task: coordinator_queue
    query: status_reader       # read-only; does not block any worker
    chitchat: coordinator_queue
  concurrency:
    max_active_workers: 4
    per_repo_limit: 2          # serialize writes to the same repo to avoid overwriting each other

per_repo_limit is one of the most easily overlooked constraints in practice. Multiple workers editing the same repository at the same time creates a much higher conflict probability than model capability limitations. A common approach is to give each worker an independent code copy or worktree, and let the coordinator handle merging during the consolidation phase.

4.3 Supervisor Loop, Illustrative Example

def supervisor_loop(store, router, dispatcher):
    while not store.drained():
        for task in store.claim_ready_tasks():          # conditional write ensures no duplicate claiming
            if not dispatcher.has_capacity(task):
                continue
            worker = dispatcher.spawn(task)              # independent context
            router.subscribe(task.id, on_event=store.record)
        msg = router.poll(timeout=0.5)                   # non-blocking poll for user input
        if msg is None:
            continue
        msg.reply_ack()                                  # acknowledge immediately: received, execution not interrupted
        match msg.label:
            case "amend":
                store.patch_current_task(msg.task_id, msg.payload)
            case "new_task":
                store.enqueue(build_task(msg.payload))
            case "query":
                msg.reply(render_status(store.snapshot()))
            case "chitchat":
                msg.reply(coordinator.answer(msg.text))

The important part is not the scheduling algorithm itself, but three “does not” behaviors: poll does not block workers, reply_ack does not wait for processing results, and patch_current_task does not directly modify a running context. It writes changes into task state, and the worker reads them at the next checkpoint. Workers pull changes themselves rather than being interrupted by external pushes. This is the key to stability for long-running tasks.

5. Why Layering Actually Saves Context

The counterintuitive part is this: adding more workers usually increases total token consumption, but the per-round cache-read volume decreases, and end-to-end completion time also decreases. The reason is that cache hits depend on prefix stability:

• The coordinator’s context only grows with task summaries. The format is fixed and changes at few points, so prefix-cache hit rates are high.
• Worker contexts are short and short-lived. They are released immediately after a single task ends and do not grow indefinitely.
• The main session no longer carries process output such as reading files, running commands, or inspecting errors. That content is often the noisiest part of the context.

A further engineering optimization is possible: workers return only structured conclusions plus artifact pointers to the coordinator, not process logs. Logs are written to a separate logging service and can be retrieved by task_id when needed. This constraint significantly reduces the growth rate of the coordinator’s context.

6. Reproducing This Structure in Existing Coding Agents

Most command-line coding tools that support custom subagents provide three types of configurable mechanisms: agent definition files, tool allowlists, and context injection rules. Using these, it is possible to build the same layered structure without modifying the source code.

Coordinator definition: Grant only task-management tools, and explicitly deny read/write and execution tools.

{
  "name": "coordinator",
  "description": "Responsible only for task judgment, dispatch, and aggregation. Does not implement code changes directly.",
  "tools": ["TaskCreate", "TaskUpdate", "TaskList", "TaskQuery", "Read"],
  "denied_tools": ["Write", "Edit", "Bash", "Delete"],
  "prompt_file": "./prompts/coordinator.md",
  "model_tier": "strong"
}

Worker definition: Grant the full toolset, but enforce constraints on work scope and delivery format.

{
  "name": "worker",
  "description": "Receives a single task, completes implementation, verification, and delivery in an isolated working copy.",
  "tools": ["Read", "Write", "Edit", "Grep", "Bash", "Test"],
  "workdir_isolation": "git_worktree",
  "cwd_policy": "per_task",
  "max_turns": 60,
  "report_schema": "task_report.v1",
  "model_tier": "strong"
}

How to write the coordinator prompt: The important point is not wording, but turning judgment criteria into executable rules:

# Responsibility
You are the dispatcher. You do not implement code, modify files, or run commands.
# Dispatch Criteria
You must dispatch the task to a worker and must not handle it yourself if any of the following conditions is met:
1. The task requires reading or writing files, or executing commands;
2. The expected number of steps is greater than 2;
3. The task can be split into independent subtasks.
# Aggregation Rules
After a worker reports, you only check whether the acceptance fields are satisfied.
If they are not satisfied, write back a patch task with specific differences.
Do not fix it yourself.

The last point is especially important. The coordinator’s biggest overreach risk is not “doing the work itself,” but “fixing the worker’s output itself.” Blocking this behavior is what makes the benefits of the layered design stable.

This structure also applies to self-built agent platforms. The coordinator can be a long-running service. Workers can be launched on demand as containers. Tasks and interruptions can be carried by message queues. Artifacts can be stored in object storage. Process logs can go to a logging service. Error rates and queue backlog can be covered by monitoring and alerting. On a cloud platform with managed container, message queue, object storage, logging, and monitoring services, this combination of general capabilities can support a controllable multi-session scheduling layer without requiring the team to build infrastructure from scratch.

7. Boundaries and Known Costs

This architecture does not solve every problem. Defining boundaries clearly is more important than exaggerating benefits:

• Tasks with strong sequential dependencies should not be parallelized. Changes with prerequisite relationships should be explicitly chained. The parent_task field exists for this purpose.
• The scheduler becomes a new single point of failure. If the coordinator fails, the whole system may stall. Task state must be persistent and recoverable. It cannot exist only in memory.
• Fixed overhead is real. For small tasks, the cost of dispatching, launching, and reporting may exceed direct execution. A task-size threshold should be defined. Tasks below the threshold should bypass the scheduler and be handled directly.
• Concurrency quota is a hard constraint. The parallelism ceiling is determined first by upstream model quota, not by code. Over-provisioning will degrade into queuing, timeouts, and retries.
• Observability must be designed upfront. At minimum, the system needs task-state distribution, queue wait time, p95 completion time, retry rate, and cache hit rate. Without these metrics, the benefits of the layered design cannot be verified, and tail latency is difficult to diagnose.

8. Conclusion

“Making an agent dispatch only” is not a prompt. It is an architectural constraint: put decision rights and execution rights into different contexts, move task state out of the conversation history, and change user input from a single channel into classifiable multi-channel input. If any of these three is missing, the main session will degrade back into an executor, and interruptions will degrade back into preemption.

If your current multi-session agent meets any of the following conditions, it is worth restructuring according to the design above: the main session holds execution tools; task state exists only in conversation history; user input enters the main context without classification; or multiple workers may write to the same repository at the same time. The benefit of the redesign will not appear at the median. It will appear in the longest 5% of tasks, which is also where users tend to give up.