AI Coding Agent Runtime Architecture: A Deep Dive into Layering and Composability

    This article breaks down the architecture of AI coding agent runtimes, explaining why a "runtime" perspective matters and how these systems are structured into decoupled, replaceable layers. It covers the four common entry points (CLI, web UI, headless mode, and script SDK) and two protocol styles (ACP-style and JSON-RPC-style), then dissects the seven-layer core: model provider, toolset, session, permission policy, sandbox execution, agent loop, and UI. It also walks through four hands-on scenarios—local deployment of all entry points, the engineering value of session logs, cross-runtime trajectory comparison under a fixed model, and a complex full-stack task with visual output—before closing with six architectural principles and a three-step adoption path. Readers will gain a practical understanding of how to select, deploy, evaluate, and govern agent runtimes as engineering assets rather than turnkey products.

    1. Introduction: Why a “Runtime” Perspective Matters

    AI coding tools are rapidly diverging into two forms. The first packages a model into a complete product with a built-in model, toolset, and interaction UI that works out of the box. The second exposes only a set of composable capability interfaces, leaving users to assemble their own model, tools, and UI. The latter is commonly referred to as an AI coding Agent Runtime.

    This split mirrors what happened in computing. Compiler toolchains kept evolving precisely because the frontend, intermediate representation, optimizer, and backend were decoupled from one another. Web development flourished largely thanks to stable interfaces such as HTTP and the DOM. For the “toolchain” in the agent space to achieve the same longevity, it likewise needs clear layering and contracts.

    Over recent years, the industry has begun releasing runtime distributions for coding agents. Model capabilities iterate month after month, yet the tool ecosystem is fragmenting fast. If every team built its own “model invocation + tool execution + permission control” stack from scratch, the duplicated investment would be enormous. The runtime approach is a direct response to this pain point: consolidate common capabilities into a stable foundation, and push differences out to configuration and plugins.

    These runtimes are complementary to turnkey coding products, not replacements. Products lower the barrier to entry; runtimes make room for deep customization.

    • When a runtime is the better choice: You need to connect a private model or a self-hosted inference service, have a specialized toolchain, enforce strict data boundaries, or want to embed coding capabilities into an existing workflow.
    • When a turnkey product is the better choice: Your team’s needs align closely with a standard coding assistant, and you value an out-of-the-box experience.

    It’s worth noting that runtime projects typically ship only a “capability core.” The default configuration is a reference starting point, and product experience and security must be filled in by the user. The configuration snippets and invocation examples below are illustrative; for actual fields, refer to the official documentation of whichever implementation you adopt.

    2. External Shape: Four Entry Points and Two Protocol Styles

    2.1 Layered Overview

    These runtimes usually expose several access methods—for example, the four main entry points below—all sharing a single core.

    1. ┌──────────────────────────────────────────────────────────┐
    2. │ Access layer: CLI · Web UI · headless mode · script SDK · │
    3. │ protocol interfaces │
    4. ├──────────────────────────────────────────────────────────┤
    5. │ Orchestration layer: agent loop (plan-execute-verify- │
    6. │ converge) │
    7. ├──────────────────────────────────────────────────────────┤
    8. │ Capability layer: tool registry · permission policy · │
    9. │ sandbox execution · session storage │
    10. ├──────────────────────────────────────────────────────────┤
    11. │ Adapter layer: model provider adaptation │
    12. ├──────────────────────────────────────────────────────────┤
    13. │ Infrastructure: filesystem · subprocesses · network · │
    14. │ version control (illustrative) │
    15. └──────────────────────────────────────────────────────────┘

    The key design principle: the access layer only handles input capture, output presentation, and interaction control, while all decision logic is pushed down into the orchestration and capability layers. As a result, no matter which entry point a user starts from, the execution semantics of the same task should remain consistent across entry points.

    2.2 Applicability Boundaries of the Four Entry Points

    Access method Interaction style Typical users Typical scenarios
    CLI entry (invoked after installing via a package manager) Interactive in the terminal Backend developers, ops engineers Everyday development, combined with terminal tools
    Web UI entry Interactive in the browser, visualized R&D teams, cross-functional reviewers Change previews, conversational review, result demos
    Headless mode entry Non-interactive, outputs results per invocation Platform engineering, test engineers Pipelines, batch jobs, scheduled tasks
    Script SDK entry Programmatic invocation Application developers, data engineers Embedding into business systems, batch processing, custom orchestration

    Headless mode is especially important. Once agent capability can be invoked as “given an input, produce a change,” it can enter CI pipelines and take on tasks such as code generation, dependency upgrades, and documentation backfill. Through this, the runtime shifts from “a tool for humans” to “part of a system.”

    2.3 Two Protocol Styles: ACP-Style and JSON-RPC-Style

    Beyond these entry points, these runtimes typically also expose two kinds of interfaces, corresponding to two integration paradigms.

    (1) ACP style: the host-IDE integration paradigm.

    The Agent Communication Protocol (ACP), widely used in the industry, emphasizes that the host editor and the agent process communicate through standard messages: the host initiates a session, the agent continuously reports its reasoning and execution progress via streamed messages, and it can request host confirmation before high-risk operations.

    • Typical interfaces: initialize; create/load a session; prompt; cancel; extension method invocation.
    • Message types: Prompt, SessionUpdate (streamed updates), RequestPermission (permission requests), and so on.

    This design hands “who controls the UI and permissions” to the host, while the runtime only handles “thinking” and “doing”—letting agent capabilities be embedded into multiple editors without each one adapting separately.

    (2) JSON-RPC style: the in-process and cross-process invocation paradigm.

    The JSON-RPC convention, widely used in the JavaScript ecosystem, gives the runtime a unified request/response channel. Integrators can manage sessions, invoke tools, and configure models through remote calls without caring about the internal implementation. In Electron-style desktop shells (browser engine + system-level Node capabilities), it can also serve as the communication bridge between the main process and the renderer process.

    1. // Illustrative: a session message interface wrapped over JSON-RPC; method names are examples only
    2. {"jsonrpc":"2.0","id":7,"method":"session/prompt",
    3. "params":{"session_id":"s-001","prompt":"Add unit tests for the current module"}}
    4. // Response (illustrative)
    5. {"jsonrpc":"2.0","id":7,"result":{"status":"accepted"}}

    It’s worth emphasizing that this is an architectural generalization, not a description of any specific implementation.

    3. Core Architecture: Seven Decoupled, Recombinable Layers

    A runtime’s true “composability” lives inside: each layer is implemented around a stable interface and can be swapped without affecting the others. Let’s dissect them layer by layer.

    3.1 Model Provider Layer: Decoupling “Capability” from “Vendor”

    Responsibility: Encapsulate the differences between models behind a unified interface, exposing a consistent “message list in, reply out” semantic to upper layers. Coverage typically includes:

    • Cloud APIs: remote model services adapted through a unified protocol;
    • Local inference: locally deployed models exposed through standardized service interfaces;
    • Custom providers: private models connected via plugins or configuration.

    Key points:

    1. A unified message format, e.g., a message list with roles divided into user / assistant / tool;
    2. A unified tool-call protocol that normalizes a model’s structured call requests into tool name + arguments;
    3. Unified streaming events, sending incremental text and incremental tool calls separately;
    4. Unified error semantics, distinguishing retryable errors (rate limits, timeouts) from non-retryable ones (invalid arguments, auth failures).

    Value: The same toolset and agent loop can run directly on top of different models; upgrading or replacing a model requires no changes to upper-layer logic.

    3.2 Toolset Layer: From “Built-in Commands” to “Registerable Capabilities”

    Tools are the agent’s only channel for interacting with the real world. The toolset layer usually includes three kinds of built-in tools:

    • File tools: read, write, edit, and list directories;
    • Execution tools: run commands and run tests inside a sandbox;
    • Retrieval tools: keyword search, semantic retrieval, or web content fetching.

    On this foundation, it supports extension:

    • Script-defined tools: declare the tool name, description, parameter schema (JSON Schema), and handler logic via configuration;
    • In-process registration: register custom tools programmatically through an SDK;
    • External tool services: mount external capabilities through tool-integration protocols such as MCP (Model Context Protocol).

    Two design lessons are worth borrowing. First, descriptions come first: whether the model picks the right tool depends largely on how clear the tool description and parameter explanations are—vague naming leads directly to misinvocation. Second, converge results: file contents and command output should support truncation, pagination, and summarization to avoid overly long results drowning the context.

    3.3 Session Layer: Making Every Run Addressable

    The session layer handles persistence of message history, tool-call records, and execution state, providing create, restore, replay, and share capabilities. Good session design generally has these traits:

    • Event-based storage: append writes in a format like JSON Lines, naturally supporting incremental reading and parsing;
    • Stable identifiers: session ID, message ID, and tool-call ID remain traceable throughout;
    • Snapshotting: periodically save context snapshots so restoration doesn’t require replaying every event one by one;
    • Portability: export to a standardized message format for easier cross-tool analysis.

    3.4 Permission Policy Layer: Separating “Can It Do This” from “Should It”

    There’s an inherent tension between an agent’s autonomy and its safety. The permission policy layer separates “what the model decides to call” from “what the system allows to be called.” Common patterns include:

    Mode Behavior Typical scenario
    Auto-allow Whitelisted tools or commands need no confirmation Read-only queries, formatting, running unit tests
    Per-call confirmation Request user or host approval before each call Writing files, running builds, accessing the network
    Policy engine Match and decide by rules on paths, commands, domains, etc. Team-wide governance, enterprise compliance
    Remembered authorization Remember the authorization scope for similar operations after user confirmation High-frequency repeated operations

    From a security standpoint, the permission layer must default to least privilege and deny unknown operations by default. In particular, guard against “remembered authorization” degenerating into “permanently allow all write operations.”

    3.5 Sandbox Execution Layer: Providing an Isolation Boundary for Code Execution

    Agents need to actually run code, so a sandbox is a mandatory capability. Common isolation strength, from weakest to strongest, is roughly:

    1. Restricted subprocess: limit working directory, timeout, and resource usage;
    2. Container isolation: provide filesystem and process isolation via a container platform or local container runtime;
    3. VM or micro-VM: strong isolation for untrusted code;
    4. Cloud sandbox: host the entire execution environment in the cloud, spun up on demand and destroyed after use.

    When choosing a sandbox, weigh isolation strength against startup overhead, state retention against reproducibility, and network access policy against the convenience of installing dependencies.

    3.6 Agent Loop: The Task’s “Control Plane”

    The agent loop orchestrates the back-and-forth “think-act-observe” cycle. It is the control plane, and its concrete implementation includes:

    • context construction and compaction;
    • tool-call dispatch;
    • error retry and fallback;
    • termination condition checks.

    A typical shape (pseudocode):

    1. def run_loop(session, max_steps=50):
    2. for step in range(max_steps):
    3. # 1. Build context: session history + system prompt + tool list
    4. ctx = build_context(session)
    5. # 2. Call the model, with streaming responses
    6. reply = model.stream_chat(ctx, tools=registry.schemas())
    7. # 3. No tool call => consider the task converged
    8. if not reply.tool_calls:
    9. return finalize(session, reply)
    10. # 4. Dispatch tool calls: permission -> sandbox -> record result
    11. for call in reply.tool_calls:
    12. if not policy.allows(call):
    13. result = denied(call, policy.reason)
    14. else:
    15. try:
    16. result = sandbox.invoke(call, timeout=120)
    17. except ToolError as e:
    18. result = error_payload(e) # Feed the failure back to the model instead of aborting
    19. session.append_tool_result(call.id, result)
    20. # 5. When context exceeds the threshold, compact while keeping key decisions and file summaries
    21. if estimate_tokens(session) > budget:
    22. session.compact()
    23. return finalize(session, reason="max_steps")

    There are four design lessons:

    1. Make convergence conditions explicit: don’t rely only on “the model stops calling tools”—also set a step limit, a timeout, and repeated-call detection to prevent the loop from spinning idly;
    2. Feed failure information back: a tool error should be returned to the model as an observation so it has a chance to self-correct, rather than throwing an exception that terminates the run;
    3. Give context a budget: in long tasks, file contents and command output expand quickly, and the compaction strategy directly affects success rate and cost;
    4. Decouple the loop from the model: the loop depends only on the unified message and tool interfaces, independent of any specific model.

    3.7 UI Layer: A Replaceable “Presentation and Interaction”

    The UI layer is responsible for displaying the execution trace, diffs, tool-call status, and permission requests, and contains no business logic itself. Common forms include a web UI, a terminal UI, and host-IDE integration. The design principle: the UI only consumes the event stream exposed by the runtime and is unaware of model and tool details.

    4. Practice 1: Validating All Four Entry Points with a Local Deployment

    Take a local deployment as an example. The validation approach is: confirm that each entry point works under the same configuration system and produces consistent session records.

    4.1 Installation and Configuration

    First, globally install the CLI via the package manager of the corresponding ecosystem (refer to the official release for the package name), or add it as a project dependency.

    Second, prepare model credentials. There are usually three ways: write them into a configuration file, inject them via environment variables, or enter them through an interactive setup flow. Whatever the method, it’s recommended to inject sensitive information through a secrets management capability and avoid committing it to version control.

    Third, prepare a minimal working configuration, which typically includes the following parts (illustrative):

    1. # Illustrative config: field names and hierarchy follow the actual implementation's docs
    2. model:
    3. provider: openai-compatible # Provider type connected via a standard protocol
    4. model: your-model-id
    5. base_url: https://your-endpoint.example/v1
    6. api_key_env: RUNTIME_API_KEY # Reference by env var name to avoid storing plaintext on disk
    7. agent:
    8. system_prompt: ./prompts/coding.md
    9. max_steps: 50
    10. context_budget: 120000
    11. tools:
    12. enable: [file_read, file_write, file_edit, shell, search]
    13. shell:
    14. timeout: 120
    15. network: false
    16. sandbox:
    17. mode: local-subprocess # can also be container / cloud
    18. workdir: ./workspace
    19. permissions:
    20. default: ask # Write operations and command execution require confirmation by default
    21. allow_read: true
    22. session:
    23. log_dir: .runtime/sessions
    24. format: jsonl

    Fourth, validate in sequence:

    1. CLI entry: launch interactively and confirm you can chat, invoke tools, and see permission requests;
    2. Web entry: start the local service and access it via a browser, confirming changes are shown as diffs;
    3. Headless mode: run a non-interactive task (e.g., “generate the project README”) and confirm that artifacts land on disk after the command exits and logs are complete;
    4. Script SDK: write a minimal invocation script to create a session, send a prompt, and collect messages programmatically.

    4.2 Dissecting the Default Configuration

    Before formal use, understanding the default configuration matters more than rushing to tune parameters. It typically includes:

    • Model defaults: default provider, default model, sampling parameters such as temperature;
    • Context budget: max tokens per session and the compaction threshold;
    • Tool whitelist: the toolset available by default;
    • Permission defaults: which operations are confirmation-free and which require confirmation;
    • Sandbox defaults: default execution mode, timeout, working directory;
    • Loop limits: max steps, max tool calls, retry count.

    It’s advisable to establish a “configuration baseline” within the team: start from the defaults, derive configuration profiles by environment (dev / staging / prod) and project type (frontend / backend / data), put them under version control, and standardize conventions for model IDs, endpoint addresses, and budget parameters.

    5. Practice 2: The Engineering Value of Session Logs

    Session logs are the most underrated capability of a runtime. They serve as debugging material, as an audit trail, and as a data source for effectiveness evaluation. A typical session log is in JSON Lines format, with one event per line.

    5.1 Event Stream Example

    1. {"seq":1,"ts":"...","type":"session.start","session_id":"s-001"}
    2. {"seq":2,"ts":"...","type":"user.message","content":"Add unit tests for the order module"}
    3. {"seq":3,"ts":"...","type":"model.request","model":"your-model-id","input_tokens":3120}
    4. {"seq":4,"ts":"...","type":"model.response","output_tokens":180,"tool_calls":2}
    5. {"seq":5,"ts":"...","type":"tool.call","tool":"file_read","args":{"path":"src/order/service.ts"}}
    6. {"seq":6,"ts":"...","type":"tool.result","status":"ok","truncated":false}
    7. {"seq":7,"ts":"...","type":"tool.call","tool":"shell","args":{"cmd":"..."},"approval":"granted"}
    8. {"seq":8,"ts":"...","type":"model.response","output_tokens":420,"tool_calls":0}
    9. {"seq":9,"ts":"...","type":"session.end","status":"completed"}

    5.2 Typical Uses

    • Restore and replay: after a process crash or deliberate interruption, rebuild context from the log and continue execution;
    • Audit and compliance: fully record model requests, tool calls, and approval results to meet traceability requirements;
    • Effectiveness analysis: tally tool-call distribution, failure rates, token consumption, and step counts to identify inefficient spots;
    • Problem localization: when abnormal output appears, trace events one by one to determine whether the prompt was unclear, a tool failed, or permission was denied;
    • Evaluation baseline: turn real sessions into regression cases to compare before and after model or prompt changes.

    5.3 Engineering Considerations

    Logs will contain code snippets, paths, and command output, and may carry sensitive information. Three kinds of handling are recommended: redact secrets, tokens, and personal data; set retention periods by data classification; and restrict log access while integrating with a unified logging and audit system.

    6. Practice 3: Cross-Runtime Trajectory Comparison Under the Same Model

    Many teams hit this problem: they switch to a stronger model, yet task success rate doesn’t improve noticeably. The cause is often not the model but the runtime. A controlled experiment can verify this.

    6.1 Experiment Design

    • Controlled variables: the same model, the same prompt, the same task, the same toolset, the same sampling parameters, the same timeout and budget;
    • Independent variable: only swap the runtime (or its loop strategy or tool implementation);
    • Observed metrics: whether the task completes, step count, tool-call count, tool failure rate, token consumption, and wall-clock time.

    Example script (illustrative; invocation follows the actual SDK):

    1. from runtime_sdk import Runtime, SessionStore
    2. runtime = Runtime(model="your-model-id", tools=TOOL_SET,
    3. max_steps=50, context_budget=120000)
    4. store = SessionStore("./sessions")
    5. task = "Add unit tests for the order service covering null values, boundary cases, and exception branches, and ensure they all pass"
    6. session = store.create()
    7. for msg in runtime.run(session, task): # Stream the execution trajectory
    8. if msg.kind == "tool_result":
    9. record(msg) # Record tool calls and results
    10. elif msg.kind == "final":
    11. report(msg)

    6.2 The Value of a Comparison View

    Placing an “integrated coding assistant” and an “open runtime” side by side under the same model and task usually reveals a clear division of labor:

    • Integrated assistants tend toward few tools and short chains, relying on the user to supply context;
    • Open runtimes tend toward many tools and long chains, approaching the answer through autonomous retrieval and execution, at the cost of higher step counts and token consumption.

    So the choice of runtime is essentially a trade-off between “autonomy and controllability.” Making this comparison part of a team’s evaluation process is far more telling than a single ad-hoc experience.

    7. Practice 4: Complex Full-Stack Tasks with Visual Artifacts

    Finally, use an end-to-end task to test a runtime’s overall capability: ask the agent to complete a full-stack delivery that includes a backend API, a frontend page, chart visualization, and a UI snapshot, and to verify the result on its own.

    7.1 Task Setup

    Goal: Build a lightweight data analytics dashboard containing a data API, page rendering, two kinds of charts, and a responsive layout, and output a runtime screenshot as acceptance evidence.

    The difficulty lies in the fact that the task spans multiple tech stacks and its artifacts include “visual results,” which are hard to judge as correct or incorrect by text alone.

    7.2 Execution Flow

    1. Planning stage: the agent first outputs an implementation plan (module breakdown, file list, dependency list) for the user to confirm;
    2. Implementation stage: sequentially generate the data layer (a local data file or a lightweight service), the API layer (read-only query endpoints), and the presentation layer (pages and chart components);
    3. Verification stage: install dependencies in the sandbox, start the service, access the page with a headless browser and take a screenshot, and run automated tests;
    4. Convergence stage: if the screenshot shows a broken layout or missing charts, the model reads the error information, self-corrects, and verifies again;
    5. Delivery stage: output a change summary, diffs of key files, and the acceptance screenshot.

    7.3 Lessons Learned

    • Make visual artifacts “observable”: only by turning rendered results into screenshots, DOM structure, or an accessibility tree can they enter the agent’s feedback loop;
    • Plan before implementing: in complex tasks, having the agent produce a structured plan first significantly reduces rework;
    • Automate verification: screenshot comparison, API smoke tests, and unit tests should all be exposed to the agent as tools, not triggered manually by humans;
    • Security boundaries cannot be skipped: full-stack tasks often involve installing dependencies and accessing the network, so they should be done within a sandbox with restricted outbound access.

    8. Architectural Design Points and Adoption Advice

    Overall, the design philosophy of these runtimes can be summarized in six points:

    1. Stable interfaces, swappable implementations: model, tools, session, permissions, sandbox, loop, and UI each face an interface, so any layer can be replaced independently;
    2. Capabilities sink, entry points float: decision logic is concentrated in the orchestration layer, while entry points only handle interaction;
    3. Protocols first: integrate with hosts and external tools through standardized protocols to lower integration costs;
    4. Observability built in: session logs, streaming events, and diff presentation are infrastructure, not add-on features;
    5. Conservative security by default: least privilege by default, confirmation required by default, isolated execution by default;
    6. Evaluation-driven iteration: use real sessions as an evaluation set, turning model upgrades and prompt optimization into a measurable process.

    For adoption, a three-step path is recommended:

    • First, get a minimal loop running: single model + basic tools + CLI entry;
    • Then, add engineering capabilities: session persistence, permission policies, sandboxing, and log auditing;
    • Finally, package for specific scenarios: distill templates, toolsets, and configuration baselines for concrete business use.

    9. Conclusion

    The “runtime rather than product” positioning determines these architectures’ value orientation: their goal is not out-of-the-box convenience, but composability, observability, and governability as design principles. For developers, understanding layer boundaries is more meaningful than memorizing a particular command; for teams, establishing configuration baselines, logging conventions, and evaluation processes is the key to turning agent capability into a genuine engineering asset. Models will keep iterating, and it’s the stable layering and protocols that endure across technology cycles.