AI Coding Agent Runtime Architecture: A Deep Dive into Kernel Design
This article deconstructs the runtime architecture behind AI coding agent frameworks and explains why developers who expect an out-of-the-box product instead receive a modular kernel. It breaks the design down across multiple entry points, seven decoupled subsystems, layered default configuration, and the Session Log, then offers a method for fairly comparing agent runtimes under the same model, a complex full-stack task example that produces visual artifacts, and an eight-question checklist for judging whether a runtime is fit to serve as team infrastructure. Readers will come away with a clear framework for evaluating whether a tool is a product or an extensible kernel.
1. Introduction: Expecting a “Product,” Getting a “Runtime”
Every time a new generation of strong reasoning models ships, the open-source agent framework that accompanies it becomes a community focal point. Many teams expect a coding assistant they can install and use out of the box. Yet after pulling the source, they often find the UI is only a shell — the real value lives in the kernel. Model integration, tools, sessions, permissions, the sandbox, and the main loop are cleanly separated; they can be swapped, combined, and embedded into your own systems.
This expectation gap is really a gap in evaluation criteria. Judging a kernel by “does it work out of the box” underestimates its extensibility; judging a product by “is the kernel flexible” ignores delivery and operational costs.
Answer three questions first, then decide whether to dig deeper:
- Can it connect to any model, without binding to a single service?
- Can tools be declared per project and authorized by risk level?
- Can sessions and execution traces be fully preserved, replayed, and evaluated?
The answers determine whether it is a “kernel” or a “product.” This article abstracts a general runtime architecture and deconstructs, in turn, the multi-entry design, subsystem decoupling, default configuration, and the Session Log. It then presents a method for comparing execution traces under the same model, plus a complex full-stack task example that produces visual artifacts.
2. Multiple Entry Points: One Kernel, Four Forms
This kind of runtime typically offers four categories of entry point:
| Entry point | Interaction form | Typical scenario | Design focus |
|---|---|---|---|
| Package-manager CLI | Local install, interactive in the terminal | Daily coding, troubleshooting | Startup speed, incremental updates, auto-discovery of project config |
| Browser web | Local server, accessed via browser | Multi-person viewing, visual review | Long-connection event streaming, concurrent sessions, access boundaries |
| Headless CLI | Single task, structured output | CI, batch scripts, scheduled jobs | Non-interactive permission policy, exit codes, parseable output |
| Language SDK | Embedded as a library in a host program | Integrating in-house platforms, secondary development | Interface stability, async model, injectable permission and sandbox |
These four entry points are not four separate implementations, but different “shells” over the same kernel:
- The interactive CLI and the web entry share the same session engine; the only difference is whether the interface is a terminal or a browser.
- The headless entry collapses interaction into “one input, one result,” which suits automation better.
- The SDK hands permission, sandbox, and model to the host as dependency-injection items.
The entry layer is only responsible for creating a session, writing user input into the event stream, and rendering the incremental events the kernel emits. As a result, the same session can be “started in the CLI and continued in the browser,” or “switched between interactive and automated modes,” changing only the authorization method rather than the task semantics.
Design point: Abstract “session creation, restoration, and event subscription” into a stable contract, and route every entry point through it to reach the kernel. On that basis, unify the following exits:
- An RPC-style interface for SDK and external-service integration;
- A standardized JSON message format for editor and external-client integration;
- Structured output for scripts to consume.
Otherwise, every new entry point forces you to copy the session-management logic, and behavioral consistency quickly spins out of control.
3. Kernel Decoupling: Responsibilities and Boundaries of Seven Subsystems
The runtime kernel can be split into seven subsystems, judged by one core criterion: “can it be replaced independently without changing the rest?”
- Model provider layer: normalizes models from different vendors, protocols, and deployment forms (cloud API or private inference service) into a unified invocation interface. It handles protocol translation of messages and tool definitions, streaming output, timeouts, retries, and rate limiting. Swapping a model only requires adding an adapter.
- Tool set: each tool declares its name, purpose description, parameter structure (such as JSON Schema), and permission level. Typical tools include reading and writing files, executing commands, searching code, and making network requests. Extension comes in two forms: custom script tools and standardized tool integration protocols.
- Session management: responsible for context assembly, window management, and compaction. As history approaches the model window, it typically compacts by “keeping the system prompt and recent messages while summarizing earlier activity.” It also provides session listing, restoration, branching, and archiving.
- Permission policy: places a gate between “what the model wants to do” and “what the system permits.” It grades actions as allow / ask / deny, combined with path allowlists, command rules, and second confirmation for dangerous operations.
- Sandbox execution: provides an isolated environment for tools to run in. It isolates the file system and network via containers or restricted processes, limits resource quotas, and constrains the working directory.
- Agent loop: orchestrates the iteration of “model inference → tool call → result feedback.” It handles parallel scheduling, step and budget limits, and infinite-loop detection.
- UI layer: consumes the event stream and renders it. It handles collapsing tool calls, showing code diffs, reference locating, and progress output in headless mode.
The seven subsystems exchange three kinds of data — session context, tool-call requests/results, and the event stream — and do not share internal state. Permission decisions should be moved forward into the tool-call dispatch step of the agent loop: no matter which entry point a call comes from, it passes through the same policy engine, avoiding the loophole where “the UI allows it but a script bypasses it.”
The agent loop can be abstracted as:
loop (until a termination condition is met):resp = model.call(message_history, tool_definitions)if resp contains no tool calls:return resp text as the final answerelse:for each tool_call (can be parallel):decision = policy.decide(tool_call) # allow / ask / denyif decision is ask and in auto mode: handle per pre-authorization policyresult = sandbox.execute(tool_call) # execute inside the isolated environmentappend (tool_call, result) to message_history
Termination conditions should include at least: the model produces a final answer, the maximum step count or budget limit is reached, multiple consecutive steps make no progress, or an unrecoverable permission denial occurs.
The real engineering difficulty lies in the friction at decoupling boundaries:
- Context compaction changes tool visibility — for example, some tools are only available in “plan mode,” and after switching modes their call records may still linger in the history.
- Parallel tool calls cause events to arrive out of order, so the UI must reassemble them by call identifier.
- When a long task resumes across processes, sandbox state and session state must be rebuilt in sync.
When designing, be explicit about where each subsystem’s state lives: what goes into the session log, what stays in the sandbox, and what the host holds.
4. Default Configuration: Usability Is Defined by Defaults
“Runs with zero configuration” means the defaults determine the first experience for the vast majority of users. A typical layering follows, where each level overrides the one above:
| Layer | Carrier | Scope | Use case |
|---|---|---|---|
| Built-in defaults | Compiled into the program | Global fallback | Guarantees out-of-box usability |
| User-level | Config file in the user’s home directory | All projects of the current user | Personal model preferences, common permissions |
| Project-level | Config file in the repo (committed with code) | All members of the current project | Unified tech stack and authorization boundaries |
| Call-level | CLI arguments or SDK constructor parameters | This session | Automated tasks, temporary overrides |
The following config is a generic illustration of how fields are organized, not a representation of any specific implementation:
model:provider: openai_compatible # any inference service compatible with the OpenAI-style APIcontext_window: 128000temperature: 0.2agent:max_steps: 50 # max steps for the main loopparallel_tool_calls: truecompaction: summarize # summarize early history near the window limittools:shell: { timeout_seconds: 60, network: false }write: { allowed_paths: ["src/", "tests/"] }web: { allowed_domains: [] }browser: { enabled: false } # visual-artifact tool, enabled on demand by defaultpolicy:default: ask # allow | ask | denydeny_commands: ["rm -rf /", "git push --force"]sandbox:mode: container # run tool calls in container isolationread_only_paths: ["/usr", "/etc"]session:persist: trueretention_days: 7
There are two lessons for default-configuration design:
- Be permissive on “read,” strict on “write,” “execute,” and “outbound.” Read-only operations such as code search and file reading can be allowed directly to reduce interaction noise; file writes are limited to the workspace; command execution and network requests default to ask or deny.
- Make the failure budget explicit. Step counts and timeouts decide “when to cut losses.” If the sandbox is so strict that common tasks frequently fail, users tend to simply turn the sandbox off, and security actually drops. So “success rate” is itself a security-configuration metric.
5. Session Log: The Basis for Observability and Reproducibility
The Session Log is an append-only event log of a session, recording “what happened” at each step with an increasing sequence number. Typical events look like this (illustrative structure):
[{"seq": 1, "type": "session_meta", "model": "chat-model-x", "cwd": "/repo", "policy": "ask"},{"seq": 2, "type": "user_message", "text": "add filter linkage to the dashboard page"},{"seq": 3, "type": "assistant_message", "text": "first locate the relevant components and interfaces..."},{"seq": 4, "type": "tool_call", "id": "c1", "name": "search", "args": {"query": "Dashboard"}},{"seq": 5, "type": "tool_result", "id": "c1", "truncated": true, "bytes": 18432},{"seq": 6, "type": "tool_call", "id": "c2", "name": "write", "args": {"path": "src/pages/Dashboard.tsx"}},{"seq": 7, "type": "policy_decision", "id": "c2", "decision": "allow", "rule": "workspace_write"},{"seq": 8, "type": "tool_result", "id": "c2", "status": "ok"},{"seq": 9, "type": "tool_call", "id": "c3", "name": "shell", "args": {"cmd": "npm test -- dashboard"}},{"seq": 10, "type": "tool_result", "id": "c3", "status": "error", "summary": "2 failed"},{"seq": 11, "type": "assistant_message", "text": "fix the assertions and rerun..."},{"seq": 12, "type": "usage", "prompt_tokens": 61230, "completion_tokens": 1842, "elapsed_ms": 734500}]
It supports four capabilities:
- Audit: who authorized which invocation of which tool, and when;
- Reproduce: replaying in order can reconstruct the full trace and pinpoint the problematic step;
- Evaluate: accumulate samples as “task–trace–result” triples to form a regression set;
- Collaborate: sharing a session is far more efficient than a verbal description.
Three engineering caveats remain:
- Tool calls and results must be paired and carry a call identifier, otherwise parallel calls cannot be aligned;
- Large outputs should be truncated or externalized, with the log keeping only summaries and references, to avoid the session file ballooning;
- Sensitive information such as paths and tokens must be redacted, and the log retention period should match compliance requirements.
6. Comparing Execution Traces Under the Same Model: How to Compare Fairly
When comparing two AI coding tools, a common approach is to “run the same task on each and see who finishes first.” If the defaults differ, the conclusion often reflects differences in defaults rather than differences in capability. A more common problem is that swapping the model loses control of the variables: if you mix models from two different vendors, the result can’t clearly say whether it’s a “tool difference” or a “model difference,” and it easily turns into promotion of a particular product, with limited reference value.
A rigorous comparison should lock the model and compare the runtime. The process:
- Unify the model and inference parameters: connect both sides to the same model, and unify temperature, maximum output length, and context strategy. Change only the runtime itself — system prompts, tool descriptions, context compaction, and permission and sandbox defaults.
Fix the task set and environment: cover three difficulty levels:
- a single-file bug fix;
- cross-module feature development (frontend–backend linkage);
- debugging tasks that require running tests to confirm.
Each task starts from the same initial snapshot, and a “runtime error” may be injected once mid-way to test recovery.
- Set a unified budget: keep maximum steps, tokens, and wall-clock time identical, to prevent “trading more resources for a higher success rate.”
- Sample repeatedly: run each task multiple independent times, take the mean, and report the dispersion. Agent traces are highly stochastic, and a single result is not enough to go on.
- Record three kinds of data: result metrics, process metrics, and trace characteristics.
| Category | Metrics | Notes |
|---|---|---|
| Result | Task completion rate, first-pass rate, rework count | Judged by tests, builds, and artifact acceptance, not subjective impressions |
| Process | Steps, tool-call count, token usage (including cache hits), wall-clock time, manual interventions | Manual interventions include permission prompts, interruptions, and clarifying remarks |
| Trace characteristics | Share of exploration / coding / verification phases, count of re-reading the same file, recovery steps after failure, context-compaction count | Computed directly from the Session Log |
Trace characteristics best reveal runtime differences:
- Too high a share of exploration may mean the retrieval tool recalls poorly or the system prompt lacks constraints;
- Concentrated re-reading usually points to an overly aggressive context-compaction strategy;
- A large number of recovery steps after failure indicates that tool error messages are not structured enough.
Conversely, if the trace distributions of two runtimes are similar, the difference may come mainly from the model, and you should narrow the scope of comparison.
7. Complex Full-Stack Task Example: An End-to-End Loop with Visual Artifacts
Beyond basic comparison, you also need to examine long-chain tasks. Consider a generic example: add an “operations data dashboard” page to an existing project, containing three linked charts, a backend aggregation API, and a data-table migration script, and produce a screenshot of the page as a visual artifact.
| Stage | Main actions | Key aspects |
|---|---|---|
| Task understanding | Search routes, the data layer, and existing chart components; map dependencies | Read-only search; quickly build a code map |
| Solution planning | Produce API contracts, table structure, and a file-change list for human confirmation | Plan mode; writes ask by default |
| Backend implementation | Write the migration script, aggregation API, and unit tests | Run tests in the sandbox; feed back failure summaries |
| Frontend implementation | Implement chart components, filter linkage, and loading states | Limit writable directories; follow the existing component style |
| Joint debugging & verification | Start frontend and backend, run end-to-end tests, do build checks | Start services in a container; isolate ports |
| Visual output | Screenshot via headless browser at a fixed viewport; archive artifacts | Store screenshots in the artifact directory and register them in a manifest |
| Summary & delivery | Summarize changes, test results, artifact paths, and remaining risks | Generate a structured summary with a Session Log excerpt |
This chain reflects the coordination of three architectural capabilities:
- Tool loop: without the ability to “execute and verify,” the model can only write code blind; without visual tools, tasks with a UI cannot prove their own correctness.
- Context management: a real codebase far exceeds the window capacity, relying on a “load on demand via retrieval, rather than reading in the whole repo” context strategy. Each stage loads only the files relevant to the current subtask.
- Permission and isolation: frontend–backend joint debugging needs to start services and listen on ports; the permission policy should limit ports and network egress, and container isolation can prevent accidental changes to the host environment.
Generating visual artifacts can be abstracted as:
artifact = browser.goto(local_url).viewport(1440, 900) # fixed viewport to keep results reproducible.wait_for(network_idle).screenshot(path, full_page=true)
Screenshots and other artifacts go into the result directory and are registered in a manifest, so logs and deliverables remain traceable to each other. Acceptance criteria should be set before the task begins: whether all tests pass, whether key interactions have automated test coverage, and whether screenshots match the reference in key regions. Otherwise, “looks fine” is only a subjective judgment.
8. Architecture Evaluation Checklist: Eight Questions
Synthesizing the above, this kind of runtime can be evaluated from eight angles:
- Model replaceable: can you switch to any model and inference parameters via configuration?
- Tools extensible: can you declare custom tools and connect to standardized tool protocols?
- Sessions portable: can sessions be persisted, restored across entry points, and replayed?
- Permissions controllable: are policies independent of the UI, and can they be graded by project and risk?
- Execution isolatable: do tool calls run in an isolated environment, with limited resources and network?
- Traces observable: is there a structured event log, and can you compute cost and failure distributions?
- Failures recoverable: are step overruns, timeouts, infinite loops, and permission denials handled explicitly?
- Boundaries embeddable: can it be embedded as a library into a host system, providing a stable interface contract?
Meeting six or more of the eight makes it more suitable as unified agent infrastructure for a team; a tool that stands out only on “out-of-box experience” while its kernel is closed is better suited as a personal productivity tool.
When deploying to production or a team environment, this kind of runtime typically needs three pieces of supporting infrastructure:
- Inference service: model invocation can be unified through a compatible interface, with latency, throughput, and rate-limiting requirements that vary by scenario;
- Storage and logging: archive session logs to object storage, and feed runtime events into a logging service and monitoring/alerting for continuous observation of cost and failure rates;
- Execution isolation: containerize tool execution, run it on a container platform, and pair it with quota and network policies.
If your team already uses Baidu AI Cloud, you can rely on its object storage, logging service, monitoring and alerting, and container platform to standardize these capabilities.
9. Conclusion
Every iteration of strong reasoning models pushes the capability boundary of AI coding tools outward. The runtime itself does not automatically produce results; what truly determines output quality is whether the architecture is open, whether the boundaries are clear, and whether the process is observable.
- For developers: treat the runtime as an assemblable kernel, swapping models, tools, and policies as needed;
- For teams: first build an evaluation system of “same model, fixed task set, complete trace,” then discuss tool selection.
Back to the opening mismatch: those who expect a “product” get a “kernel” — a higher short-term learning cost, but in the long run a controllable, evolvable, reusable agent infrastructure.
