A Five-Layer Architecture for AI Coding Assistants

AI coding assistants often look easy to read not because one algorithm is especially clever, but because complex flows are split into layers with clear boundaries: the entry layer only handles startup and routing, the loop layer only drives the back-and-forth between the model and tools, the tool layer only turns intent into executable operations, the permission layer only decides whether an action is allowed, and the prompt layer only assembles context and constraints for the model. Each layer manages its own concerns, so changing one layer does not force the others to move.

From the outside in, the whole flow can be summarized as follows: user input enters the command-line entry point, which selects the run mode and loads the corresponding modules; once inside the agent loop, the system repeatedly calls the model, parses tool requests, executes tools, and writes results back to the model until the task ends; every concrete operation passes through permission checks; and the instructions the model sees are dynamically assembled by the system prompt layer for the current scenario. The value of layering is straightforward: readability comes from single responsibilities, evolvability from one-way dependencies, and security from decoupling permissions from execution.

1. Entry Layer: Skip Work Whenever Possible

After the user types a command, the entry layer is the first thing that runs. Its core job is not “start the agent,” but parse arguments, choose a mode, and import as little as possible. Common practices include: version queries take a minimal path that loads almost no business modules, prints quickly, and exits; subcommands such as background daemon and remote control are loaded dynamically on demand, so unrelated dependencies do not enter the default startup path; and only after confirming an interactive session does the entry layer import the full main interface and session engine.

This “lazy loading + fail fast” approach has a large impact on developer experience. Command-line tools are invoked frequently, and users directly perceive cold-start time. The entry layer separates “confirm what to do” from “load what is needed,” which lowers average startup cost and makes the code immediately clear to readers: there is no business logic here, only routing and startup strategy. An example:

async function main() {
  const args = process.argv.slice(2)
  if (args.length === 1 && args[0] === '--version') {
    printVersion()
    return
  }
  if (args[0] === 'daemon') {
    const { runDaemon } = await import('./daemon.js')
    await runDaemon(args.slice(1))
    return
  }
  const { runRepl } = await import('./session.js')
  await runRepl()
}

The entry layer should also handle differences between run modes: one-shot questions, persistent sessions, piped input, and non-interactive scripts. Different modes imply different termination conditions, output formats, and whether interactive confirmation is allowed. By placing these decisions in the entry layer, the loop layer can remain a pure driver and no longer needs to care how the user arrived.

2. Loop Layer: Drive Models and Tools with an Explicit State Machine

The loop layer is the heart of the agent. It is usually a while structure: assemble messages, call the model, parse the response, execute tool requests if present and append the results, otherwise finish or ask the user. A readable loop does not stuff filesystem, network, and permission details into the same function; instead, it narrows “what should happen this round” into a finite set of states: waiting for the model, waiting for tools, waiting for user confirmation, complete, and error retry.

Key problems the loop layer must solve include: ordering and parallelism of multi-round tool calls; recovery when model output is incomplete or malformed; truncation and summarization as the context window grows; and cancellation and timeouts. By concentrating these problems in the loop layer, tool implementations can remain stateless. When reading the code, readers only need to understand “when to stop and when to ask the model again” in the loop layer, rather than guessing the session lifecycle inside every tool.

The loop layer should also separate “model decisions” from “side effects.” The model produces only structured intent—for example, read a file, modify a file, or run a command—and the real side effects occur in the tool layer. This makes log auditing easier: each round can reconstruct input messages, model output, tool inputs, and tool outputs, so troubleshooting does not require searching through prompt strings.

3. Tool Layer: Turn Intent into Verifiable Operations

The tool layer maps model intent to concrete operations: reading directories, reading and writing files, searching, executing commands, calling internal interfaces, and so on. Readability comes from a unified contract: each tool declares its name, parameter schema, side-effect level, and whether confirmation is required. The executor validates parameters against the contract, calls the implementation, normalizes output, and returns it to the loop layer.

Tool implementations should be small and orthogonal. For example, “read file” only reads, “edit file” only edits, and “run command” only executes; a read-file tool should not secretly write a cache. Once tools are orthogonal, permission policies can be configured at tool granularity, and tests can be written per tool. Output should be structured: success and failure, truncation notices, and key paths. Avoid pushing large raw logs back to the model, which wastes context and adds parsing noise.

The tool layer also handles adaptation work: workspace root, encoding, line-number ranges, ignore rules, and similar details. These adaptations are execution details and should not leak into prompts or loop conditions. With clear layering, adding a new tool only requires registering its contract and implementation; there is no need to change the entry layer or the main body of the permission framework.

4. Permission Layer: Gate Every Operation Before Execution, Not After

The permission layer decides whether each operation is allowed. It should be independent of tool implementations: tools only declare “what I want to do,” while the permission layer combines session policy, workspace boundaries, and user authorization records to return allow, deny, or confirmation required. Common dimensions include: whether the path is inside the workspace, whether it touches sensitive directories, whether the command belongs to a high-risk set, whether network access is enabled, and whether one-time or session-level authorization already exists.

Making permissions a separate layer has two benefits. First, security policy can evolve in one place instead of copying if statements into every tool. Second, deny-by-default and least privilege become easier to implement: unauthorized writes, unknown commands, and out-of-bounds paths are blocked before execution. Interactive confirmation should also be orchestrated by the permission layer; the loop layer only consumes “authorized / pending confirmation” results, preventing UI logic from leaking into tool code.

Permission decisions should be explainable: when denying an action, give a short reason so users can understand it and auditors can review it. Policy configuration should be layered: global defaults, project overrides, and temporary session authorization. When reading the code, readers can look at the policy table first and then the executor, rather than hunting for implicit restrictions inside business branches.

5. Prompt Layer: Assemble Instructions and Context Dynamically, Not as Hard-Coded Long Text

The system prompt layer assembles instructions for the model: role and goals, available tool list, output format, workspace summary, user rules, and current task constraints. A readable prompt module separates “templates” from “runtime variables”: templates stay stable, while variables are injected by mode. Interactive sessions, one-shot tasks, and read-only exploration require different constraints, and these should be composed rather than copied and pasted as multiple extremely long strings.

The prompt layer also handles context governance: which file summaries enter the prompt, how history is compressed, and how error messages are fed back. These governance rules are tightly related to model calls but unrelated to tool implementations. Once isolated, changes to prompts can be evaluated for behavioral impact without simultaneously changing tool code.

Prompts should not encode permission details in natural language and then expect the model to comply on its own. The real gate is in the permission layer. Prompts provide behavioral style and task understanding; enforcement is handled by code. This is another meaning of layering: the model is not trusted, so policy must live outside the model.

6. How Layering Improves Readability and Evolution Efficiency

Once the five layers are connected, the reading path becomes linear: startup from the entry layer, control flow from the loop layer, capability surface from the tool layer, security boundary from the permission layer, and the model-visible world from the prompt layer. Changes also have clear landing points: if startup becomes slow, inspect entry-layer imports; if a tool loops forever, inspect loop termination conditions; if files are deleted by mistake, inspect permission policy; if the model calls tools wildly, inspect tool descriptions and parameter schemas in the prompt.

Engineering conventions can reinforce this: layers communicate only through narrow interfaces; tools must not read global session objects directly; the loop layer must not assemble long system prompts directly; and the entry layer must not call concrete tools. Conventions matter more than frameworks. The complexity of AI coding assistants grows with capability, and without layering, the entry, loop, and tool layers quickly become tangled. Layering does not eliminate complexity, but it can confine complexity to the rooms where it belongs.

In practice, start by drawing a responsibility table, then check whether the code directories map one-to-one to it. When directories align with layers, new engineers can read by layer instead of jumping across the whole repository by call stack. Tests can also be organized by layer: entry tests for routing and startup time, loop tests for state transitions, tool tests for contracts, permission tests for denial and confirmation, and prompt tests for the stability of assembled output. Once layering becomes a test boundary, regression cost drops noticeably.

Conclusion

Breaking an AI coding assistant into entry, loop, tools, permissions, and prompts is essentially turning “user input to model decision to side-effect execution” into a reasoning-friendly pipeline. The entry layer aims for minimal loading, the loop layer for a terminating state machine, the tool layer for orthogonal contracts, the permission layer for pre-execution gates, and the prompt layer for dynamic composition instead of extremely long hard-coded text. When each layer manages its own concerns, the code is easier to read, security policy is easier to audit, and capability expansion does not require changing the entire chain at once. For AI coding workflows that need to run in production, this layering is not a style preference; it is a basic way to contain uncertainty within clear boundaries.