Agent Runtime Architecture: Layered Design and Engineering Practices

    This article explains how an AI agent runtime turns language model outputs into controlled, executable actions. It covers runtime layering, model and tool interfaces, session management, authorization, sandboxing, the agent loop, and practical safeguards for validation, auditing, and failure control.

    1. From Model Capabilities to an Executable System

    A large language model generates its next output based only on the available context. It can propose intentions such as “read a file” or “run a command,” but a separate runtime is required to actually access the filesystem, reach external services, and maintain state across multiple turns. The runtime translates model output into controlled actions, then feeds execution results back to the model so that the next reasoning step is based on real feedback.

    This determines where the engineering focus should be in an agent system. Many teams initially concentrate on prompts and model selection. Once the system enters a real environment, however, permission boundaries, state recovery, and execution auditing often have a greater impact on usability and reliability.

    It is also important to distinguish between a product and a runtime. A product is a complete application for end users, including its interface, default configuration, and predefined workflows. Its goal is to work out of the box. A runtime is a modular, composable foundation designed to be stable, replaceable, observable, and testable. The same runtime can support multiple product experiences or be embedded into an organization’s existing systems.

    2. A Seven-Layer Architecture

    A reusable agent runtime typically includes the following components:

    1. Model adapter layer: Hides differences between model services and provides a unified interface for message input, tool-call output, and streaming responses.
    2. Tool layer: Describes each tool with a declarative schema covering its name, parameters, and return values. The runtime is responsible for validating arguments.
    3. Session layer: Stores message history, the working directory, authorized scopes, and the results of context compression.
    4. Authorization layer: Determines whether an action should be allowed directly, require human approval, or be denied.
    5. Sandbox layer: Executes commands and file operations in an isolated environment while restricting resource usage and network egress.
    6. Agent loop: Repeatedly calls the model, parses actions, checks permissions, executes tools, and writes results back until a termination condition is reached.
    7. Interaction layer: Exposes the system through a web interface, a headless CLI, a language SDK, or a JSON-RPC-based interface.

    Layers should communicate only through clearly defined data structures. The interaction layer should not call the sandbox directly. Instead, it should submit a task to the session layer, after which the loop layer handles orchestration. These boundaries make it possible to replace a model service or add a new interaction entry point without requiring changes across the rest of the system.

    2.1 The Model Adapter Layer

    The main responsibility of the adapter layer is to provide a unified internal message format. The following example illustrates a possible interface; it is not the actual signature of any specific service:

    1. from dataclasses import dataclass
    2. from typing import Protocol, Iterable
    3. @dataclass
    4. class ToolCall:
    5. id: str
    6. name: str
    7. arguments: dict
    8. @dataclass
    9. class ModelReply:
    10. text: str
    11. tool_calls: list[ToolCall]
    12. class ModelProvider(Protocol):
    13. def complete(self, messages: list[dict], tools: list[dict]) -> ModelReply: ...
    14. def stream(self, messages: list[dict], tools: list[dict]) -> Iterable[str]: ...

    As long as a new service implements these two methods, the loop layer does not need to change. In production, the adapter must also handle timeouts, rate limits, and exponential backoff retries. Each call should record latency and usage so that the system can later analyze operational costs.

    2.2 The Tool Layer and Argument Validation

    A tool schema acts as the contract between the model and the runtime. Arguments generated by the model are untrusted input. Before execution, the runtime must validate them, including at least the following checks:

    • Verify types and required fields, and reject calls with invalid structures.
    • Normalize paths before checking them, and ensure that the resolved result remains within an authorized root directory. This prevents directory traversal through relative paths or symbolic links.
    • Pass command arguments as a list rather than constructing a command string. User-controlled fragments should be escaped individually to reduce the risk of command injection.

    Tool results also need a structured format containing the status, a content summary, and error information. Structured results make it easier for the model to interpret execution outcomes and for logging systems to categorize failure causes.

    3. A Minimal Agent Loop

    The agent loop is the orchestration core of the runtime. The following example shows its basic structure:

    1. def run_turn(session, provider, registry, policy, sandbox, max_steps=20):
    2. for _ in range(max_steps):
    3. reply = provider.complete(session.messages, registry.schemas())
    4. session.append_assistant(reply.text, reply.tool_calls)
    5. if not reply.tool_calls:
    6. return session.final_answer()
    7. for call in reply.tool_calls:
    8. decision = policy.check(call, session)
    9. if decision.action == "deny":
    10. result = {"status": "denied", "reason": decision.reason}
    11. elif decision.action == "ask":
    12. approved = session.request_approval(call, decision.reason)
    13. result = sandbox.run(call) if approved else {"status": "rejected"}
    14. else:
    15. result = sandbox.run(call)
    16. session.append_tool_result(call.id, result)
    17. raise RuntimeError("Maximum step count exceeded; task terminated")

    This logic contains three important safeguards.

    First, there are two termination conditions: the model stops requesting tools, or the loop reaches its maximum step count. The step limit is the final protection against an uncontrolled loop and unnecessary quota consumption.

    Second, every tool call must pass through the policy check. There should be no execution path that bypasses the authorization layer.

    Third, every result is written to the session layer. This makes the complete process replayable and auditable.

    4. Authorization Policies and Sandboxing

    4.1 Policies Should Be Data

    Representing authorization rules as configuration rather than scattering them throughout application code makes them easier to review and version. The following is an illustrative policy file:

    1. rules:
    2. - tool: read_file
    3. allow_paths: ["./workspace/**"]
    4. action: allow
    5. - tool: write_file
    6. allow_paths: ["./workspace/**"]
    7. action: ask
    8. - tool: shell
    9. action: ask
    10. - tool: "*"
    11. action: deny

    Three principles are useful when designing such policies.

    First, deny by default. Any tool that is not explicitly listed should be unavailable.

    Second, define rules at the tool and path level instead of relying on a single global switch. This allows permissions to match the actual scope of an operation.

    Third, human approval prompts must show the specific action, target path, and potential impact. A dialog that only asks whether to “allow” an operation turns approval into a formality rather than a meaningful security control.