Pluggable Agent Runtime Architecture: Moving the Model Out of the Center

1. Reframing the problem: the bottleneck is not the model

Over the past two years, discussions about agents have often reduced the problem to a single variable: model capability. But when you open up an agent system, what determines whether it can actually do the work is often not the foundation model itself. It is the runtime around the model: how tools are registered, how context is trimmed, how tasks are decomposed, where code executes, how failures are retried, and where state is stored.

If a framework ships “model + tool chain + interaction interface” as a single product, the runtime becomes fixed inside the product form. It may run well, but it is difficult to decompose. Switching model providers requires code changes. Switching sandboxes requires code changes. Adding your own policy layer to a session still requires code changes.

So the more useful question is not “how do we build another coding assistant?” but:

What components make up an agent? Can those components be replaced, evolved, and tested independently?

This is what a pluggable agent runtime is designed to solve.

2. A decomposable formula: Agent = Model + Harness

An agent can be described with a simple layered structure:

Agent = Model (inference and decision-making) + Harness (everything that turns decisions into actions)

• Model: responsible for understanding, reasoning, and generating decisions. It is stateless: it receives context and outputs actions or text.
• Harness: the external runtime of the model. It includes the tool layer, context and memory management, task planning and orchestration, execution sandbox, session and state storage, loop control, permissions and auditing, and observability.

There are three key conclusions:

1. The model is a replaceable component, not the center. Different tasks have very different trade-offs in cost, latency, context length, and function-calling stability. Hard-coding the model gives up tuning space.
2. The complexity of the harness is higher than model invocation over the long term. For a usable agent, 80% of the engineering problems live in the harness.
3. Clear layering is necessary for meaningful evaluation. Model capability, tool design quality, loop strategy, and prompt quality must be attributable separately. Otherwise, online issues can never be located.

3. Harness component checklist: what is worth abstracting

Component	Responsibility	Suitability for pluginization	Typical reasons to replace
Model access	Unified request/response, streaming, function calling, retries	High	Cost, capability, private deployment
Tool set	Tool registration, schema generation, parameter validation, permissions	High	Largest business variance, most frequent changes
Context management	Trimming, summarization, retrieval injection, caching	High	Key to the success of long-running tasks
Planning and orchestration	Task decomposition, parallel subtasks, dependency scheduling	Medium	Needed only for complex workflows
Execution sandbox	Isolated environment, filesystem, network egress	High	Large differences in security levels
Session and state	Persistence, resumability, multi-turn context	Medium	Storage selection and compliance requirements
Main loop	Termination conditions, retries, error recovery strategy	Low to medium	Highest change risk; open cautiously
Observability and evaluation	Tracing, metrics, offline evaluation sets	High	Must be a first-class concern

A practical rule of thumb:

The closer a component is to the business, and the more frequently it changes, the earlier it should become a plugin. The closer it is to the core control flow, the more it should remain constrained.

Fully opening the main loop to developers outside a mature team usually results in unmaintainable forks.

4. Six mechanisms required to make “everything is a plugin” work

The slogan is easy to write, but the engineering is hard. A usable plugin system needs at least the following six things to hold at the same time.

1. Stable contracts

All plugins depend on a small set of minimal interfaces. The contract must be narrow, stable, semantically clear, and explicit about error semantics.

# Minimal contract for a tool plugin
class ToolPlugin(Protocol):
    name: str
    description: str
    # JSON Schema used to generate function-calling definitions
    def parameters_schema(self) -> dict: ...
    # Returns a structured result; exceptions must be converted into structured errors, not raised
    def run(self, **kwargs) -> ToolResult: ...

A frequently overlooked point is that plugins must not throw raw exceptions to the main loop. The contract must define unified error types, such as parameter errors, permission denial, timeouts, and downstream unavailability. Otherwise, the main loop cannot decide whether to retry or terminate.

2. Registration and discovery

Plugins should be decoupled from core code and discovered through declarative metadata rather than hard-coded imports.

plugin:
  name: repo_search
  version: 1.2.0
  entry: plugins.repo_search:RepoSearchTool
  capabilities: [read, network]
  requires_config: [index_endpoint, token_ref]

3. Declarative assembly

System behavior should be determined by configuration, not code branches. This allows the same binary to support different agent forms for different scenarios.

agent:
  model: provider_a.chat          # The model is just a referenced component
  loop: loops.conservative        # Change strategy without changing code
  context_manager: ctx.sliding_window_summarize
  sandbox: sandbox.container
  session_store: store.object_storage
  tools:
    - repo_search
    - shell_exec
    - http_request
  policy:
    max_steps: 40
    require_approval_for: [shell_exec]

4. Lifecycle management and dependency injection

Initialization, configuration injection, resource release, and health checks need unified orchestration. Dependencies between plugins should be explicitly declared as a directed acyclic graph and validated at startup, not discovered at runtime through failures.

5. Isolation and permission boundaries

Plugins are one of the most easily overlooked attack surfaces. At minimum, there should be three layers of constraints:

• Capability declaration: plugins declare the permissions they need in metadata, such as network access, file read/write, and command execution. These can be reduced at runtime.
• Resource quotas: timeouts, memory limits, and output-length limits prevent a single plugin from dragging down the whole system.
• Human approval points: high-risk actions, such as deletion, deployment, or external sending, must enter an approval flow.

6. Built-in observability

Every plugin invocation should produce a unified structured record: an input summary, duration, token consumption, return code, and the associated session and step IDs. Without this layer, pluginization does not create flexibility; it creates a black box.

5. The complete loop for writing your own plugin

Here is an example: adding an internal retrieval tool to an agent. The complete loop has four steps.

Step 1: Implement the contract. Define the tool’s semantic boundary: what it does, what it does not do, and what it returns on failure.

class DocSearchTool:
    name = "doc_search"
    description = "Retrieve relevant snippets from an internal enterprise documentation repository."
    def parameters_schema(self):
        return {
            "type": "object",
            "properties": {
                "query": {"type": "string"},
                "top_k": {"type": "integer", "default": 5},
            },
            "required": ["query"],
        }
    def run(self, query: str, top_k: int = 5) -> ToolResult:
        if not query.strip():
            return ToolResult.error("EMPTY_QUERY")
        try:
            hits = self._backend.search(query, top_k=min(top_k, 20))
        except TimeoutError:
            return ToolResult.error("UPSTREAM_TIMEOUT", retryable=True)
        return ToolResult.ok(hits)

Step 2: Declare metadata. Declare permissions and configuration requirements so the runtime can enforce admission control.

Step 3: Assemble. Add the tool name to the configuration, then restart or hot-load. No changes to the main loop are required.

Step 4: Evaluate. Prepare a set of offline test cases, including scenarios that intentionally trigger timeouts, empty parameters, and insufficient permissions. Verify that the tool’s error branches are handled correctly by the main loop, and then verify whether end-to-end task success improves.

This step is where pluginization delivers its real benefit: a tool can be tested, rolled out gradually, and rolled back independently.

6. The cost: pluginization is not free

This design has clear costs. Ignoring them is more dangerous than not pluginizing at all.

• Premature abstraction: once a contract is published, it is hard to change. A practical suggestion is to abstract only components for which a second implementation has actually appeared.
• Contract leakage: if the text returned by a tool is directly concatenated into a prompt, the wording of the tool output becomes part of model behavior. This creates implicit coupling between plugins.
• Longer call chains: configuration parsing, dependency injection, and registry lookup all add overhead. Hot paths need caching. Cold paths should not be over-engineered.
• Diluted observability: dynamic assembly makes it difficult to answer what components actually ran during a request. The assembly result must be snapshotted into the trace for each execution.
• Supply chain risk: a plugin ecosystem means third-party code enters your execution path. Signature verification, version locking, sandbox isolation, and least privilege are baseline requirements, not optional ones.

7. Adoption roadmap and selection checklist

If you want to move in this direction in your own engineering work, proceed in four levels. Do not try to do everything at once.

1. Abstract the model access layer first. The benefit is the most direct and the risk is the lowest. You immediately gain the ability to switch providers and support private deployment.
2. Fully pluginize the tool layer. Business differences are concentrated here, and it is also the interface boundary for parallel development across teams.
3. Pluginize context management and sandboxing. These directly determine long-running task performance and security.
4. Open the main loop last, and by default keep it closed. Two or three validated preset strategies are usually enough.

When evaluating whether a framework is truly decoupled, these five questions are useful:

• Does switching models require modifying business code?
• Does adding a new tool require modifying core modules?
• Can the same agent logic switch to different execution sandboxes?
• Can you obtain per-component latency, cost, and failure attribution?
• Are high-risk actions subject to mandatory approval and complete auditing?

Only when the answer is yes to all five can the architecture be called pluggable.

8. Conclusion

The selling point of several recently popular agent frameworks is not “stronger coding ability.” It is a runtime design that decomposes an agent into replaceable components. The value of this type of project is primarily architectural: it removes the implicit assumption that the model is the center, and brings agents back from a product form to an engineering system.

The practical advice for engineering teams is clear: the architecture is worth reading, the layering and contract design are worth borrowing, and pluginization efforts can start with the model access layer and the tool layer. But do not open the main loop completely to business teams, and do not rush to abstract before a second implementation exists.

For deployment and operation, using general-purpose cloud container platforms, object storage, logging, monitoring, and alerting to support session persistence and end-to-end tracing is usually more cost-effective than building these capabilities in-house. These are general-purpose capabilities. When selecting them, focus on isolation level, observability integration, and data retention policies.