Agent Runtime Frameworks: Engineering with a “Everything as a Plugin” Architecture
This article explores an alternative architecture for AI agents: treating the model as a replaceable execution unit rather than the system’s center of gravity. It explains how plugin-based runtime frameworks can modularize models, tools, planners, sandboxes, storage, agent loops, and governance, while examining plugin contracts, implementation practices, production risks, testing requirements, and suitable adoption scenarios. The approach is valuable for internal agent platforms and controlled automation, but should not support critical production workflows until isolation, observability, version governance, and regression evaluation are in place.
1. Reframing the AI Agent: The Model Is an Execution Unit
Traditional agent products often place the model at the center. After selecting a model, teams build tools, prompts, conversations, and execution workflows around it. This article examines a different architectural perspective: move the model out of the center, treat it as a replaceable execution unit, and make the runtime framework a first-class component.
A simplified description of an agent system is:
Agent = Model + Runtime Framework
The model handles understanding, generation, reasoning, and decision-making. The runtime framework turns those capabilities into actual work. It typically includes tool registration and invocation, task planning, scheduling, sandboxing, storage, sessions, context management, loop control, permission policies, auditing, and observability.
The value of this formula is not its simplicity, but the way it changes system boundaries. In an integrated product, the model, toolchain, and user experience are often tightly coupled. This provides a consistent, out-of-the-box experience, but replacing the model, integrating enterprise systems, or customizing execution policies may require changes across the entire implementation.
A plugin-based runtime framework attempts to separate these parts through explicit contracts. The model can be replaced, as can tools, sandboxes, session storage, and even the agent loop itself.
From an engineering perspective, the runtime can be divided into four planes:
- Control plane: Planning, model routing, permission policies, quotas, plugin lifecycle management, and task scheduling.
- Execution plane: Tool invocation, sandbox execution, file operations, network access control, and resource limits.
- Data plane: Session state, long-term memory, artifact storage, context compression, and retrieval-augmented generation.
- Observability plane: Logs, metrics, distributed tracing, auditing, evaluation, and replay.
“The model is not the center” means that the model should focus on the work it does best. Model selection, cost control, failure fallback, and tool-calling format adaptation should be handled by model adapters and the runtime framework rather than scattered across individual business prompts.
2. What Does “Everything as a Plugin” Mean?
“Everything as a plugin” does not mean that every piece of code must be implemented as a plugin. It means that the variable parts of the runtime are extended through stable contracts.
To prevent the system from becoming unmanageable, the framework should retain a small, auditable core. This core is responsible for plugin discovery, registration, dependency resolution, permission checks, lifecycle management, the event bus, and basic telemetry. Other capabilities can be integrated as plugins.
Typical plugin candidates include:
| Component | Primary responsibility | Value of replacement | Key risks |
|---|---|---|---|
| Model adapter | Unify chat, streaming output, tool calls, structured results, and usage tracking | Enables multi-model routing, gradual rollout, and fallback | Capability differences, unstable output formats, uncontrolled cost |
| Tool plugin | Query data, call internal systems, and perform specific actions | Quickly integrates enterprise capabilities and supports selective activation | Side effects, privilege escalation, timeouts, and retries |
| Planner | Break down tasks, generate steps, and re-plan | Adapts to different levels of business complexity | Plan drift, infinite loops, and limited explainability |
| Scheduler | Control concurrency, priority, retries, budgets, and quotas | Improves throughput and reliability | Resource contention, retry amplification, and starvation |
| Sandbox executor | Isolate processes, files, networks, and resources | Reduces execution risk and supports safer execution | Incomplete isolation, escape risks, and performance overhead |
| Storage and session layer | Store messages, artifacts, memories, and checkpoints | Supports long-running task recovery and multi-tenancy | Data consistency, privacy, and lifecycle management |
| Agent loop | Observe, reason, call tools, feed results back, and terminate | Enables customized reasoning and execution strategies | Unclear termination conditions, poor observability, and difficult replay |
| Policy and observability layer | Handle approvals, auditing, rate limiting, tracing, metrics, and evaluation | Makes the system controllable and measurable | Complex rules, missing instrumentation, and misleading evaluations |
A mature plugin-based runtime should not treat every plugin as equally trusted. Tool plugins may create external side effects. Model adapters may access sensitive context. Sandbox plugins directly affect isolation boundaries. Policy plugins may determine whether an operation is allowed.
Therefore, plugins must not merely be “registerable”; they must also be governable.
3. Plugin Contracts: From Slogan to Engineering Practice
The first principle of plugin architecture is contract first. Without stable contracts, the more plugins a system has, the harder it becomes to maintain. A general plugin manifest should declare at least an identifier, version, API version, capabilities, required permissions, configuration constraints, and dependencies.
// Generic pseudocode: independent of any specific implementationPluginManifest {id: stringapiVersion: stringversion: stringcapabilities: string[]permissions: string[]configSchema: objectdependencies: Dependency[]}ToolPlugin {execute(ctx: ExecutionContext, input: unknown): Promise<ToolResult>}ModelAdapter {complete(req: CompletionRequest): AsyncIterable<ModelEvent>}SandboxExecutor {run(task: TaskSpec): Promise<TaskResult>}AgentLoop {run(ctx: RunContext, input: UserInput): AsyncIterable<RunEvent>}PluginHost {register(manifest, factory)resolve(capability, versionRange)async invoke(capability, ctx, input) {checkPermission(ctx, manifest)validateInput(input, manifest.contract)applyTimeoutAndQuota(ctx)emitTraceEvent("plugin.invoke.start", ctx, manifest)result = await plugin.execute(ctx, input)emitTraceEvent("plugin.invoke.end", ctx, manifest, result)return normalize(result)}}
This pseudocode highlights several important principles:
- Capability declaration: Plugins expose capabilities through
capabilities, so callers depend on capabilities rather than concrete implementations. - Permission declaration: The resources a plugin requires must be explicit in its manifest. The default should be deny, with access granted only when needed.
- Version and API compatibility: Plugin versions and contract versions should be managed separately, preventing a core upgrade from breaking every plugin.
- Input validation: Inputs should be validated before invocation. Failures should produce structured errors rather than exposing raw exceptions to the model.
- Unified timeouts and quotas: A single plugin must not consume unlimited resources, and retries must not amplify without control.
- Observable events: Every invocation should emit trace events containing duration, result status, error type, and the necessary audit information.
A typical runtime follows a clear sequence:
- User input enters the session context.
- The planner generates a task plan.
- The scheduler selects executable steps.
- The model adapter generates a tool-call intent.
- The plugin host validates permissions and invokes the tool.
- The sandbox executor handles operations with side effects.
- The result is normalized and added back to the context.
- The loop controller determines whether execution should continue.
- The final result is returned, while audit and evaluation data are recorded.
Every stage should be observable, replayable, and replaceable.
4. What Is It Like to Build a Plugin?
Building a plugin is not mysterious. The core requirements are to declare capabilities, implement the contract, and declare permissions.
Consider an enterprise ticket-query plugin. It can be designed as a read-only tool: the input consists of a search keyword and a result limit, while the output contains ticket summaries in a standard format. The plugin should not read global configuration directly or hard-code credentials. Instead, it should obtain tenant information, credential handles, timeout settings, and tracing identifiers from the execution context.
// Generic pseudocode: enterprise ticket query toolclass TicketQueryTool implements ToolPlugin {manifest = {id: "ticket.query",apiVersion: "v1",version: "1.2.0",capabilities: ["ticket.read"],permissions: ["ticket:read"],configSchema: {endpointRef: { type: "secretRef" },maxLimit: { type: "number", default: 20 }}}async execute(ctx, input) {validate(input, { keyword: "string", limit: "number" })limit = min(input.limit ?? 20, ctx.config.maxLimit)ctx.audit("ticket.query", { keywordHash: hash(input.keyword), limit })response = await ctx.connector.call("ticket.read", {keyword: input.keyword,limit,tenant: ctx.tenantId})return {items: response.items.map(item => ({id: item.id,title: item.title,status: item.status,updatedAt: item.updatedAt})),truncated: response.hasMore,source: "ticket.read"}}}
This example demonstrates several important constraints:
- Least privilege: The plugin requests read access only, not write access.
- Input validation: It validates types and limits the result count so that oversized results are not pushed directly into the context.
- Side-effect isolation: A read-only query does not modify an external system. Write operations should require approval or a second confirmation.
- Result normalization: Stable fields make results easier for the model to understand and easier to test.
- Auditing and redaction: Search keywords may contain sensitive information, so logs should use hashes or redacted values.
- Timeouts and fallback: If the external system is unavailable, the plugin should return a structured error. The loop controller can then decide whether to retry, switch tools, or terminate the task.
After implementation, a plugin should pass three categories of tests:
- Contract tests to verify input and output formats.
- Permission tests to verify that unauthorized requests are rejected.
- Failure tests to verify that timeouts, empty results, and abnormal responses do not break the main loop.
If a plugin will be shared by multiple teams, version-compatibility tests and configuration migration instructions should also be provided.
5. Benefits and Costs of a Plugin-Based Architecture
The main benefits of a plugin architecture include the following:
- Replaceable models: The same business workflow can use different models through adapters and route requests based on task type, cost budget, or availability. However, replacing a model does not guarantee equivalent behavior. Evaluation datasets must be used to verify tool-call accuracy, task completion rate, and format compliance.
- Composable tools: Enterprise systems can be split into capability-oriented plugins and enabled or disabled by tenant, environment, or business line without modifying the main workflow each time.
- Evolving execution strategies: Planners, loops, and schedulers can be upgraded independently, making experimentation and gradual rollout easier.
- Clearer testing boundaries: Tool plugins, model adapters, and sandbox executors can each receive focused unit and integration tests.
- A stronger foundation for platformization: Once manifests, permissions, versions, and observability are standardized, organizations can build internal plugin marketplaces, capability catalogs, and governance processes.
The costs are equally significant:
- Greater contract complexity: The more granular the plugins, the more complex interface, version, and dependency management become.
- Longer debugging paths: A single task may pass through many plugins, requiring distributed tracing and event replay.
- Additional performance overhead: Registration, validation, serialization, and isolation all introduce cost.
- A larger security boundary: Every plugin is a potential attack surface, requiring default-deny policies, least privilege, and isolated execution.
- More difficult evaluation: Model nondeterminism combined with tool side effects makes regression testing and production evaluation more complex.
Therefore, “everything as a plugin” does not mean that every component should be split into a plugin. A more practical engineering principle is:
Minimize the core and maximize extensibility. Keep the high-trust core auditable, isolate low-trust plugins strongly, and establish stable contracts before increasing the plugin count.
6. Production Readiness Checklist
Before using a plugin-based agent runtime in production, verify at least the following capabilities:
- Permission and tenant isolation: Plugin calls must be bound to tenant, user, and task identities. Undeclared permissions should be denied by default. Sensitive operations should require approval and a second confirmation.
- Sandboxing and resource limits: Tools with side effects should run in controlled environments with limits on processes, files, networks, and execution time. Plugins must not bypass the host to access host resources directly.
- Version and compatibility governance: Core contracts, plugin versions, and configuration structures need explicit compatibility policies. The system should support gradual rollout, rollback, and parallel operation of older versions.
- Observability and auditing: Record task traces, plugin calls, model input and output summaries, permission decisions, and error categories. Sensitive fields must be redacted.
- Evaluation and replay: Establish metrics for task success, tool-call accuracy, latency, cost, and the rate of human intervention. Support session replay and the collection of failed samples.
- Failure fallback: When a model is unavailable, a tool times out, or a sandbox fails, the system should have a defined fallback path, such as returning an explainable error, switching to read-only mode, or routing the task to a human.
- Cost and quota controls: Set budgets and rate limits by tenant, task, and plugin to prevent retry storms and uncontrolled context growth.
- Plugin admission and retirement: Before release, plugins should pass contract tests, permission reviews, failure tests, and documentation checks. Plugins that are no longer maintained should be removed.
- Data governance: Define retention periods, access scopes, and deletion mechanisms for sessions, memories, artifacts, and logs.
- Release process: Validate the framework in low-risk internal scenarios first, then perform a small-scale rollout before allowing it to support critical workflows.
7. Appropriate Boundaries and an Adoption Strategy
Plugin-based runtime frameworks are particularly suitable for three types of scenarios:
- Agent platforms that need to integrate multiple internal systems, tools, and models.
- Research and engineering teams that continuously experiment with different models, loop strategies, and tool combinations.
- Organization-wide infrastructure for evaluation, auditing, and permission governance.
A phased adoption strategy is recommended.
During the first phase, use controlled internal scenarios. Standardize the plugin manifest, permission model, and tracing format, without focusing on the number of plugins.
During the second phase, add evaluation replay, version compatibility, and failure drills. This makes “replaceability” verifiable rather than merely theoretical.
During the third phase, consider a multi-team plugin marketplace and more granular runtime isolation.
If a business requires extremely high real-time performance, strong consistency, or strict security, but the team has not yet established sandboxing, auditing, and rollback capabilities, it should not rush into production merely because the architecture appears advanced.
Conclusion
“Everything as a plugin” offers an important engineering perspective: move the model out of the center and place the runtime framework, contracts, and governance at the center instead.
The approach is worth studying and applying in controlled environments, particularly for its decoupling model and platform boundaries. However, production adoption should not be judged by the number of plugins. The more important question is whether permissions, isolation, observability, evaluation, and rollback form a complete control loop.
The value of the architecture is not that everything can be replaced. It is that every replacement can be verified, traced, and rolled back.
