LLM Agent Runtime Architecture: A Plugin-Based Approach
Modern agent runtimes are moving toward a plugin-based architecture, treating nearly every capability—models, tools, skills, sessions, storage, sandboxes, agent loops, scheduling, and interfaces—as a pluggable component. Drawing on an MIT-licensed open-source framework as an example, this article explains the core design behind this approach: unified plugin contracts, lifecycle management, and dependency injection through a plugin container. It then walks through the four working modes such runtimes offer—Standard, PTC (programmatic tool calling), Minimal, and Create—along with the engineering essentials of core plugins and practical guidance on security, evaluation, and observability. The key takeaway for developers: a runtime's real ceiling is set not by how many plugins it has, but by the rigor of its plugin contract, the strength of its sandbox isolation, and the completeness of its evaluation system.
1. Background: Why Agent Runtimes Are Moving Toward Plugins
Over the past couple of years, agents have grown from experimental chat demos into engineering tools that can read code, execute commands, retrieve information, and operate systems. A complete agent system typically has four core parts: the LLM that handles reasoning and decisions, the tools that execute actions, the agent loop that drives the repeated “reason—act—observe” cycle, and the underlying infrastructure that manages session context, persistent state, and isolated execution environments.
Most teams’ first implementation puts all of these in the same codebase. That is the most efficient approach during prototyping, but problems surface once tasks become complex:
- High change costs: Adding an external tool often means modifying the loop logic or the main flow. The tighter the coupling, the larger the regression scope.
- Hard to replace capabilities: Comparing different models, storage backends, or execution environments usually means copying the code and starting a new branch.
- Blurred permission boundaries: Sensitive capabilities like terminals, files, and networks are scattered across business code, making unified least-privilege control difficult.
- Difficult scenario adaptation: Batch jobs don’t need a complex interface, and security-audit tasks allow only a minimal toolset, but a monolithic design struggles to “trim on demand.”
To solve all of these at once, the runtime needs a stable extension layer: one that decouples each capability implementation from the core loop and turns it into a plugin that is declarative, registrable, replaceable, and observable. This is the fundamental reason many agent runtimes in the open-source community have chosen a plugin-based architecture in recent years. This article analyzes an MIT-licensed open-source, plugin-based agent runtime framework, abstracts its architectural approach and engineering practices, and illustrates them with example code.
2. “Everything Is a Plugin”: Architecture Essence and Key Design
2.1 Nine Pluggable Capability Domains
The most notable feature of these frameworks is that “everything is a plugin.” Beyond the intuitive elements like models, tools, and skills, sessions, storage, sandboxes, agent loops, scheduling, and even interfaces are all provided as plugins. These can be grouped into nine categories:
| Capability domain | Responsibility | Typical reason to replace |
|---|---|---|
| Model | Provides reasoning and generation, with support for multi-model routing | Choose different models per task, balancing quality, latency, and cost |
| Tool | Executes external actions (commands, retrieval, API calls) | Add new capabilities, or tighten/replace sensitive tools |
| Skill | Encapsulates reusable operational experience and workflow templates | Turn domain knowledge into shareable assets |
| Session | Manages multi-turn context and state | Control context length and compression strategy |
| Storage | Persists session records and intermediate artifacts | Switch between local and cloud storage |
| Sandbox | Isolates the execution environment and limits system resources | Switch between development and restricted environments |
| Agent loop | Drives the main “reason—act—observe” flow | Change the task strategy, or adjust termination and budget conditions |
| Scheduling | Manages task concurrency and execution order | Support batch jobs and multi-agent collaboration |
| Interface | Carries interaction forms (CLI, web, GUI desktop, etc.) | Provide different entry points for different users |
The core value of this design is not the number of plugins but the separation of “capability implementation” from “loop logic.” The runtime kernel keeps only a relatively stable skeleton, and specific capabilities are assembled by plugins at startup or runtime. The result: swapping a model requires no loop-code changes, adding a tool only requires registering a new plugin, and trimming capabilities only requires adjusting the plugin manifest.
2.2 Plugin Contracts: Declaration, Lifecycle, and Extension Points
To let any plugin integrate safely, there must be a unified plugin contract containing at least:
- Metadata and configuration: name, version, capability description, parameter definitions, and their validation rules.
- Lifecycle: initialize, enable, disable, destroy. Resources should be released on disable, and state should be cleaned up on destroy.
- Extension points: for example, interception hooks before and after tool calls, usable for unified permission checks and audit logging.
- Dependency declarations: explicitly declare the other plugins a plugin depends on, avoiding implicit coupling.
Example code (only to illustrate the form of the contract):
# Tool plugin example (Python-style pseudocode)@tool(name="repo_search",description="Search for code snippets by keyword in a code repository",config_schema=SearchConfig, # Parameter declaration and validationpermission="read_only", # Capability declaration, checked by the permission layer)class RepoSearchTool:def __init__(self, context):self.context = context # Runtime context injected by the plugin containerdef run(self, keyword: str, max_results: int = 10) -> list[dict]:# Read plugin configuration instead of hardcoding parameterscfg = self.context.config("repo_search")# Use the capability provided by the storage plugin rather than touching the filesystem directlyrepo = self.context.get("storage").repository(cfg.repo_path)return repo.search(keyword, limit=max_results)
This reflects two principles of plugin-based design: externalized configuration and capabilities obtained through the plugin container. Together, they ensure plugin portability.
2.3 Plugin Container and Dependency Injection
Bringing a plugin architecture to life requires a “plugin container.” It handles plugin discovery and registration, instantiation, configuration injection, lifecycle management, and error isolation. Taking one plugin as an example: when the container creates it, it passes a unified runtime context into the constructor, and the plugin then obtains configuration, calls other plugins, and registers extension points through that context. This pattern is called dependency injection, and it brings three benefits:
- Plugins don’t need to know the specific implementation of other plugins, and can be swapped for mock implementations during testing;
- Different implementations of the same capability can coexist, with configuration deciding which one is enabled;
- When problems occur at runtime, they can be traced to a specific plugin rather than to a whole block of main-flow code.
If a plugin contains generic logic such as error handling, parameter validation, logging instrumentation, and permission declarations, consider splitting it into several smaller plugins to be used in combination, rather than making a single plugin shoulder all responsibilities. After splitting, each plugin can be tested and evolved independently.
3. Four Working Modes: Different Forms of Capability Orchestration
To fit needs ranging from everyday use to deep customization, these desktop runtimes offer agents four working modes.
3.1 Standard Mode: The Default Complete Workflow
Standard mode is the default option. The agent can select and call tools from the full toolset as needed, and the loop advances across multiple “reason—act—observe” cycles.
It suits interactive scenarios such as day-to-day development assistance, research, and document organization. Its advantages are broad capability coverage and a low learning curve. One thing to watch is context growth: in long tasks, observation results accumulate continuously, so truncation and compression should be handled together with the session plugin.
3.2 PTC Mode: Let the Model “Write a Program” to Call Tools
PTC mode (Programmatic Tool Calling) changes the granularity of tool calls. Instead of calling tools one by one, the model first writes a program; the runtime executes that program in a controlled environment, and the program itself organizes and calls tools in batch, in loops, and concurrently.
An analogy: Standard mode is like a person clicking buttons one at a time, pausing after each click to see the result before deciding the next step; PTC mode is like writing the whole sequence of operations into a script and running it all at once.
# PTC mode example: a "tool orchestration program" generated by the model (pseudocode, API names are illustrative)# 1. List the files to processfiles = tools.list_files(path="src", suffix=".py")targets = [f for f in files if "deprecated" in tools.read_file(f)]# 2. Read in parallel and locate the positions that need modification (tool calls happen in batch inside the program)tasks = [(f, "TODO_FIXME") for f in targets]hits = tools.parallel_search(tasks, concurrency=8)# 3. Generate patches by rule and write them infor file, matches in hits.items():patch = build_patch(matches)tools.apply_patch(path=file, patch=patch)# 4. Summarize execution results and return only a summary to the modelsummary = {"scanned": len(files), "modified": len(hits)}tools.write_report("report.json", summary)print(summary)
The benefits of this approach come in three main areas:
- Reduced round-trip overhead: Calling tools one by one might require dozens of round-trips between the model and the tools for a few dozen files; writing them into a program needs only one generation and one execution.
- Context savings: Intermediate results flow within the program, and only a summary is returned, preventing the context from being flooded by large amounts of intermediate data.
- Programmatic capability: Logic such as loops, conditionals, exception handling, concurrency, sorting, and aggregation can be expressed directly, instead of forcing the model to do it “manually” over and over in the conversation.
Its risks are equally clear: the correctness of generated programs depends on validation and testing inside the sandbox; execution timeouts and resource limits must be set to prevent infinite loops or resource exhaustion; and strict permission control is needed whenever scripts involve credentials and network access.
3.3 Minimal Mode: Terminal Tool Only
Minimal mode provides the agent with only the terminal tool, with all other capabilities disabled. The model can rely only on the command line to observe the environment, perform operations, and verify results on its own.
It suits two types of scenarios: first, security-sensitive or resource-constrained environments where fewer tools mean a smaller attack surface; second, as a control baseline for evaluating a model’s independent reasoning and self-checking ability. Two points need attention:
- Command output should be truncated and structured appropriately to avoid massive logs instantly filling the context;
- It must run in a sandbox with limits on the working directory, allowed executables, and network access.
3.4 Create Mode: Writing Plugins and Custom Modes
Create mode targets developers, allowing users to write plugins that extend the runtime’s capabilities and define custom working modes. For example, you can add query tools for specific data sources, encapsulate internal approval workflows, or change the agent loop’s advancement strategy. It is the key entry point for upgrading the runtime from “using tools” to “building tools,” but it must be paired with strict plugin review, permission declarations, and version management to keep uncontrolled code out of the production pipeline.
3.5 Comparison of the Four Modes
| Dimension | Standard | PTC | Minimal | Create |
|---|---|---|---|---|
| Tool scope | Full toolset | Full toolset (mainly invoked via programs) | Terminal only | Depends on custom plugins |
| Calling method | Model calls one by one | Model generates a program, then batch calls | Model executes commands directly | Determined by custom logic |
| Context pressure | Higher | Lower (only a summary is returned) | Medium (command output can be large) | Depends on design |
| Main risk | Context bloat | Program correctness and sandbox isolation | Model reasoning and operation risk | Plugin code quality and permissions |
| Target users | Everyday users | Scenarios needing batch operations | Restricted / high-security scenarios | Developers and platform teams |
4. Engineering Essentials of Core Plugins
These runtimes usually ship with several official plugins in the distribution package, which can be grouped into four categories by responsibility:
| Category | Plugins | Responsibility |
|---|---|---|
| Driving & input | Agent loop, voice input | Drive the main flow; speech-to-text |
| Execution | Terminal, web search | Execute commands; retrieve external information |
| Collaboration | Sub-agent, agent team | Isolate subtasks; multi-agent coordination |
Agent loop plugin: The loop is the runtime’s “heartbeat,” deciding when to call the model, when to execute tools, and when to end. Termination conditions should be defined explicitly, such as maximum steps, a cumulative token limit, a timeout, or goal achievement; mid-run intervention should also be preserved so humans can correct course.
Voice input plugin: Transcribes speech into text and hands it to the agent for processing. Its design focus is decoupling the transcription pipeline from the main loop, protecting privacy (whether audio is uploaded, whether transcripts are retained), and providing a fallback path when transcription fails.
Terminal plugin: The most powerful and highest-risk. It should enable whitelist-style command and path restrictions by default, disable or explicitly confirm dangerous operations, and limit output length.
Web search plugin: Result quality directly affects task success. Results should be cleaned and their sources labeled, timeliness should be considered, and citations should be preserved when summarizing so results can be verified.
Sub-agent and agent team plugins: The former creates an independent context for a single subtask, keeping the main session from being polluted by intermediate details; the latter supports multiple agents collaborating by role. Both must clearly define task boundaries, result return formats, and conflict-resolution rules, or redundant work—even mutual overwriting—can easily occur. The agent team is an experimental capability, and it is recommended to validate it on non-critical tasks first.
5. Implementation Practices: Security, Evaluation, and Observability
The sandbox is the first line of defense. Both PTC mode and the terminal tool mean “model-generated code/commands will be executed,” so the sandbox must provide filesystem isolation, network access control, CPU/memory/execution-time limits, and artifact cleanup.
Evaluation is the basis for mode selection. Build a small evaluation set that covers typical tasks, run Standard, PTC, and Minimal modes separately under the same model and prompts, and compare success rate, average steps, token consumption, and human-intervention rate before deciding the default mode. The evaluation set should be regression-tested in sync with plugin updates.
Observability determines operations quality. Record a complete trace for each session: which plugins were called, input/output summaries of each tool call, time taken, and failure reasons. Traces should be exportable and searchable to make troubleshooting easier.
Cloud deployment can reuse general-purpose capabilities. Runtime services can be containerized; sessions and artifacts can be stored in object storage; task scheduling can be delegated to message queues; runtime logs can connect to log services, and key metrics to monitoring and alerting. On Baidu AI Cloud, such an environment can be built and operated using general-purpose cloud products such as containers, object storage, message queues, log services, and monitoring and alerting.
Keep prompts and tool descriptions under version control. Model behavior depends heavily on the wording of tool descriptions and system prompts, so both should be reviewed, changed, and rolled back like code—avoiding “changing one sentence and causing behavior drift.”
6. Summary
A plugin-based design is a key step in engineering agent runtimes. Through “everything is a plugin,” it breaks the coupling of capability implementation so that models, tools, loops, sandboxes, and interfaces can all be replaced independently. On top of this, the four working modes—Standard, PTC, Minimal, and Create—provide a continuous spectrum from “out of the box” to “deep customization,” while core plugins cover the four core needs of driving, input, execution, and collaboration. For developers, what really sets a system’s ceiling is not the number of plugins but the rigor of the plugin contract, the tightness of sandbox isolation, and the completeness of the evaluation system. Understanding these three points is what turns an agent from a demo into engineering infrastructure that can evolve sustainably.
