LLM Agent Runtime Architecture: A Plugin-Based Approach

    Modern agent runtimes are moving toward a plugin-based architecture, treating nearly every capability—models, tools, skills, sessions, storage, sandboxes, agent loops, scheduling, and interfaces—as a pluggable component. Drawing on an MIT-licensed open-source framework as an example, this article explains the core design behind this approach: unified plugin contracts, lifecycle management, and dependency injection through a plugin container. It then walks through the four working modes such runtimes offer—Standard, PTC (programmatic tool calling), Minimal, and Create—along with the engineering essentials of core plugins and practical guidance on security, evaluation, and observability. The key takeaway for developers: a runtime's real ceiling is set not by how many plugins it has, but by the rigor of its plugin contract, the strength of its sandbox isolation, and the completeness of its evaluation system.

    1. Background: Why Agent Runtimes Are Moving Toward Plugins

    Over the past couple of years, agents have grown from experimental chat demos into engineering tools that can read code, execute commands, retrieve information, and operate systems. A complete agent system typically has four core parts: the LLM that handles reasoning and decisions, the tools that execute actions, the agent loop that drives the repeated “reason—act—observe” cycle, and the underlying infrastructure that manages session context, persistent state, and isolated execution environments.

    Most teams’ first implementation puts all of these in the same codebase. That is the most efficient approach during prototyping, but problems surface once tasks become complex:

    • High change costs: Adding an external tool often means modifying the loop logic or the main flow. The tighter the coupling, the larger the regression scope.
    • Hard to replace capabilities: Comparing different models, storage backends, or execution environments usually means copying the code and starting a new branch.
    • Blurred permission boundaries: Sensitive capabilities like terminals, files, and networks are scattered across business code, making unified least-privilege control difficult.
    • Difficult scenario adaptation: Batch jobs don’t need a complex interface, and security-audit tasks allow only a minimal toolset, but a monolithic design struggles to “trim on demand.”

    To solve all of these at once, the runtime needs a stable extension layer: one that decouples each capability implementation from the core loop and turns it into a plugin that is declarative, registrable, replaceable, and observable. This is the fundamental reason many agent runtimes in the open-source community have chosen a plugin-based architecture in recent years. This article analyzes an MIT-licensed open-source, plugin-based agent runtime framework, abstracts its architectural approach and engineering practices, and illustrates them with example code.

    2. “Everything Is a Plugin”: Architecture Essence and Key Design

    2.1 Nine Pluggable Capability Domains

    The most notable feature of these frameworks is that “everything is a plugin.” Beyond the intuitive elements like models, tools, and skills, sessions, storage, sandboxes, agent loops, scheduling, and even interfaces are all provided as plugins. These can be grouped into nine categories:

    Capability domain Responsibility Typical reason to replace
    Model Provides reasoning and generation, with support for multi-model routing Choose different models per task, balancing quality, latency, and cost
    Tool Executes external actions (commands, retrieval, API calls) Add new capabilities, or tighten/replace sensitive tools
    Skill Encapsulates reusable operational experience and workflow templates Turn domain knowledge into shareable assets
    Session Manages multi-turn context and state Control context length and compression strategy
    Storage Persists session records and intermediate artifacts Switch between local and cloud storage
    Sandbox Isolates the execution environment and limits system resources Switch between development and restricted environments
    Agent loop Drives the main “reason—act—observe” flow Change the task strategy, or adjust termination and budget conditions
    Scheduling Manages task concurrency and execution order Support batch jobs and multi-agent collaboration
    Interface Carries interaction forms (CLI, web, GUI desktop, etc.) Provide different entry points for different users

    The core value of this design is not the number of plugins but the separation of “capability implementation” from “loop logic.” The runtime kernel keeps only a relatively stable skeleton, and specific capabilities are assembled by plugins at startup or runtime. The result: swapping a model requires no loop-code changes, adding a tool only requires registering a new plugin, and trimming capabilities only requires adjusting the plugin manifest.

    2.2 Plugin Contracts: Declaration, Lifecycle, and Extension Points

    To let any plugin integrate safely, there must be a unified plugin contract containing at least:

    • Metadata and configuration: name, version, capability description, parameter definitions, and their validation rules.
    • Lifecycle: initialize, enable, disable, destroy. Resources should be released on disable, and state should be cleaned up on destroy.
    • Extension points: for example, interception hooks before and after tool calls, usable for unified permission checks and audit logging.
    • Dependency declarations: explicitly declare the other plugins a plugin depends on, avoiding implicit coupling.

    Example code (only to illustrate the form of the contract):

    1. # Tool plugin example (Python-style pseudocode)
    2. @tool(
    3. name="repo_search",
    4. description="Search for code snippets by keyword in a code repository",
    5. config_schema=SearchConfig, # Parameter declaration and validation
    6. permission="read_only", # Capability declaration, checked by the permission layer
    7. )
    8. class RepoSearchTool:
    9. def __init__(self, context):
    10. self.context = context # Runtime context injected by the plugin container
    11. def run(self, keyword: str, max_results: int = 10) -> list[dict]:
    12. # Read plugin configuration instead of hardcoding parameters
    13. cfg = self.context.config("repo_search")
    14. # Use the capability provided by the storage plugin rather than touching the filesystem directly
    15. repo = self.context.get("storage").repository(cfg.repo_path)
    16. return repo.search(keyword, limit=max_results)

    This reflects two principles of plugin-based design: externalized configuration and capabilities obtained through the plugin container. Together, they ensure plugin portability.

    2.3 Plugin Container and Dependency Injection

    Bringing a plugin architecture to life requires a “plugin container.” It handles plugin discovery and registration, instantiation, configuration injection, lifecycle management, and error isolation. Taking one plugin as an example: when the container creates it, it passes a unified runtime context into the constructor, and the plugin then obtains configuration, calls other plugins, and registers extension points through that context. This pattern is called dependency injection, and it brings three benefits:

    • Plugins don’t need to know the specific implementation of other plugins, and can be swapped for mock implementations during testing;
    • Different implementations of the same capability can coexist, with configuration deciding which one is enabled;
    • When problems occur at runtime, they can be traced to a specific plugin rather than to a whole block of main-flow code.

    If a plugin contains generic logic such as error handling, parameter validation, logging instrumentation, and permission declarations, consider splitting it into several smaller plugins to be used in combination, rather than making a single plugin shoulder all responsibilities. After splitting, each plugin can be tested and evolved independently.

    3. Four Working Modes: Different Forms of Capability Orchestration

    To fit needs ranging from everyday use to deep customization, these desktop runtimes offer agents four working modes.

    3.1 Standard Mode: The Default Complete Workflow

    Standard mode is the default option. The agent can select and call tools from the full toolset as needed, and the loop advances across multiple “reason—act—observe” cycles.

    It suits interactive scenarios such as day-to-day development assistance, research, and document organization. Its advantages are broad capability coverage and a low learning curve. One thing to watch is context growth: in long tasks, observation results accumulate continuously, so truncation and compression should be handled together with the session plugin.

    3.2 PTC Mode: Let the Model “Write a Program” to Call Tools

    PTC mode (Programmatic Tool Calling) changes the granularity of tool calls. Instead of calling tools one by one, the model first writes a program; the runtime executes that program in a controlled environment, and the program itself organizes and calls tools in batch, in loops, and concurrently.

    An analogy: Standard mode is like a person clicking buttons one at a time, pausing after each click to see the result before deciding the next step; PTC mode is like writing the whole sequence of operations into a script and running it all at once.

    1. # PTC mode example: a "tool orchestration program" generated by the model (pseudocode, API names are illustrative)
    2. # 1. List the files to process
    3. files = tools.list_files(path="src", suffix=".py")
    4. targets = [f for f in files if "deprecated" in tools.read_file(f)]
    5. # 2. Read in parallel and locate the positions that need modification (tool calls happen in batch inside the program)
    6. tasks = [(f, "TODO_FIXME") for f in targets]
    7. hits = tools.parallel_search(tasks, concurrency=8)
    8. # 3. Generate patches by rule and write them in
    9. for file, matches in hits.items():
    10. patch = build_patch(matches)
    11. tools.apply_patch(path=file, patch=patch)
    12. # 4. Summarize execution results and return only a summary to the model
    13. summary = {"scanned": len(files), "modified": len(hits)}
    14. tools.write_report("report.json", summary)
    15. print(summary)

    The benefits of this approach come in three main areas:

    1. Reduced round-trip overhead: Calling tools one by one might require dozens of round-trips between the model and the tools for a few dozen files; writing them into a program needs only one generation and one execution.
    2. Context savings: Intermediate results flow within the program, and only a summary is returned, preventing the context from being flooded by large amounts of intermediate data.
    3. Programmatic capability: Logic such as loops, conditionals, exception handling, concurrency, sorting, and aggregation can be expressed directly, instead of forcing the model to do it “manually” over and over in the conversation.

    Its risks are equally clear: the correctness of generated programs depends on validation and testing inside the sandbox; execution timeouts and resource limits must be set to prevent infinite loops or resource exhaustion; and strict permission control is needed whenever scripts involve credentials and network access.

    3.3 Minimal Mode: Terminal Tool Only

    Minimal mode provides the agent with only the terminal tool, with all other capabilities disabled. The model can rely only on the command line to observe the environment, perform operations, and verify results on its own.

    It suits two types of scenarios: first, security-sensitive or resource-constrained environments where fewer tools mean a smaller attack surface; second, as a control baseline for evaluating a model’s independent reasoning and self-checking ability. Two points need attention:

    • Command output should be truncated and structured appropriately to avoid massive logs instantly filling the context;
    • It must run in a sandbox with limits on the working directory, allowed executables, and network access.

    3.4 Create Mode: Writing Plugins and Custom Modes

    Create mode targets developers, allowing users to write plugins that extend the runtime’s capabilities and define custom working modes. For example, you can add query tools for specific data sources, encapsulate internal approval workflows, or change the agent loop’s advancement strategy. It is the key entry point for upgrading the runtime from “using tools” to “building tools,” but it must be paired with strict plugin review, permission declarations, and version management to keep uncontrolled code out of the production pipeline.

    3.5 Comparison of the Four Modes

    Dimension Standard PTC Minimal Create
    Tool scope Full toolset Full toolset (mainly invoked via programs) Terminal only Depends on custom plugins
    Calling method Model calls one by one Model generates a program, then batch calls Model executes commands directly Determined by custom logic
    Context pressure Higher Lower (only a summary is returned) Medium (command output can be large) Depends on design
    Main risk Context bloat Program correctness and sandbox isolation Model reasoning and operation risk Plugin code quality and permissions
    Target users Everyday users Scenarios needing batch operations Restricted / high-security scenarios Developers and platform teams

    4. Engineering Essentials of Core Plugins

    These runtimes usually ship with several official plugins in the distribution package, which can be grouped into four categories by responsibility:

    Category Plugins Responsibility
    Driving & input Agent loop, voice input Drive the main flow; speech-to-text
    Execution Terminal, web search Execute commands; retrieve external information
    Collaboration Sub-agent, agent team Isolate subtasks; multi-agent coordination

    Agent loop plugin: The loop is the runtime’s “heartbeat,” deciding when to call the model, when to execute tools, and when to end. Termination conditions should be defined explicitly, such as maximum steps, a cumulative token limit, a timeout, or goal achievement; mid-run intervention should also be preserved so humans can correct course.

    Voice input plugin: Transcribes speech into text and hands it to the agent for processing. Its design focus is decoupling the transcription pipeline from the main loop, protecting privacy (whether audio is uploaded, whether transcripts are retained), and providing a fallback path when transcription fails.

    Terminal plugin: The most powerful and highest-risk. It should enable whitelist-style command and path restrictions by default, disable or explicitly confirm dangerous operations, and limit output length.

    Web search plugin: Result quality directly affects task success. Results should be cleaned and their sources labeled, timeliness should be considered, and citations should be preserved when summarizing so results can be verified.

    Sub-agent and agent team plugins: The former creates an independent context for a single subtask, keeping the main session from being polluted by intermediate details; the latter supports multiple agents collaborating by role. Both must clearly define task boundaries, result return formats, and conflict-resolution rules, or redundant work—even mutual overwriting—can easily occur. The agent team is an experimental capability, and it is recommended to validate it on non-critical tasks first.

    5. Implementation Practices: Security, Evaluation, and Observability

    The sandbox is the first line of defense. Both PTC mode and the terminal tool mean “model-generated code/commands will be executed,” so the sandbox must provide filesystem isolation, network access control, CPU/memory/execution-time limits, and artifact cleanup.

    Evaluation is the basis for mode selection. Build a small evaluation set that covers typical tasks, run Standard, PTC, and Minimal modes separately under the same model and prompts, and compare success rate, average steps, token consumption, and human-intervention rate before deciding the default mode. The evaluation set should be regression-tested in sync with plugin updates.

    Observability determines operations quality. Record a complete trace for each session: which plugins were called, input/output summaries of each tool call, time taken, and failure reasons. Traces should be exportable and searchable to make troubleshooting easier.

    Cloud deployment can reuse general-purpose capabilities. Runtime services can be containerized; sessions and artifacts can be stored in object storage; task scheduling can be delegated to message queues; runtime logs can connect to log services, and key metrics to monitoring and alerting. On Baidu AI Cloud, such an environment can be built and operated using general-purpose cloud products such as containers, object storage, message queues, log services, and monitoring and alerting.

    Keep prompts and tool descriptions under version control. Model behavior depends heavily on the wording of tool descriptions and system prompts, so both should be reviewed, changed, and rolled back like code—avoiding “changing one sentence and causing behavior drift.”

    6. Summary

    A plugin-based design is a key step in engineering agent runtimes. Through “everything is a plugin,” it breaks the coupling of capability implementation so that models, tools, loops, sandboxes, and interfaces can all be replaced independently. On top of this, the four working modes—Standard, PTC, Minimal, and Create—provide a continuous spectrum from “out of the box” to “deep customization,” while core plugins cover the four core needs of driving, input, execution, and collaboration. For developers, what really sets a system’s ceiling is not the number of plugins but the rigor of the plugin contract, the tightness of sandbox isolation, and the completeness of the evaluation system. Understanding these three points is what turns an agent from a demo into engineering infrastructure that can evolve sustainably.