From Open Weights to Managed Agents: A New Paradigm for Deploying AI Agents

1. Two Hurdles for Agent Deployment: Model Capability and Engineering Runtime

Over the past two years, the focus of large language model (LLM) applications has shifted from “chatting” to “working,” making AI Agents the recognized technological mainstream. However, teams with practical deployment experience know that a stable, business-ready Agent is far more complex than just “connecting to a model API.” It must overcome at least two major hurdles.

The first is the model capability hurdle. Agent tasks require models to possess long-term planning, multi-turn tool invocation, and self-correction based on environmental feedback—capabilities fundamentally different from single-turn Q&A. A model that performs brilliantly on conversational leaderboards may not reliably complete a multi-step task chain, as errors accumulate and amplify over long execution cycles.

The second is the engineering runtime hurdle. Beyond the model itself, a complete execution framework is required to handle task decomposition, tool invocation, context management, exception retries, sandbox isolation, and log auditing. Building this “runtime” in-house is costly and heavy to maintain, making it the biggest obstacle for enterprises deploying Agents.

Two recent industry trends directly address these hurdles. First, open-weight models are continuously approaching or even surpassing some closed-source models in agent evaluations, making open source the main competitive frontier. Second, the rise of “managed agents” services provides execution frameworks directly as cloud services, allowing enterprises to obtain a 24/7 managed environment without building it themselves. This article explores these two key trends.

2. Open Weight Models: Why Open Source is the New Competitive Frontier

First, let’s clarify the concept. “Open weight” means that model developers publish the model weight files in open-source communities, allowing users to download, locally deploy, and fine-tune them. However, the training data and training code are typically not open. This differs from strict “fully open source” but has become the mainstream compromise in current industry collaboration—it preserves the commercial boundaries for developers while providing users with autonomous control.

For enterprises, the value of open weights is reflected in at least four aspects:

• Data Sovereignty: Weights can be deployed in private environments, eliminating the need to send sensitive data externally.
• Deep Customization: Fine-tuning based on business data to create domain-specific models.
• Auditability: Model behavior is reproducible and can be analyzed offline, meeting compliance requirements.
• Avoiding Vendor Lock-in: Multiple models can be swapped out, reducing the risk of supply lock-in.

Meanwhile, cloud providers generally offer model hosting and fine-tuning services. Enterprises can deploy open-weight models to cloud inference platforms, balancing private capabilities with elastic scaling. Platforms like Baidu AI Cloud also provide general capabilities from model services to agent development, serving as a comparison option during technical selection.

3. Agent Evaluation: How to Scientifically Measure an Agent’s Quality

Traditional static benchmarks measure models using fixed questions, which suffer from saturation, susceptibility to “gaming the leaderboard,” and weak correlation with production performance. Therefore, third-party evaluation platforms are gradually shifting toward dynamic evaluations featuring “real users, long cycles, and real task sessions”: global users submit real work tasks, the system executes them in a controlled environment, and scoring incorporates user feedback. These evaluations typically involve tens of thousands of sessions, accurately reflecting true production performance.

Key metrics can be categorized into five types:

1. Net Improvement Rate: The net value after weighting positive and negative user feedback, measuring whether “the new generation is genuinely better than the previous one.” In a public evaluation, for instance, a new generation open-weight model achieved a net improvement rate of over +5%, compared to only about +1% for the previous generation, showing significant generational progress.
2. Command Execution Reliability: The first-time success rate of an agent executing Shell commands, directly reflecting engineering robustness. In an example, this metric jumped from around -4.5% in the previous generation to +7.9%—for Ops and Dev Agents, “daring to execute and executing correctly” is a hard metric.
3. Controllability: The model’s ability to adjust its execution direction promptly based on user feedback. An Agent is not a one-shot deal; whether it can understand corrective instructions like “don’t touch that file for now, check the logs” determines the collaboration experience.
4. Confirmation Success Rate: The proportion of users explicitly confirming “task completed,” measuring end-to-end delivery quality.
5. Satisfaction (Praise vs. Complaint Ratio): A quality metric synthesizing user sentiment signals.

These five metrics point to one conclusion: the core of agent evaluation is not “how much it knows,” but “how much it gets done and how stably.” Enterprises building in-house evaluations should also prioritize real task replays and user feedback loops over static scores.

4. Managed Agents: Turning Execution Frameworks into Cloud Services

An agent system can be abstracted into four layers: model, execution framework, tools, and environment. The essence of a managed agent is to servitize the execution framework and environment layers:

• 24/7 Managed Runtime: No need to build and maintain frameworks yourself; the platform handles availability and scaling.
• Long-Horizon Task Support: Tasks can run across hours or days, with resumable states.
• Multi-Agent Collaboration: Multiple agents can divide tasks and cooperate to complete complex workflows.

Architecturally, a typical managed agent service can be divided into five layers:

1. Access Layer: Connects to internal systems via API, Webhook, or SDK, and can also be embedded into external user-facing products.
2. Orchestration Layer: Responsible for task planning, session state management, and multi-agent scheduling.
3. Execution Layer: A sandboxed runtime environment hosting code execution and tool invocations.
4. Tool Layer: Pre-built skills and plugins, supporting the integration of enterprise-owned tools and data sources.
5. Governance Layer: Permissions, auditing, quotas, reporting, and security hooks.

A schematic configuration (pseudo-code, illustrating structure only, not a real API):

agent:
  name: ops-watchdog
  runtime:
    sandbox: isolated            # Independent sandbox
    network:
      egress: allowlist          # Outbound allowlist
  tools:
    - skill: web_search          # Pre-built skill
    - plugin: internal_cmdb      # Enterprise-owned tool
  guardrails:
    pre_inference_check: true    # Check input and tool returns before inference
    sensitive_action: block      # Block if rules are triggered
  observability:
    audit: enabled               # Full trace logging
    cost_report: per_agent       # Usage reports per project

Comparison between in-house and managed approaches:

Dimension	In-House Runtime	Managed Agent
Initial Cost	High (framework dev, environment setup)	Low (configuration ready)
Maintenance Burden	Self-managed upgrades, patches, scaling	Platform responsible
Security Governance	Self-built sandbox and audit system	Built-in sandbox, allowlist, audit
Control Level	Fully autonomous, deeply customizable	Constrained by platform capabilities
Suitable Teams	Large teams with platform engineering capabilities	Rapid prototyping and SMB teams

5. Security and Governance: Baseline Design for Enterprise-Grade Agents

Because agents have the ability to “execute actions,” security design must upgrade from “preventing data leaks” to “preventing misoperations, preventing privilege escalation, and ensuring accountability.” Managed agents typically provide four categories of capabilities in the governance layer:

1. Isolation: Each agent runs in an independent sandbox with isolated file systems, processes, and networks to prevent lateral impact.
2. Permission Boundaries: Outbound network allowlists and operation permission declarations define what an agent “can access and do.” Sessions can be immediately terminated or handed over to humans if necessary.
3. Traceability and Auditing: The entire execution process supports trace querying and audit exports, meeting post-mortem tracking and compliance checks.
4. Pre-Inference Checks: Before model inference, task inputs and tool return contents are checked against enterprise-defined rules to block privacy and sensitive data—proactive interception prevents risks before they occur.

A “defense in depth” approach is critical: pre-checks intercept high-risk content, sandboxes and permissions limit the blast radius, trace logging ensures accountability, and usage reports keep administrators informed about resource consumption and costs across agent projects. Together, these form a complete governance loop for enterprise-grade agents.

6. Tool Ecosystem: Layered Design of Pre-built Skills and Custom Tools

Tools determine an agent’s capability ceiling. Mainstream managed services generally adopt a layered ecosystem of “pre-built skills + plugins + custom tools”:

• Pre-built Skills: Cover high-frequency tasks like web search, professional data queries, office document processing, and content generation. There can be hundreds available out-of-the-box.
• Plugin System: Dozens of plugins connecting to common external systems and data services.
• Custom Tool Integration: Enterprises encapsulate internal APIs, databases, and scripts as standard tools (providing name, description, parameter schema, and auth info). The agent can then invoke them just like pre-built skills.

Schematic tool definition (pseudo-code):

{
  "name": "query_internal_cmdb",
  "description": "Query service asset information in the internal configuration management database",
  "parameters": {
    "type": "object",
    "properties": {
      "service_name": { "type": "string" }
    },
    "required": ["service_name"]
  },
  "auth": "internal_token"
}

The significance of this layering is that general capabilities reuse the platform ecosystem, while differentiated capabilities are supplemented by enterprise-owned tools, avoiding reinventing the wheel.

7. Typical Deployment Scenarios and Selection Guidelines

Managed agent scenarios can be divided into internal and external categories:

• Internal: Business monitoring and alert triage, Ops response, data auditing, internal knowledge Q&A. These are characterized by heavy internal system calls and high requirements for security boundaries and auditing.
• External: Investment research assistants, customer service bots, content generation assistants. These directly serve end-users and demand high stability and compliance reviews.

A selection checklist:

1. Evaluate First: Conduct a POC with real business tasks, focusing on command execution reliability and controllability, rather than just leaderboard rankings.
2. Security and Compliance: Verify that sandbox isolation, network allowlists, audit exports, and pre-inference content checks are complete.
3. Cost Transparency: Require per-project, per-agent usage reports and cost estimation tools to avoid black-box billing.
4. Evolution Path: Start with official documentation and templates, run a single scenario through, then integrate custom tools, and finally introduce multi-agent collaboration.
5. Ecosystem Compatibility: Prioritize solutions that support open-weight models and standardized tool protocols to preserve migration capabilities.

8. Conclusion

Open-weight models solve the problem of “where the model comes from,” while managed agents solve “how the agent runs.” As evaluation systems shift from static scores to real task sessions, and execution frameworks move from in-house builds to cloud services, the path for enterprises to deploy agents has become clearer than ever: choose a model validated on real tasks, place it in a secure, auditable, well-tooled managed environment, and start small with a measurable business scenario. The second half of the AI agent competition will be defined by engineering and governance capabilities, not just model parameters.