From Open Weights to Managed Agents: A Practical Guide to Enterprise Agent Deployment

1. Background: Open Weights Solve Availability, Managed Services Solve Usability

In recent years, leading model teams have progressively released model weights to the public. Enterprises can now legally obtain weights, deploy them in private environments, and fine-tune them with proprietary business data. For enterprises, open weights deliver value in four key areas:

• Autonomy and control: Models run in your own environment, and sensitive data never leaves the domain, satisfying compliance requirements in finance, government, and healthcare.
• Cost structure optimization: For high-frequency, long-running workloads, the marginal cost of a self-hosted inference cluster can be lower than per-token API calls.
• Deep customization: Continued training and preference alignment on the base model allow domain knowledge to become proprietary assets.
• Reduced lock-in risk: With weights in hand, business operations are not immediately disrupted even if upstream service policies change.

However, open weights only solve the “do we have a model” question. An agent that actually performs work requires more than the model itself—it needs an entire runtime framework: task planning loops, tool call orchestration, context and memory management, failure retry, result verification, and log observability. This engineering layer beyond the model can be collectively described as “execution scaffolding.” Building this system from scratch is not trivial—sandbox isolation, permission control, and audit trails are each standalone engineering challenges.

This is where “managed agent services” emerge as a new model: cloud platforms package the runtime framework directly as a service. Developers only need to define the agent’s goals, tools, and boundaries, and they receive a 7×24 managed execution environment with native support for long-horizon task execution and multi-agent collaboration. Once model capability and engineering capability are decoupled, enterprises can focus on business logic itself.

2. Understanding Agent Evaluation: What to Look at Beyond Leaderboards

2.1 Evaluation Methods Are Changing: From Static Benchmarks to Real Tasks

Unlike traditional static benchmarks, agent evaluation increasingly relies on real user long-horizon task sessions. Users come with real needs, the model iteratively calls tools, executes commands, and retrieves information, and ultimately the user verifies the delivered results. Evaluation teams typically compute a “net improvement rate” based on large volumes of real sessions—the net difference between wins and losses against a reference model. This approach filters out “good at exams but bad at real work” models and more closely reflects production performance.

2.2 Four Key Metric Dimensions

1. Command execution reliability: The accuracy and error-recovery ability of the agent when executing commands in a terminal/shell environment. Much of an agent’s real work happens in code and command-line environments. A single mistaken deletion or incorrect dependency installation can destroy the execution environment. This metric therefore reflects engineering usability more than “can it write elegant code.”
2. Positive-to-negative feedback ratio (user satisfaction): The net value of user praise versus complaints across sessions, reflecting end-to-end experience quality.
3. Steerability: Whether the model can promptly adjust its execution direction when the user corrects course mid-task. An agent with poor steerability charges ahead on the wrong path, ignoring multiple corrections—a fatal flaw in long-horizon tasks.
4. Confirmation success rate: The proportion of tasks explicitly confirmed as complete by users—a hard metric directly tied to delivery quality.

2.3 Enterprises Should Build Their Own Evaluation System

• Extract tasks from real business operations to build an internal evaluation set, covering high-frequency scenarios rather than showcase scenarios.
• Use gray-release comparison to compute net improvement rates instead of relying on single-run scores.
• Split metrics by task type—for example, “operations monitoring” separated from “report generation”—to avoid averages masking weak spots.

3. Layered Architecture of Managed Agent Services

3.1 Three Layers at a Glance

• Control plane: Agent definitions and version management, permission and quota configuration, template reuse.
• Execution plane: Session scheduling, sandboxed execution environments, tool gateway, long-horizon task queues, multi-agent collaboration orchestration.
• Observability plane: Full-trace logging, metric monitoring, usage reports, and cost allocation.

3.2 Self-Built vs. Managed: The Trade-Off

The advantage of self-building is complete control and deep customization. The advantage of managed services is zero-ops, elastic scaling, and out-of-the-box security and auditing capabilities. The most practical path for most teams: use managed services during the validation phase for rapid experimentation, then evaluate partial migration to self-hosted infrastructure at scale. If a team does not plan to build from scratch, they can also combine general-purpose cloud platform capabilities—for example, Baidu AI Cloud’s model services, container platform, logging, and monitoring products—to assemble an equivalent architecture.

3.3 Agent Definition Example

agent:
  name: ops-watchdog
  runtime:
    sandbox: isolated          # Independent sandbox
    network_policy: allowlist  # Outbound allowlist
    max_session_minutes: 120   # Max session duration
  tools:
    - log_query
    - metric_alert
    - ticket_create
  guardrails:
    pre_inference_check: data-leak-filter  # Content check before inference
    human_approval: destructive_ops        # Human approval for destructive operations
  audit:
    trace: enabled
    export: standard_format

This declarative definition can be placed under version control, supporting rollbacks and A/B experiments.

4. Security and Governance: Five Defenses for Enterprise Deployment

1. Isolated sandboxes: Each session runs in an isolated environment—file systems, process spaces, and networks are separated. Sessions are destroyed upon completion, preventing cross-session contamination and lateral movement.
2. Network access control: Outbound destinations are restricted via allowlists following the principle of least privilege. Agents cannot reach core internal systems by default.
3. Operation boundaries and session termination: High-risk operations like data deletion, external sends, and fund-related actions require manual confirmation. Administrators can terminate sessions at any time.
4. Audit trails: Full execution records are maintained, supporting conditional search and export, and integrating with existing enterprise compliance and risk-control systems.
5. Pre-inference content checks: Check hooks are inserted before model inference and after tool result returns, using enterprise-defined rules to identify and block private and sensitive data, preventing data exfiltration through the model.

The check hook logic is illustrated below (pseudocode):

function pre_inference_hook(task_input, tool_results):
    for item in [task_input] + tool_results:
        risk = dlp_scan(item)              # Sensitive data detection
        if risk.level >= HIGH:
            return Block(risk.type)        # Block directly
        if risk.level == MEDIUM:
            item.redact()                  # Redact and allow
    return Allow()

In addition, cost observability is also part of governance: project-level usage reports and cost allocation, combined with quota alerts, prevent runaway loop calls from causing bill explosions.

5. Tool Ecosystem and Business Integration

5.1 Built-In Skills and Plugins

Mature managed services typically include built-in skills and plugins covering web search, specialized data queries, office document processing, and content generation, ranging from dozens to hundreds. For developers, the focus should be on evaluating the output quality and scenario coverage of these skills, not the raw count.

5.2 Integrating Custom Tools and Data

Registering enterprise internal APIs and private data sources through a unified function-calling protocol is key to generating real business value from agents. It is recommended to define clear input/output schemas, timeout and retry policies, and error semantics for each tool, so the model can make reasonable decisions on failure rather than retrying indefinitely.

5.3 Typical Scenarios

• Internal-facing: Business monitoring and alert analysis, operations response and change execution, periodic report generation.
• External-facing: Investment research assistants, customer service assistants, document Q&A embedded in products.

6. From PoC to Production: A Six-Step Roadmap and Common Pitfalls

Recommended roadmap: Evaluation and selection → Minimal closed loop → Tool integration → Security hardening → Gray release → Continuous operations.

Common pitfalls:

1. Evaluation sets are disconnected from real task distributions—great scores in testing, failures in production.
2. Tool-returned content is not validated, leaving the system exposed to prompt injection.
3. Sandbox and network permissions are configured too broadly, enlarging the blast radius of incidents.
4. Lack of cost observability leads to runaway loop calls and uncontrolled bills.
5. Agent definitions are not versioned, making rollbacks and comparison experiments impossible.

7. Conclusion

Open weights have lowered the barrier to model acquisition to a historic low, and managed agent services have dramatically reduced the engineering barrier. What will separate enterprises from here on is the quality of their evaluation systems, the depth of their security governance, and the maturity of their business integration. Get these three things right, and agents can move from demos to production—from “usable” to “truly useful.”