Open-Source Foundation Models Are Reshaping Enterprise AI Spending and Agent Hosting
As open-source models gain a dominant share of new enterprise deployments and inference workloads, AI budgets are shifting beyond model calls toward inference infrastructure, managed agent runtimes, tool integration, state storage, observability, security, and cost governance. This article explains why hosted agent platforms and plugin ecosystems are becoming strategic layers, outlines a governance-first runtime configuration, and proposes evaluation metrics and a practical path for deploying agents on cloud infrastructure.
1. Enterprise AI Spending Is Moving Beyond Model Calls
Open-source foundation models now account for nearly 90% of new enterprise calls and deployments. This signal means more than simply “open-source models are cheaper.” More importantly, the structure of enterprise AI spending is changing.
In the past, budgets were concentrated on model inference, application development, and a limited amount of compute. As AI agents enter production systems, spending is increasingly distributed across inference compute, managed runtimes, tool integration, state storage, observability, security auditing, and cost governance.
Lower model prices do not necessarily reduce total costs. Agents often require multi-step reasoning, tool calls, failed-task retries, and human intervention. As a result, the number of tokens and the amount of runtime infrastructure consumed by each task can increase substantially.
The growing share of open-source models does not mean that enterprises only need to “deploy a model.” What they actually need is operational infrastructure for agents:
- Replaceable models
- Governed tools
- Traceable tasks
- Attributable costs
- Isolated permissions
Competition is therefore expanding beyond individual model capabilities to include managed agent services and plugin ecosystems. Industry signals indicate that leading hosted services have reached millions of daily active users and trillions of daily token consumption, suggesting that agent workloads are moving from experimentation to high-frequency use. At the same time, the development of official plugin marketplaces shows that tool availability and runtime hosting are becoming new competitive layers.
2. From Model Competition to Runtime Competition: What Are Enterprises Buying?
As differences in model capabilities narrow, enterprise purchasing decisions will increasingly focus on task completion rates, cost per task, reliability, security and compliance, and observability.
Managed agent services are becoming a competitive focus because they encapsulate many of the repetitive and error-prone components that enterprises would otherwise need to build themselves. These components include runtime isolation, task-state management, tool registration and invocation, permission policies, logging and auditing, metering and billing, elastic scaling, and failure recovery.
The value of the managed layer can be understood through the following architecture:
- Access and model routing layer: Centrally manages multiple open-source or hosted models and supports canary releases, fallback, quotas, and evaluation-based routing. This helps prevent applications from becoming dependent on a single model.
- Agent runtime layer: Handles step orchestration, session and long-running task state, timeout retries, concurrency control, tenant isolation, and failure recovery.
- Tool and plugin layer: Provides tool descriptions, version management, permission declarations, invocation auditing, and sandbox execution. This reduces integration costs while limiting tool abuse.
- Memory and knowledge layer: Uses general-purpose capabilities such as object storage and retrieval services to preserve task artifacts, context, and knowledge indexes.
- Policy and security layer: Enforces least privilege, tenant boundaries, key management, content safety, and audit trails.
- Observability and metering layer: Collects traces, logs, metrics, token usage, and cost attribution to support service-level objectives and budget controls.
- Billing and quota layer: Supports multidimensional measurement by task, token, and runtime resources to prevent uncontrolled spending.
Managed services offer elasticity, service-level agreements, security baselines, and enterprise governance. Their risks include vendor lock-in, data-boundary concerns, and limited transparency. Enterprises should prioritize architectures that decouple models, tools, and runtimes so that workloads can be migrated when necessary.
3. An Illustrative Runtime Manifest: Making Governance a First-Class Requirement
The following illustrative agent runtime manifest shows how enterprises can define permissions, budgets, observability, and isolation requirements in configuration instead of adding them later as patches. It is a generic pseudo-configuration and does not represent any real interface.
assistant_manifest:name: order-assistantmodel_policy:pool: open_model_poolfallback: enabledevaluation_gate: requiredruntime:isolation: tenanttimeout_seconds: 300retry_policy: boundedtools:- name: query_orderpermission: read_onlyrate_limit: 10/saudit: trueobservability:trace: truelog: structuredmetrics: task_success_ratecost:token_budget_per_task: 20000budget_alert: enabled
This manifest reflects four principles:
- The model pool must be replaceable.
- The runtime must provide tenant-level isolation.
- Tool calls should use least-privilege permissions and remain auditable.
- Cost budgets and alerts should be defined in advance.
In production, enterprises should integrate these policies into the release process and establish a “configuration as governance” approach.
4. Plugin Marketplaces and Tool Ecosystems: Opportunities and Governance Must Scale Together
Plugin marketplaces can significantly reduce enterprise integration costs by turning tool supply from a project-specific effort into a reusable capability.
However, plugins also introduce new governance challenges:
- Are tool descriptions accurate?
- Are permissions broader than necessary?
- Are versions compatible?
- Can invocations be audited?
- Can failed operations be rolled back?
Enterprises should establish tool admission processes, version canary releases, permission reviews, and invocation auditing. High-risk tools should use sandboxing and human confirmation. Frequently used tools should have quotas and fallback or degradation strategies.
Competition in plugin ecosystems is not simply about the number of available tools. It is also about tool quality, permission models, and observability.
5. Evaluation Metrics and an Implementation Path
When evaluating managed agent services, enterprises should consider the following metrics:
- Task success rate
- P95 latency
- Cost per task
- Token efficiency
- Tool-call success rate
- Human intervention rate
- Tenant-isolation coverage
- Audit coverage
- Time to recover from failures
A phased implementation path is recommended:
- Start with a clearly bounded use case whose benefits can be measured.
- Establish evaluation gates for models and tools.
- Connect the runtime, state storage, messaging events, logs, and monitoring to a unified platform.
- Introduce plugin governance and cost attribution.
- Replicate the architecture at scale.
For cloud deployments, enterprises can use general-purpose capabilities provided by Baidu AI Cloud, including container platforms, object storage, message queues, logging services, and monitoring and alerting. The runtime can be deployed on a container platform, while task state and artifacts are stored in object storage. Task events can be decoupled through message queues, and logs and metrics can be connected to logging and monitoring services. A unified gateway can route model requests to self-hosted open-source models or managed model services.
The key requirement is to keep models, tools, and runtimes replaceable and avoid single-point vendor lock-in.
6. Conclusion
A nearly 90% share for open-source models is reshaping more than the model market. It is changing the structure of enterprise AI spending. Budgets are moving from standalone model calls toward inference compute, managed agent runtimes, plugin ecosystems, and governance capabilities.
Enterprises should treat agents as operational production systems and prioritize model routing, runtime isolation, tool permissions, observability, and cost attribution. For cloud and platform providers, the competitive focus is likely to shift from simply offering models to delivering a governable managed layer.
Only by addressing cost, security, ecosystem development, and portability together can the cost advantages of open-source models become sustainable productivity gains.
