Enterprise Spending on Open-Source LLMs: The Engineering Competition Behind Market Share
Recent industry data indicates that Chinese AI models account for nearly 90% of enterprise spending on open-source large language models. The top three vendors account for 61%, 15%, and 11%, respectively, while the overseas open-source model family with the highest share accounts for about 7%. This article clarifies that this figure refers to open-source model spending rather than total AI spending, and examines what enterprise buyers actually pay for: engineering delivery, inference architecture, cost governance, and selection risk. It provides practical guidance for comparing hosted APIs with self-hosted deployments, controlling costs, planning capacity, and managing model-selection risk.
A recent industry research report has disclosed that, in global enterprise spending on open-source large language models, Chinese AI models together account for nearly 90%. The top three vendors are at 61%, 15%, and 11%, accounting for about 87% in total, while the overseas open-source model family with the highest share accounts for only about 7%.
This kind of data is easily simplified into “who won.” For enterprise technology decision-makers, however, the more important question is another one: when global enterprises pay, what are they actually paying for? This article first clarifies the data scope, then analyzes engineering delivery, inference architecture, cost governance, and selection risk, and provides practical implementation guidance.
1. Clarify the metric first: 89% refers to open-source model spending, not total AI spending
To interpret this kind of data, it is necessary to define the boundary first. “89%” refers to the share of global enterprise spending on open-source models. It is not equal to total enterprise AI spending, nor is it equivalent to model download volume, developer count, or benchmark rankings. Closed-source frontier models, foundational software stacks, cloud ecosystems, and high-end training compute remain areas where other providers have long held advantages.
The key to understanding why this data matters is the word “spending”:
- Downloads are free, but production has a threshold: model weights can be downloaded for free, but production inference compute, hosting services, operations support, and supporting tools all require payment.
- Payment means entering production: when enterprises are willing to pay continuously, it usually means the model has entered a production or pre-production system, rather than remaining only in trials or demonstrations.
- Spending is more real than rankings: benchmark scores reflect model performance under specific test conditions; spending reflects the combined judgment of real business traffic on stability, cost, and delivery capability.
At the same time, this is still structural data for a specific segment. A more accurate description is: “In the open-source model segment, enterprise spending is highly concentrated among Chinese vendors.” Expanding this into a claim of “overall leadership” would lack data support and could mislead technology decisions.
2. A shift in procurement logic: from open-source brand to delivery certainty
Take an overseas open-source model family as an example. It was once regarded as a flagship open-source model, but its current spending share is only 7%. This gap shows that enterprises are no longer buying simply for the open-source brand or parameter scale, but are moving toward choices that are closer to deployment cost and delivery certainty. Two factors have driven this shift:
- API calls have become mainstream: the popularization of generative AI has made API calls the main access method. Models are embedded directly into business processes through hosted services, so enterprises do not need to build inference clusters themselves. A pay-as-you-go, elastic-scaling API usage model has become the main spending form.
- Evaluation dimensions have expanded: open weights allow enterprises to deploy models themselves and maintain control; however, self-deployment also brings responsibilities for operations, scaling, and high availability. Enterprises must weigh hosted API usage against self-deployment during selection.
Early open-source models were attractive mainly because they were free, downloadable, and community-driven. Today, the evaluation focus has shifted to the practical indicators shown in the table below:
| Evaluation dimension | Focus | Typical question |
|---|---|---|
| Unit token cost | Input, output, and cache-hit pricing | What will the monthly bill be at the same workload? |
| Inference stability | Latency distribution, error rate, rate-limiting strategy | Are time to first token and P99 latency controllable? |
| Private deployment | Weight availability, hardware compatibility, resource usage | Can it run independently in our own data center? |
| Toolchain compatibility | Framework APIs, function calling, structured output | How much effort is required to integrate with existing applications? |
| Version and compatibility | Iteration pace, deprecation policy, migration path | Will business operations be disrupted after an upgrade? |
| Long-term service capability | Technical support, SLA commitments, industry solutions | Can incidents be recovered quickly and issues resolved in time? |
It should be emphasized that 89% is a spending-amount metric, not a call-volume metric. This may mean that the average price per call for leading models is relatively high, for example because they offer longer context, stronger reasoning capability, or higher-value hosted services. For high-frequency, lightweight tasks, lower-priced models or smaller-parameter models may be more economical.
3. The victory of the open-source ecosystem is first a victory of engineering delivery
The rising share of China’s open-source large language models is not achieved by “open weights” alone. What enterprises truly need is a complete set of integration capabilities from model to application, including model weights, inference APIs, development frameworks, AI agent runtimes, tools and plugin systems, hosted services, and industry solutions. Open sourcing is only the first step to reduce trial-and-error costs. Whether a model can become a callable, operable, and scalable production tool is the decisive factor in where spending flows.
This trend is already visible in the industry chain: some leading vendors have begun providing AI agent runtimes as cloud-hosted services and continue to invest in tool calls, task orchestration, and session persistence. This marks a shift in competitive focus from “model capability” to “AI agent delivery,” and the reliability of AI agents depends heavily on the engineering system.
1. Inference layer: from “producing results” to “stably serving traffic”
Production inference services must solve four problems at the same time:
- Latency optimization: distinguish time to first token from total generation latency, and apply dynamic batching according to request characteristics.
- Cache reuse: reuse caches for common prefixes such as system prompts and long documents to reduce repeated computation.
- GPU memory management: key-value cache usage grows with context length and concurrency, so paging management and scheduling are required.
- Traffic scheduling: multi-instance load balancing, rate limiting by token usage, and timeout and circuit-breaker protection.
2. Tool layer: enabling models to reliably call external systems
In production, models rarely work alone. They usually need to retrieve knowledge bases, query databases, and call business APIs. The engineering focus includes:
- Standardizing tool definitions, with explicit schema constraints on input and output parameters;
- Enforcing permission checks and timeout controls for tool calls;
- Providing structured feedback for failed results so the model can correct itself;
- Introducing human confirmation for write operations involving funds or permissions.
3. Adaptation layer: avoid being locked to a single model
Regardless of which open-source model is used, it is recommended to add a unified adaptation layer between business applications and the model to isolate model differences and unify retries, fallback, and usage metering. The following example code only illustrates the design idea and does not represent any specific interface:
from dataclasses import dataclass# Unified request model: hides parameter differences across models@dataclassclass ChatRequest:model: str # logical model name; routing decides the actual backendmessages: list # standardized conversation messagesmax_tokens: int = 1024temperature: float = 0.7tools: list | None = None # standardized tool definitions# Unified response structure: supports metering, observability, and auditing@dataclassclass ChatResponse:content: strusage: dict # prompt_tokens / completion_tokens, etc.latency_ms: intmodel_version: str # records the actual service version for traceabilityclass InferenceAdapter:"""Business code depends only on this interface. The backend can bea self-hosted inference service or a hosted API; implementation andcaller are decoupled."""def chat(self, req: ChatRequest) -> ChatResponse:backend = self.route(req) # route by task type and cost constraintstry:resp = backend.invoke(req)except TimeoutError:resp = self.fallback(req) # fall back to a backup backendself.record_usage(req, resp) # record usage to support cost allocationreturn resp
4. From demo to production: inference architecture and cost governance
To determine whether model-related spending is sustainable, cost, capacity, and operations must be evaluated within the same framework.
1. Cost model: calculate clearly before optimizing
Large model inference cost can be simplified as follows (this is only an illustrative model):
Cost per request ~ input tokens * input price + output tokens * output priceTotal spending = sum of all per-request costs + fixed resource costs (self-hosted GPU instances, storage, bandwidth, etc.)Number of self-hosted instances ~ peak concurrent requests / stable concurrency per instance * (1 + redundancy factor)
From this, four cost-reduction paths emerge:
- Compress input: streamline prompts, separate stable common prefixes from dynamic content, and use cache hits to reduce repeated computation.
- Control output: set the maximum generation length according to actual task needs and avoid unconstrained long-text generation.
- Tiered scheduling: route simple tasks to smaller-parameter models or rule engines, and escalate complex tasks to higher-capability models.
- Elastic scaling: self-hosted deployments should scale according to actual load to avoid idle resources during low-peak periods; hosted API usage is usually pay-as-you-go and does not require reserved capacity.
2. Capacity planning: use load-test data to drive decisions
Capacity planning should not rely only on experience-based estimates. Load testing is recommended to cover the following scenarios:
- Normal load: throughput and latency baselines under average concurrency;
- Peak load: rate-limiting and queuing behavior during traffic peaks such as promotional events or seasonal demand surges;
- Long-tail requests: latency distribution when extremely long contexts and multiple tool calls are combined;
- Abnormal scenarios: degradation and recovery capability after backend timeouts or rate limits are triggered.
Load-test metrics should include at least tokens per second, P95/P99 time to first token, average latency per token, error rate, and GPU utilization. If a hosted service is used, it is also necessary to confirm each model’s context length limit, rate limit, and stability during peak hours, and to treat pricing, quotas, and deprecation policies as verification items against the official documentation.
3. Observability and governance: make every expense explainable
- End-to-end tracing: record latency from user request to model call and tool execution to determine whether the bottleneck lies in model inference or external dependencies.
- Usage allocation: group token usage by business line, application, and tenant, clarify cost ownership, and curb ineffective calls.
- Quality regression: establish a small-scale evaluation set and run regression tests on key business scenarios before each model upgrade, preventing the situation where “cost decreases but quality also decreases.”
5. Deployment form: hosted API usage vs self-hosted deployment
When enterprises choose between “open source means self-hosting” and “hosted and ready to use,” they can evaluate four dimensions:
- Data sensitivity: for scenarios involving sensitive data where data must not leave a controlled domain, private deployment should be considered first;
- Traffic profile: when request volume is stable and predictable over the long term, the marginal cost of self-hosting is usually lower; when traffic fluctuates sharply, hosted API elasticity is more economical;
- Team capability: whether the team has capabilities in GPU operations, inference-engine tuning, and high-availability support;
- Compliance requirements: whether specific data residency, auditing, or industry regulatory requirements need to be met.
In practice, a hybrid form is more common: sensitive data and high-value tasks run on self-hosted deployments, while general tasks and elastic traffic use hosted API usage, with a unified adaptation layer handling routing and fallback. Regardless of the form, the following general cloud capabilities are essential:
- Container platform: runs inference services and enables elastic scaling and rolling updates;
- Object storage: stores model weights, knowledge-base documents, and call logs;
- Message queue: smooths traffic peaks and decouples long-running tasks from real-time requests;
- Logging, monitoring, and alerting: tracks latency, error rates, and resource utilization.
6. Seven-item checklist for enterprise selection and deployment
- Business validation: build an evaluation set from real business data and do not rely solely on public benchmarks.
- Cost estimation: estimate the monthly bill based on the actual input/output token mix and set budget alerts.
- Stability validation: run load tests during business peak hours and confirm latency distribution and rate-limiting strategy.
- Private deployment assessment: confirm weight licensing terms, hardware requirements, and minimum resource scale.
- Compatibility check: verify integration cost with existing development frameworks, toolchains, and AI agent runtimes.
- Governance design: define data masking, log retention, audit traceability, and content safety policies.
- Exit mechanism: maintain an adaptation layer and backup options, and prepare rollback plans for model upgrades or migrations.
7. Interpret market share rationally: strengths and risks coexist
A spending share of nearly 90% indicates that open-source models have achieved scalable supply capability in a specific market segment. This reflects the engineering capability of China’s AI ecosystem and also means that global enterprises have formed significant demand for model services, inference calls, and industry integration. However, it does not mean that China’s AI is leading across all layers, nor does it mean that leadership will continue naturally.
For enterprises, a concentrated market share also creates risks:
- Supply chain concentration: if a small number of providers experience service interruptions or policy changes, the impact may be amplified;
- Rapid version iteration: capability upgrades may change the behavior of prompts and tool calls, requiring regression validation;
- Pricing and terms changes: pricing and commercial terms may change, and these should be included in contract and budget risk assessments.
The solution lies in “loose coupling” at the architecture level: through a unified inference adaptation layer, models can become replaceable infrastructure components rather than black boxes deeply tied to business logic.
Conclusion
The real implication of this spending data is: what determines the flow of enterprise AI spending is never a single leaderboard ranking, but the combined competitiveness of unit token cost, inference stability, private deployment capability, toolchain compatibility, and long-term service. Whoever can run models stably, economically, and maintainably in real business environments can win long-term enterprise spending. For enterprise technical teams, rather than chasing market-share rankings, it is more valuable to make selection evaluation, cost governance, and architectural decoupling solid. Only in this way can they maintain their own technical autonomy in a rapidly iterating open-source ecosystem.
