China’s Open-Source LLMs: From API Sales to Delivering Value
Chinese open-source LLMs are being used at scale, but usage is not converting into revenue at the same rate. This article examines the gap between token consumption and spending, explains why per-token API pricing and open weights create margin pressure, and outlines a practical path from API supply to value delivery. It covers tiered commercial models, a model gateway and usage ledger, inference cost optimization, enterprise delivery, success-based unit economics, and a phased implementation roadmap.
1. Scale and Profit Are Out of Sync: Usage Is Not Revenue
Industry research suggests that open-source LLMs have increased their share of enterprise token consumption from roughly 11% to 19%, while their share of corresponding spending has fallen from about 6% to 4%. At the same time, in some international enterprise procurement of open-source models, Chinese open-source LLMs have already gained a significant share, approaching 90%. Taken together, these two sets of data point to the same fact: model usage is spreading quickly, but paid adoption is not growing in step.
For developers, this is good news: capability supply is abundant and the barrier to invocation is falling. For enterprises, it creates a window of controllable costs and more choices. But for model providers, it signals a transition from “interface supply” to “value delivery.” An API is an entry point, not the end state. If the business model stops at per-token billing, growth can easily become low-quality expansion: the more calls are made, the thinner the margin becomes.
The core question here is this: Chinese open-source LLMs have already achieved a meaningful global usage share. How can the next stage convert invocation scale into profit and long-term customer relationships through engineering, productization, and governance?
2. Where the Scissors Gap Comes From
2.1 Single-Dimension Token Billing Pushes Competition Toward Compute Price
Token billing is simple and transparent, which makes it suitable for developers getting started. But it implicitly assumes that different requests have roughly equal value. Reality is different: a sentence polish, a contract-clause extraction, and a multi-step tool-calling workflow may consume similar numbers of tokens while differing by orders of magnitude in business value. When every capability is compressed into a “price per million tokens,” the easiest thing for customers to compare is price, and suppliers are easily dragged into homogeneous competition.
2.2 Open Weights Reduce Lock-In
Open-source models allow enterprises to self-host, privatize, and integrate secondarily. This raises adoption, but it also weakens the irreplaceability of any single API. Customers will calculate: if building their own inference cluster is cheaper, why buy a managed API over the long term? The premium for a managed API must therefore come from elasticity, stability, operational simplicity, compliance auditing, model routing, observability, and enterprise-grade support, not merely from “being able to call a model.”
2.3 Complex Workflows Consume More Than Tokens
Real enterprise scenarios usually include retrieval augmentation, tool calls, context compression, retries, caching, human review, log retention, and security filtering. These costs are not fully reflected in output tokens. If the billing dimension covers only input and output tokens, the more complex the workflow becomes, the higher the provider’s cost burden, while revenue may not increase.
2.4 High Developer Traffic and Enterprise Payment Are Separated by a Delivery Gap
High invocation volume may come from trials, evaluations, demos, and low-frequency tools. Converting it into revenue requires crossing the gap from “it runs” to “it can go live”: identity and permissions, budget control, SLAs, data isolation, auditing, invoicing and contracts, incident response, and version upgrades. None of these can be missing. An API solves only “connectivity”; enterprise delivery solves “willingness to use, continuous use, and use at scale.”
3. Rebuilding the Business Model: From Selling Interfaces to Selling Outcomes
A more reasonable path is layered supply. Different customers should not be constrained by the same token price.
| Layer | Offering | Primary Metering Method | Value Proposition |
|---|---|---|---|
| Basic inference layer | Model inference API | Requests, input/output tokens | Low-barrier access and rapid validation |
| Managed service layer | Elastic inference, monitoring and alerts, upgrade and rollback | Instance hours, request volume, SLA | Reduced operational complexity |
| Scenario capability layer | Retrieval augmentation, structured extraction, workflow orchestration, evaluation sets | Successful tasks, active seats, documents processed | Direct mapping to business output |
| Assurance service layer | Dedicated isolation, audit logs, compliance support, dedicated quota | Subscription fee, committed spend | Risk control and long-term stability |
“Billing by successful task” deserves particular attention, but only if “success” is clearly defined. A successful API response does not equal business success. Business success also needs to satisfy structural validation, human spot-check pass rates, timeout limits, policy compliance, and other conditions. A combination of “base fee + successful-task fee + overage usage” can preserve predictability while linking price to value.
4. Engineering Foundation: Model Gateway and Cost Ledger
To turn usage into revenue, the first step is to record clearly what happens on every call. A unified model gateway should be added before model inference, with metering events carried throughout the entire chain.
A typical architecture can be abstracted as follows:
- Access layer: authentication, tenant identification, rate limiting, budget validation;
- Routing layer: select the model tier based on task type, context length, quality requirements, and real-time requirements;
- Inference layer: batching, caching, context trimming, output-length control;
- Capability layer: retrieval, tool calls, structured-output validation;
- Observability layer: logs, metrics, tracing, auditing;
- Billing layer: metering-event aggregation, cost allocation, invoice generation, and reconciliation.
Pseudocode:
type UsageEvent = record(request_id,tenant_id,app_id,user_id,model_alias,route_reason,input_tokens,output_tokens,gpu_ms,cache_hit,retrieval_units,tool_call_count,storage_mb_hour,latency_ms,status,policy_tags,created_at)function compute_charge(event):base = price_input * event.input_tokens+ price_output * event.output_tokensretrieval = price_retrieval * event.retrieval_unitstools = price_tool * event.tool_call_countstorage = price_storage * event.storage_mb_hourplatform = shared_ops_rate * event.gpu_mschargeable = event.status == "success"and "policy_blocked" not in event.policy_tagsand event.over_quota == falsereturn chargeable ? base + retrieval + tools + storage + platform : 0
This design has three key points. First, model aliases should be decoupled from the actual underlying model so that models can be switched without breaking customer interfaces. Second, metering events must be idempotent, deduplicable, and auditable to avoid duplicate billing caused by retries. Third, the cost ledger must cover shared-resource allocation, such as idle capacity, monitoring storage, operations labor, and compliance costs; otherwise, book gross margin will be distorted.
5. Lowering Inference Costs: Without Cost Optimization, There Is No Room to Cut Prices
Price competition is not what to fear most; the real danger is lacking a cost advantage. Inference-side optimization should form a portfolio of strategies that are observable and reversible.
- Model cascading: use small models for simple intent recognition, classification, and summarization; escalate complex reasoning to larger models. The key is to build automated evaluation sets to prevent quality loss after downgrading.
- Caching strategy: use exact caching for high-frequency, stable requests with low freshness requirements; semantic caching must be handled carefully, with similarity thresholds, tenant isolation, and expiration policies.
- Batching and concurrency scheduling: use off-peak batching, request merging, and priority queues to improve resource utilization; real-time interactive requests and offline tasks should use isolated resource pools.
- Context governance: compress conversation history, trim irrelevant retrieval snippets, and limit maximum output length. These often reduce costs more effectively than simply upgrading hardware.
- Inference-kernel optimization: quantization, operator fusion, paged key-value cache management, and speculative decoding can reduce unit inference costs, but they must be paired with quality regression testing.
- Budget-aware generation: carry budget and time limits in the request context. When the budget is exceeded, return a summary, recommendation, or handoff to a human instead of generating without limits.
6. Enterprise Delivery: Standardized Capabilities Are More Sustainable Than Project-Based Work
Enterprise customers are not buying “a model”; they are buying “a capability that can run over the long term.” Therefore, deliverables should include at least the deployment topology, upgrade and rollback processes, monitoring metrics, incident-handling manuals, security policies, license inventories, and acceptance criteria.
Private deployment and managed services should share the same control plane: unified metering, unified logging, unified policies, and unified upgrade packages. Otherwise, every project becomes custom engineering, and revenue growth will be consumed by delivery costs. On the data side, tenant isolation, encryption in transit and at rest, sensitive-information redaction, retention periods, and deletion mechanisms are required. On the model side, open-source license review is required to clarify weight distribution, redistribution, commercial use, and restrictions on derivative models, while maintaining a component inventory to avoid compliance risks surfacing only after scale.
7. Operating Metrics: Replace “Cost per Thousand Tokens” with “Cost per Successful Task”
It is advisable to shift the core metric from “price per thousand tokens” to “cost per successful task” and “gross margin per successful task.”
cost_per_successful_task =(inference_resource_cost + retrieval_and_tool_cost + operations_allocation + compliance_cost)/ successful_tasksgross_margin_per_successful_task =revenue_per_successful_task - cost_per_successful_task
Supporting metrics include paid conversion rate, GPU utilization, cache hit rate, P95 latency, failure and retry costs, human-escalation rate, tenant retention, and committed-spend completion rate. Growth is healthy only when invocation growth brings positive gross margin and renewals. If every call has negative gross margin, the larger the scale, the faster the losses.
8. Implementation Roadmap
In the first phase, establish a unified model gateway, metering events, and budget control so that every call is traceable, limitable, and reconcilable. In the second phase, introduce model routing, caching, and context governance, and build a quality regression set to ensure cost reduction does not reduce quality. In the third phase, standardize enterprise delivery and security governance to create replicable private deployment, managed service, and operations solutions. In the fourth phase, gradually transition from token billing to a combined pricing model of “base subscription + usage + successful tasks,” incorporating customer success into revenue design.
9. Conclusion
Chinese open-source LLMs have already proven that they can be used, used heavily, and adopted quickly. But the next stage of competition is no longer only about parameters and invocation volume; it is about unit inference cost, enterprise delivery efficiency, security governance capability, and value metering. An API is the first layer connecting developers, but it is not the endpoint of profit. The engineering path from market share to actual profit is to package model capabilities into reliable workflows, convert invocation volume into successful tasks, and convert successful tasks into auditable bills and long-term customer relationships.
