Open-Weight LLM Usage Is Rising While Unit Revenue Is Falling: Why

    Open-weight large language models are moving from proof-of-concept to regular enterprise use, but rising token consumption is not translating into proportional revenue. This article explains why enterprise token share has increased while spending share has declined, examining four structural drivers: task stratification, hosting homogenization, falling inference costs, and self-hosted shadow consumption. It then outlines how vendors and enterprises can move beyond token-level pricing through private deployment, productization, task- or outcome-based billing, and technical practices such as tiered routing, caching, batching, and metering.

    1. The signal: token usage is rising, but paid revenue is not keeping up

    Open-weight models—models whose weights can be downloaded and self-hosted—are becoming a major choice for enterprise AI adoption. One industry report indicates that about 90% of enterprise open-weight model spending in the United States flows to Chinese models. Another set of figures is more notable: the share of open-weight models in enterprise token consumption has risen from 11% to 19%, while their share of spending has fallen from 6% to 4%.

    Usage growth is also visible. Data from an AI aggregation platform shows that token invocation volume for Chinese models has exceeded that of overseas models for more than 20 consecutive weeks, with four of the top five weekly call-volume models being Chinese models. One Chinese model recorded approximately 15 trillion token calls over seven days.

    Usage curves up, revenue curves down. This widening “usage–revenue scissors gap” is one of the most important signals in the open-model ecosystem today.

    For developers, the comparison has two implications:

    • Demand signal: enterprises first vote with tokens before deciding whether to expand investment. A 19% consumption share indicates that open-weight models have completed POC validation and entered regular production use.
    • Commercialization signal: whether market share can convert into revenue determines whether open-weight models can form a positive loop where revenue feeds back into R&D and supports continuous iteration.

    From a system perspective, the gap appears in a typical two-layer market:

    • Upper layer: model and compute supply, including weights, inference services, and hosting platforms.
    • Lower layer: business consumption, including agents, workflows, and enterprise applications.

    Upper-layer revenue equals call volume multiplied by unit price, while the value created in the lower layer is much higher than the token fees paid. If the upper layer becomes replaceable infrastructure, profit will accumulate closer to the business. Therefore, Chinese model vendors are shifting from “selling APIs” to more complete delivery forms: selling capabilities, services, and outcomes. This shift is not merely a commercial strategy adjustment; it is a systemic restructuring of technical architecture and product form.

    2. Why the gap exists: four structural drivers

    2.1 Task stratification: not all tokens have the same value

    The global large-model market has formed a relatively clear value stratification:

    • Models with leading complex reasoning capabilities mainly handle high-value tasks such as complex logical reasoning, deep code generation, and long-horizon agent planning.
    • Open-weight models handle large-scale parallel tasks that are cost-sensitive, such as web scraping and parsing, batch information extraction and classification, content moderation, and document structuring.

    Ranked by business value per token, typical scenarios roughly form the following gradient:

    Tier Typical tasks Business value per unit token Price sensitivity
    High Architecture design, difficult problem diagnosis, complex code generation High Low
    Medium Report writing, code review, data analysis Medium Medium
    Low Batch extraction, classification, summarization, scraped-content cleaning Low High

    Open-weight models have penetrated the fastest in medium- and low-tier scenarios. These tasks scale linearly with business volume and are naturally high-volume, low-margin businesses. In such cases, lower unit prices are the result of market rationality, not model capability failure.

    The key point is that tokens in medium- and low-tier scenarios are consumables. They do not inherently create premium value; the real value lies in the pipelines and deliverables built around them. If vendors price only by tokens, they can capture only a small part of the value in the pipeline.

    2.2 Hosting homogenization: basic calls converge toward infrastructure

    Once weights are open, the same model can be hosted by multiple parties, and the basic inference capabilities offered by vendors become homogeneous. When output differences among providers are small, procurement decisions are left with only two variables: price and latency.

    The competitive logic of basic token calls is therefore similar to basic resources in cloud computing. Take cloud hosts as an example. General-purpose instances with the same specifications are highly comparable across major cloud providers. Prices remain under long-term pressure, and vendors rely more on managed services, supporting capabilities, and ecosystems for profit.

    The price of a standard general-purpose instance is usually lower than that of specialized instances. Specialized instances refer to dedicated instances optimized for specific workloads, as well as pricing systems created by combinations of on-demand and subscription billing. Model services are following a similar path: basic calls increasingly resemble public infrastructure, and pricing converges toward inference cost.

    For vendors, this is not necessarily bad news. Low price itself is an effective customer-acquisition tool: Chinese models entered global developers’ workflows through compute cost-effectiveness and engineering optimization. The problem is that if vendors remain at the basic-call layer, the larger the scale, the thinner the profit.

    2.3 Cost reduction: prices converge toward inference cost

    Model inference is a typical cost-engineering problem. The cost per request depends on:

    • the number of input and output tokens;
    • the effective number of parameters involved in parallel computation;
    • utilization of GPU memory and compute resources.

    The industry already has a set of mature cost-reduction methods:

    Technique Basic principle Main effect
    Quantization Represent weights and activations with lower-precision numerical formats Reduce memory usage and improve throughput
    Speculative decoding Use a lightweight model to “guess” multiple tokens, then verify them in parallel with the main model Reduce decoding latency while keeping output quality largely unchanged
    Prefix caching Reuse the KV cache for identical prefixes Reduce repeated prefill computation; especially useful for long system prompts and multi-turn conversations
    Continuous batching Dynamically batch requests at the request level instead of waiting for an entire batch to finish Improve GPU utilization and overall throughput

    These techniques continuously lower the compute cost per token. However, in the open-weight market, competition quickly converts cost advantages into price reductions, causing unit revenue to converge toward inference cost. Some researchers point out that low price alone is difficult to scale sustainably. A vendor can obtain a high paid share only if the model can reliably support enterprise production scenarios. A true source of competitiveness is the continued reduction of compute consumption for the same tasks.

    For developers, the lesson is to use cost per task, not token price, as the core selection metric. For example, when completing one document extraction, model A may have a lower unit price, but its outputs often require human review; model B may have a higher unit price but passes on the first attempt. Model B often has lower total cost.

    2.4 Procurement structure: open weights drive “shadow consumption”

    Open weights create a significant amount of “shadow consumption”: enterprises can directly download model weights and run inference on their own GPUs, private clouds, or self-built clusters. These calls are not measured through any API and do not appear in API billing, yet they still reflect real model usage.

    This partially explains the phenomenon of consumption share exceeding spending share:

    • Consumption statistics count total calls, including self-hosted usage.
    • Spending statistics cover only API-billed usage.

    Of course, self-hosting is not free. GPU depreciation, operations staffing, and inference-engine optimization are all explicit costs. This value is currently absorbed by enterprises and has not been captured by model vendors. How to convert users who “download once and self-host forever” into sustainable paying customers is a key question for Chinese model vendors.

    3. Breaking out: from API billing to productized monetization

    An API is only one delivery form of model capability, and it is the most easily replaceable one. To escape the usage–revenue gap, vendors need to package capabilities into products that are harder to replace. Common monetization paths can be summarized as follows:

    1. Open-weight model (hostable by multiple parties; capabilities converge)
    2. |
    3. v
    4. +-------------------------------------------------------------+
    5. | Productization and engineering layer |
    6. | Tiered routing | tools and agent frameworks | RAG/knowledge base |
    7. | Evaluation and observability | cost and usage governance |
    8. +-------------------------------------------------------------+
    9. |
    10. v
    11. +----------------------+-----------------------------+
    12. | Usage-based API | Private/dedicated deployment|
    13. | Low price, high scale| High average contract value |
    14. +----------------------+-----------------------------+
    15. |
    16. v
    17. Billing by task, seat, or business outcome (enterprise products)

    3.1 Private and dedicated deployment: from token business to delivery business

    For scenarios where data cannot leave the enterprise, where latency is sensitive, or where compliance is required, public cloud APIs are not the only choice. Private deployment, dedicated deployment, and hybrid cloud approaches have become mainstream alternatives:

    • Private deployment: the model and inference services are deployed in the enterprise’s own data centers or private cloud.
    • Dedicated deployment: the cloud allocates independent compute resources to a single customer, isolated from multi-tenant shared pools.
    • Hybrid deployment: sensitive data is processed locally, while elastic peak requests are routed to the cloud.

    Here, isolation is not optional but a hard requirement. Under GB/T 22239-2019, China’s cybersecurity classified protection framework, centrally managed systems may require independent or logically isolated compute and storage resources to protect the confidentiality of authentication information, important business data, and important personal information during storage. If multi-tenant model services lack strict tenant isolation and log masking mechanisms, they may struggle to meet such requirements.

    The key to private delivery is operability. Vendors usually need to provide:

    Capability Purpose
    One-click deployment and version upgrades Reduce customer launch and upgrade costs
    Multi-hardware backend support Compatible with different domestic and general-purpose accelerator cards
    Inference-engine tuning Optimize throughput and latency for the customer’s hardware and workload
    Observability integration Make model calls as traceable as business services
    Tenant isolation and auditing Meet compliance and data-security requirements

    Once the delivery form changes, the revenue structure changes as well: a one-time license fee plus annual maintenance, or subscription by node/compute. The average contract value is much higher than API calls, but it depends more on delivery and operations capabilities.

    3.2 From selling tokens to selling outcomes: moving up the billing unit

    If value lies in the pipeline, the billing unit should move up. Evolving from token billing to task, seat, or outcome billing is a path many software products have followed. It can be divided into four stages:

    Stage Billing unit Customer perception
    Stage 1: usage-based Tokens, API calls Unpredictable usage; difficult to budget
    Stage 2: seat-based Users, concurrency Predictable cost; tied to team size
    Stage 3: task-based Completed documents, tickets, code reviews Linearly correlated with business volume
    Stage 4: outcome-based Valid deliverables, acceptance pass rate Tied to business results; highest average contract value

    Take batch document processing as an example again. The differences between two billing models are as follows:

    1. Token-based billing (illustrative; data for demonstration only)
    2. Unit price: CNY 0.001 per 1K tokens; about 4,000 input tokens per document
    3. API cost per document ≈ CNY 0.004
    4. Customer view: low unit price, but total cost grows with document volume and is hard to budget in advance
    5. Task-based billing (illustrative)
    6. Subscription: CNY 299 per seat per month, including 5,000 documents per month
    7. Cost per document ≈ CNY 0.06
    8. Customer view: higher unit price, but predictable cost and includes
    9. parsing, validation, and human-review workspace capabilities

    The second option has a higher apparent cost per document in token terms. Why are customers still willing to pay? Because it sells a finished outcome—processed documents—which includes substantial engineering work such as preprocessing, quality validation, and exception handling. These activities cannot be covered by token pricing alone.

    Regardless of the billing model, three technical foundations are required:

    • Trusted metering: usage data must be accurate and reconcilable; this is a prerequisite for any billing model.
    • Verifiable outcomes: there must be clear criteria for whether a task succeeded and whether the output meets requirements.
    • SLA guarantees: high-value billing models must be supported by availability commitments and compensation mechanisms.

    3.3 Competitiveness anchor: compute consumption for the same task

    The financial report of a leading Chinese model vendor provides an observation point: revenue from its open platform and API-related business grew by about 4x year over year. This supports a judgment: call volume can convert into revenue, provided enough calls land in billable and value-added areas.

    The underlying capability for this conversion is the continuous reduction of compute consumption for the same task. To judge whether a model has cost competitiveness, a “capability-to-cost ratio” metric can be introduced:

    1. Capability-to-cost ratio = task pass rate on evaluation sets / inference cost per task
    2. Inference cost per task ≈ average tokens per request × blended unit price
    3. / batch-throughput improvement factor

    This metric is closer to real experience than looking at token price alone:

    • A higher pass rate means fewer retries, human reviews, and fallback calls.
    • A shorter average token count means more efficient context management.
    • Higher throughput means lower amortized compute cost per unit of fixed compute.

    In engineering, vendors can use quantization-aware training, distillation, and reinforcement-learning post-training to let models complete the same tasks with fewer tokens. Developers can use agent-development technology stacks, default templates, replaceable model-adapter layers, and tools such as model routing and fine-tuning to adapt to different models at lower cost.

    4. Technical implementation: making every token spend worthwhile

    For enterprises using open-weight models, the same gap appears in their own bills. On the vendor side, the strategy is to trade cost reduction for scale. On the user side, the strategy should be to trade architecture for efficiency. Common measures can be grouped into four main lines: routing, caching, batching, and metering.

    4.1 Tiered routing: allocate compute budget by task value

    Not every request needs the strongest model. Using general capabilities such as model routing and model evaluation, a system can dispatch requests to models of different sizes based on task complexity, latency budget, and value level. Illustrative code:

    1. import hashlib
    2. import json
    3. from dataclasses import dataclass
    4. @dataclass
    5. class ModelTier:
    6. """Model tier description (illustrative definition)"""
    7. name: str
    8. quality_score: float # weighted score on evaluation set, in [0, 1]
    9. cost_per_1k_tokens: float # blended unit price (CNY per 1K tokens)
    10. p95_latency_ms: int # latency metric
    11. @dataclass
    12. class TaskRequest:
    13. scene: str # business scenario
    14. input_text: str
    15. value_level: str # low | mid | high
    16. latency_budget_ms: int # latency budget
    17. def choose_tier(req: TaskRequest, tiers: list[ModelTier]) -> ModelTier:
    18. """Illustrative: choose a model tier based on scene, task value, and latency budget"""
    19. # Low-value scenarios: batch extraction, classification, cleaning, etc.; prefer low-cost tiers
    20. low_value_scenes = {"extract", "classify", "clean"}
    21. if req.scene in low_value_scenes or req.value_level == "low":
    22. return tiers[0]
    23. # High-value or high-latency-budget scenarios: complex reasoning, code generation, etc.; use high-quality tiers
    24. if req.value_level == "high":
    25. return tiers[-1]
    26. # Middle scenarios: choose the tier with the lowest cost per quality unit while meeting latency budget
    27. candidates = [t for t in tiers if t.p95_latency_ms <= req.latency_budget_ms]
    28. return min(candidates, key=lambda t: t.cost_per_1k_tokens / t.quality_score)
    29. def route(req: TaskRequest, tiers: list[ModelTier]) -> str:
    30. """Illustrative: generate a routing decision and output audit-friendly logs"""
    31. tier = choose_tier(req, tiers)
    32. decision = {
    33. "scene": req.scene,
    34. "value_level": req.value_level,
    35. "selected": tier.name,
    36. # Hash the input to avoid leaving sensitive original text in routing logs
    37. "input_hash": hashlib.sha256(req.input_text.encode()).hexdigest()[:16],
    38. }
    39. print(json.dumps(decision, ensure_ascii=False))
    40. return tier.name

    4.2 Context engineering and cache reuse

    Requests within the same business often share a large amount of fixed content, such as system prompts, tool descriptions, knowledge-base snippets, and few-shot examples. Leveraging this pattern can significantly reduce repeated computation:

    • Prompt-prefix caching: keep invariant content at the front of the prompt; in multi-turn conversations, only append new content and reuse the existing KV cache.
    • Semantic caching: return historical results for semantically equivalent requests; suitable for common questions.
    • Retrieval-augmented generation: send only the most relevant snippets to the context, avoiding stuffing an entire document into the context window.
    1. def build_prompt(system_prefix: str, docs: list[str],
    2. question: str, budget: int = 8000) -> str:
    3. """Illustrative: keep the system prefix unchanged for prefix-cache reuse and trim retrieved snippets within budget"""
    4. reserved = 1000 # token budget reserved for question and output
    5. used, blocks = 0, []
    6. for doc in docs:
    7. n = estimate_tokens(doc) # use the actual tokenizer for estimation
    8. if used + n > budget - reserved:
    9. break
    10. blocks.append(doc)
    11. used += n
    12. context = "\n".join(blocks)
    13. # A constant system prefix improves cache hit rate; dynamic content is placed at the end
    14. return f"{system_prefix}\n[Reference]\n{context}\n[Question]\n{question}"

    4.3 Batching and asynchronous processing

    Tasks such as batch extraction and classification have low real-time requirements and are suitable for asynchronous processing:

    1. Requests enter a message queue, and callers retrieve results through polling or callbacks.
    2. The consumer uses a continuous-batching inference service to dynamically form batches during high concurrency.
    3. Use server-side batch interfaces and schedule computation during off-peak hours when possible to obtain more favorable unit pricing.

    4.4 End-to-end metering and observability

    Regardless of the billing model, observability of usage and cost is a prerequisite. It is recommended to move metering points down to a unified gateway and integrate them with business metrics:

    1. def record_usage(tenant: str, model: str, resp, ctx: dict) -> None:
    2. """Illustrative: record usage data at the model gateway; requires an actual observability backend"""
    3. u = resp.usage
    4. total_in = u.get("input_tokens", 0)
    5. cached_in = u.get("cached_input_tokens", 0)
    6. billable_in = max(total_in - cached_in, 0) # cached tokens are usually billed separately
    7. out = u.get("output_tokens", 0)
    8. metrics.record({
    9. "tenant": tenant, "model": model,
    10. "scene": ctx["scene"], "request_id": ctx["request_id"],
    11. "input_tokens": total_in, "cached_tokens": cached_in,
    12. "output_tokens": out,
    13. "latency_ms": ctx["latency_ms"],
    14. # Correlate with business outcome to calculate cost per valid output
    15. "task_success": ctx.get("task_success"),
    16. # Record models that actually failed in fallback paths to analyze fallback rate
    17. "model_error": ctx.get("model_error", ""),
    18. })

    Along with metering, quality gates should be built: use a fixed evaluation set to run regression tests for model upgrades, prompt changes, and routing-policy changes to ensure that cost reduction does not come at the expense of quality.

    4.5 Cost governance and elastic degradation

    • Budget guardrails: set token quotas and alert thresholds by tenant and scenario. When thresholds are approached, automatically degrade to lower-cost tiers or queue requests.
    • Automatic fallback: high-value requests first go through the main model; on timeout or errors, degrade along a predefined chain to preserve availability.
    • Regular reviews: calculate cost per valid output by scenario to identify high-cost, low-value call chains.

    In practical engineering, capabilities such as model services, elastic inference containers, message queues, object storage for knowledge-base documents and logs, log services, and monitoring alerts can be combined to form a complete governance loop. For example, enterprises can use general capabilities from Baidu AI Cloud’s Qianfan large-model platform, such as model routing, model evaluation, and private deployment, to perform multi-model integration and effectiveness governance on a unified platform.

    1. Basic calls become public infrastructure: basic inference for open-weight models will become a standard resource like cloud hosts or CDNs. Prices will remain under long-term pressure, while competition will shift to stability, latency, and ecosystem compatibility.
    2. Value migrates to both ends: one end is the model with the strongest complex-reasoning capability and the highest business value per token; the other end is workflow products and outcome delivery closest to the business. Players that only resell APIs in the middle layer will see their space continuously compressed.
    3. Delivery forms diversify: public cloud APIs, private deployment, hybrid deployment, and outcome-based billing will coexist for the long term. Vendors need technical and operational capabilities to support multiple delivery forms at the same time.

    5.2 Recommendations for model vendors

    • Build an iteration system guided by the capability-to-cost ratio, continuously reducing compute consumption for the same tasks.
    • Package API capabilities as standardized products that can be privately delivered, lowering the barrier for enterprise adoption.
    • Build trusted metering, SLA, and observability systems to lay the foundation for higher-level billing models.

    5.3 Recommendations for enterprise users

    • Use cost per task and task pass rate as core model-selection metrics.
    • Use tiered routing, caching, batching, and similar methods to reduce inference costs to a business-acceptable range.
    • Prioritize private or dedicated deployment when sensitive data is involved.
    • Introduce usage transparency and portability clauses in procurement negotiations to avoid lock-in to a single model.

    Open-weight models won developers’ tokens with performance and cost-effectiveness, but winning market share is only the first step. Converting call volume into sustainable revenue requires long-term investment by vendors in inference efficiency, engineering delivery, and product design. For enterprises, the winner is the one that can deliver more valid outcomes under the same budget and truly benefit from this efficiency dividend.