Open-Source LLMs: How to Turn Market Share into Profit

    Chinese open-source models now capture nearly 90% of US enterprise spending on open-source models, yet their share of enterprise token consumption is growing faster than their share of spending—a "usage–revenue scissors gap" that keeps pushing per-token prices down. This article breaks down the three structural forces behind that gap, lays out a three-tier monetization path from selling tokens to selling tasks to selling workflows, and details the engineering foundations—unit-cost control, task-tiered routing, and stability/security/observability—along with the unit-economics metrics needed to judge whether market share is actually turning into profit.

    1. The Phenomenon: Winning Calls, Not Winning Price

    A recent industry report from a US venture capital firm projects that global enterprise AI spending will reach $176.8 billion in 2026. One data point stands out: nearly 90% of what US enterprises spend on open-source models flows to products built on Chinese models. The top three vendors account for 87% combined—61%, 15%, and 11% respectively.

    A different comparison in the same report is the signal that should really raise concern:

    Metric Prior period Current period Change
    Open-source models’ share of enterprise token consumption 11% 19% +8 pp
    Open-source models’ share of enterprise model spending 6% 4% -2 pp

    When a model’s usage share grows faster than its spending share, you can define a “usage–revenue scissors gap.” Dividing one share by the other yields the open-source model’s “relative price index”—the ratio of its per-token charge to the enterprise market average:

    • Prior period: 6% ÷ 11% ≈ 0.55
    • Current period: 4% ÷ 19% ≈ 0.21
    • Ratio: 0.21 ÷ 0.55 ≈ 0.39

    Measured against the market-average unit price, the relative price of open-source models has fallen to roughly 40% of its prior level—a drop of about 60%. Put differently, even as usage share nearly doubled, open-source models not only failed to lift overall revenue share but saw their per-token price keep sliding.

    Call-side data tells the same story. According to statistics from a global AI aggregator platform, in the week of October 26–November 1, token calls on Chinese large models exceeded those on US models for the 27th consecutive week, and Chinese models held four of the top five spots that week. One lightweight model built for high-speed inference logged roughly 15 trillion token calls over the prior seven days, ranking second on the platform.

    In individual segments, Chinese models are already the winners. The real difficulty now surfaces: how to turn market share into hard profit.

    2. Breaking Down the Causes: Three Structural Forces

    2.1 Task stratification: taking the “high-volume, low-price” layer

    Some researchers point out that the global division of labor in AI has become fairly clear. Leading closed-source vendors hold high-value complex-reasoning tasks, while domestic models take on large-scale parallel workloads such as agent web scraping and batch information processing. These tasks have two characteristics:

    • Short individual calls, with inputs and outputs mostly structured text, valuing throughput over “cleverness.”
    • Enormous volume, with customers highly sensitive to price and facing almost no switching cost to move to an equivalent model.

    Taking the layer with the largest usage but not the layer with the greatest pricing power makes price decline almost unavoidable.

    Once weights are open, the same model can be hosted on multiple platforms. As long as output quality shows no meaningful difference, customers naturally gravitate to the cheaper option. Base token calls therefore come to resemble compute and storage, drifting toward infrastructure, with prices converging toward inference cost. The capability itself hasn’t degraded—it has gone from a “scarce good” to a “standard part.”

    2.3 Market stratification: low prices won scenarios, but also locked in the anchor

    Some researchers attribute this scissors gap to stratification in the global AI market, compounded by the open-source model itself. Low pricing is indeed an efficient way to acquire customers at this stage: domestic vendors, leveraging compute cost-performance, have successfully inserted themselves into global developers’ workflows. But once customers anchor their budget expectations to a low price, selling higher-priced services on top becomes especially hard. This mirrors a common problem with foundational libraries: extremely high usage, but very low conversion to paid plans.

    Other researchers offer a different perspective. Capturing an 89% share of paid spending presupposes that the model can reliably support enterprise production scenarios; low price alone rarely achieves large-scale adoption. That competitiveness comes from sustained engineering iteration—pushing down compute consumption on the same task is what lets a vendor keep a price advantage while defending gross margin.

    On balance, the scissors gap is the combined result of task stratification × open-source homogeneity × phased pricing, and is partly a temporary cost of gaining share. But if usage and revenue keep diverging over the long term, the share advantage will struggle to settle into profit. The first-half 2026 financials of an open-source model vendor offer supporting evidence: revenue of 954 million yuan, up 399.7% year over year, with revenue from the open platform and related APIs an important growth source. Rapid revenue growth shows that usage now has a basis for monetization—but “growing fast” is not the same as “earning more.” Whether a vendor can raise the gross margin per task is the real dividing line.

    3. Monetization Paths: From Selling Tokens, to Tasks, to Workflows

    A token is a cost unit, not a value unit. What customers actually buy is a “completed task.” The essence of upgrading monetization is to move the pricing unit ever closer to the value unit.

    Layer What the customer buys Typical pricing Competitive barrier Main risk
    L1 Sell tokens Model capability Per-token billing Cost-performance, stability Homogenization; price converges to cost
    L2 Sell tasks Completion of a single task Per task, per item Output quality, unit cost Quality swings directly erode profit
    L3 Sell workflows End-to-end business processes Subscription, per seat, or a share of savings Tool integration, observability, compliance Heavy delivery; requires ongoing operations

    Path 1: From L1 to L2—switch the pricing unit from tokens to tasks

    Take enterprise document extraction as an example. Suppose a document-processing product consumes 12,000 tokens per document at a price of 4 yuan per million tokens, giving a per-document inference cost of about 0.048 yuan. Charging 0.3 yuan per document can yield a gross margin above 80%—provided the per-document cost is held steadily under 0.1 yuan, which relies on techniques such as prefix caching, batching, and quantization.

    The point isn’t being “cheaper”; it’s being “predictable.” A business can budget for a definite task outcome, but it struggles to budget for fluctuating token consumption. Moving to per-task pricing essentially transfers the cost-fluctuation risk from the customer to the vendor, forcing the vendor to perfect its inference engineering.

    Path 2: From L1 to L3—extend upward along the workflow

    A model can be swapped out, but a toolchain embedded in a business process is hard to replace:

    • Tool calling: the model emits standardized structured results and hands them to downstream systems.
    • Retry and validation: failed requests are retried automatically, and results are first checked by rules or a small model.
    • Human review loop: low-confidence results are routed automatically to manual review.
    • Audit and observability: every call is traceable, and both cost and quality are measurable.

    The “task completion rate” these capabilities produce is what enterprises are willing to pay for over the long term.

    Path 3: Hybrid deployment—meeting the needs of data residency and predictable cost

    For data-sensitive industries, fully public calls are often not viable. Good compatibility between open weights and mainstream acceleration hardware fits this need well. A typical pattern is “process sensitive data locally, supplement elastic peaks from the cloud,” forming layered collaboration among open-source model vendors, integrators, and cloud platforms.

    4. Engineering Foundations: Three Capabilities That Set the Monetization Ceiling

    4.1 Unit-cost control

    Technique How it works Implementation notes
    Weight quantization Reduces VRAM use and raises throughput The calibration evaluation set must stay close to real tasks; ship only after output quality shows no meaningful drop
    KV-cache quantization Cuts the cache overhead of long contexts Watch for compatibility with the inference engine implementation
    Prefix-cache reuse Identical prefixes are computed only once Put system prompts, long documents, and example libraries up front; on the business side, place fixed content at the start of the prompt
    Continuous batching Dynamically batches requests to raise accelerator utilization Cap context length and output length so long requests don’t slow the whole batch
    Speculative decoding A small model drafts and a large model verifies, reducing decode steps The acceptance rate depends on draft quality and needs task-specific tuning
    Structured-output constraints Constrain the model to emit an agreed format Reduces rework and wasted tokens

    Prefix caching only works if the cache key is designed correctly. If you splice a timestamp or random number into the prefix every time, the cache can never hit:

    1. # Example code: partitioning the prefix-cache key
    2. def build_prefix_key(system_prompt: str, knowledge_chunk: str, user_query: str):
    3. """
    4. The prefix cache matches on a "byte-level prefix":
    5. - system_prompt / knowledge_chunk are the stable prefix; never concatenate volatile content into them
    6. - timestamps, request IDs, random numbers, etc. all go into the volatile suffix (the user_query side)
    7. """
    8. stable_prefix = f"[SYS]{system_prompt}\n[KNOW]{knowledge_chunk}\n[TASK]"
    9. volatile_suffix = f"[REQ_ID]request_id\n[QUERY]{user_query}"
    10. return stable_prefix, volatile_suffix

    For long-document scenarios, you can first run a lightweight model or rule engine to filter segments, dropping irrelevant passages before sending them to the main model and directly cutting input tokens.

    4.2 Task-tiered routing

    Not every task deserves the strongest model. Consider processing support tickets:

    1. # Example code: selecting a model tier by task value
    2. def select_model(task):
    3. # High-stakes tasks (amounts, entitlements, etc.) prioritize quality
    4. if task.stakes == "high":
    5. return "flagship"
    6. # Structured batch tasks prioritize cost
    7. if task.schema is not None and not task.needs_reasoning:
    8. return "lightweight"
    9. return "balanced"

    Routing shouldn’t be a one-time configuration; it should be calibrated continuously:

    1. Run offline evaluations for each tier, recording task success rate and unit cost.
    2. Sample real production traffic and re-check against the actual task distribution.
    3. When a lightweight model’s success rate stays below threshold, automatically bump the tier up.

    The tuning objective is to minimize the “cost per successful task”—total cost ÷ number of successful tasks—rather than per-token cost alone.

    4.3 Stability, security, and observability

    • Rate limiting and quotas: set quotas by tenant and endpoint to keep burst traffic from crowding out shared resources.
    • Circuit breaking and graceful degradation: when request queues time out or a downstream service fails, degrade automatically—for example, by shortening context, switching to a lightweight model, or moving to an async queue.
    • Data security: detect and mask sensitive information such as national ID numbers, phone numbers, and bank card numbers before calling; for sensitive-data scenarios, prefer private or dedicated deployment.
    • Audit trail: log the request ID, model version, token consumption, latency, and status code—useful for both cost accounting and fault localization.

    In production, it’s better not to call model APIs directly from business code; a unified access layer is preferable:

    1. Business services
    2. │
    3. ▼
    4. Model gateway (quotas / auth / routing / degradation / cost accounting / audit)
    5. ├── On-premise open-source model cluster (sensitive data)
    6. └── Cloud inference service (elastic peaks)

    5. Measuring Value: Replacing “Call Volume” with “Unit Economics”

    Usage metrics answer “how much was used”; unit-economics metrics answer “how much was earned.” To judge whether monetization is healthy, keep an eye on these metrics:

    Metric How it’s calculated What to watch
    Per-task inference cost Total task token consumption × the corresponding unit price The floor for pricing
    Per-task gross margin Per-task revenue − per-task inference cost − allocated storage and operations cost Must cover retries, reviews, and all other overhead
    Paid conversion rate Paying customers ÷ active customers Whether the free-quota strategy is working
    Cost per successful task Total cost ÷ number of successful tasks The core metric for tuning routing and caching
    Renewal rate and retention Renewing customers ÷ customers up for renewal; cohort retention curve How deeply the business is embedded

    One detail that’s easy to overlook: retries consume resources too. If the failure rate climbs from 2% to 10%, the cost of effective work rises by about an extra 8%.

    6. Conclusion

    Chinese open-source models capturing nearly 90% of US enterprise spending on open-source models is proof of a dual competitiveness in engineering and cost-performance. But the scissors gap between token consumption and revenue shows that share by itself generates no profit.

    For developers, the next stage comes down to three tasks: use quantization, caching, and routing to push unit cost below the price point; use task-level and workflow-level services to lift the pricing unit to the value side; and adopt a metric system that can quantify gross margin and retention to keep testing whether this path holds up. Open source delivers distribution—but profit goes only to those who turn a “call” into “value.”