Open-Source LLMs and the "Scissors Gap": Why Token Volume Keeps Rising While Revenue Share Shrinks

    This article examines the widening gap between usage volume and revenue share for open-source large language models. As enterprise adoption deepens and open weights push basic token calls toward commoditization, call volume climbs while the share of spending declines—a divergence widely discussed as the "scissors gap." Drawing on 2026 enterprise AI spending data, it breaks down the four forces behind the gap (usage mix, token commoditization, falling inference costs, and discounted effective pricing), argues that the trend is structural rather than a sign of weak competitiveness, and lays out two paths forward: shifting billing from tokens to delivered results, and building value engineering around model gateways and inference optimization. It also covers the key metric of cost per successful task, includes cost-aware routing and accounting code samples, presents a three-layer reference architecture, and closes with a selection checklist for both model providers and enterprise developers.

    An annual enterprise AI survey projects that global enterprise AI spending will reach $176.8 billion in 2026. Within US enterprise spending on open-source models, roughly 89% flows to Chinese models. Open-source models’ share of enterprise token consumption has also climbed from 11% to 19%—nearly doubling.

    The same report offers a second set of numbers: over the same period, the share of spending corresponding to open-source models fell from 6% to 4%. Call volume rises steadily while the revenue share contracts. This divergence is what the industry calls the “scissors gap.”

    From an engineering perspective, this data says two things:

    • The good news: Open-source models have entered enterprise production workflows; they are no longer toys in a test bed. A 19% share of calls means they are real productivity.
    • The bad news: Usage is not converting into revenue proportionally, and the value-distribution gap with closed-source models keeps widening.

    For developers and technical decision-makers, three questions need answers:

    1. Where does the scissors gap come from?
    2. Is it a transitional phenomenon or a structural trend?
    3. What should model vendors and enterprise users each do about it?

    This article attempts to answer each in turn.

    1. The Underlying Logic of the Scissors Gap: Division of Labor, Commoditization, and Efficiency

    1. Division of labor by layer: task type determines unit price

    A clear division of labor is taking shape in the global LLM market:

    • Leading closed-source vendors hold the high-value, high-difficulty complex reasoning tasks—frontier research, complex decisions, hard coding.
    • Open-source models absorb large volumes of cost-sensitive, massively parallelizable work—web content scraping and cleaning, batch information extraction, retrieval and tool calls in agent workflows, long-document summarization.

    The defining trait of these tasks is “high volume, low unit price, tolerable failure.” They naturally account for most of the token volume while bringing in only limited unit price. Put differently, the growth in call volume is itself built up from low-cost tasks. The higher the usage, the larger the share that low-priced tokens may occupy in the total.

    2. Open weights: tokens are being “commoditized”

    Open weights are a double-edged sword. They dramatically lower the trial and migration barriers for developers and help a model win de facto standard status. But once the weights are public, the same model can be hosted by multiple platforms, and basic token calls rapidly homogenize.

    Differentiation in inference services is compressed into a few dimensions: throughput and latency, context-caching capability, stability, and supporting toolchain. The token itself increasingly resembles a commodity, with its price continuously converging toward inference cost. So the decline in base call prices is not entirely the result of a price war; it is also the inevitable outcome of a well-supplied, competitive market.

    3. Efficiency gains: more tokens from the same compute

    The unit cost per token is still falling rapidly, driven mainly by the following directions:

    • Architecture: Mixture of Experts (MoE) activates only part of the parameters per token, decoupling inference cost from total parameter count.
    • Memory management: PagedAttention-style paged KV-cache management reduces memory fragmentation and improves effective throughput under high concurrency.
    • Decoding acceleration: Speculative decoding uses a small draft model to guess tokens consecutively, then has the target model verify them in parallel to raise decoding speed.
    • Reuse and quantization: Prefix caching reuses repeated prompts and context; low-precision quantization such as FP8 and INT4 reduces memory footprint and computation.

    The same GPU can serve far more tokens today than two years ago, giving vendors room to pass the savings on to customers. But the other edge of this double-edged sword is that efficiency gains spread quickly across the value chain, and the early-mover’s cost-advantage window is short.

    4. Declining effective unit price: not a price cut, but a “discount”

    There is another easily overlooked factor: billing itself is being discounted. Mainstream vendors typically cache and reuse repeated input parts (system prompts, tool definitions, long documents) and bill them at a rate clearly below standard input. Most platforms also offer discounts for batch and offline requests, and together with free tiers and limited-time offers, the actual “effective unit price” settled often falls below the list price.

    This can be written as:

    Revenue = Call volume × Effective unit price
    Effective unit price = Σ(token volume per category × corresponding rate) ÷ Total token volume

    When the share of cache-hit tokens rises and the share of batch tasks rises, the effective unit price declines even if list prices stay flat.

    5. Model routing: the amplifier

    Agent and workflow applications commonly add model routing: simple requests go to small models, and only complex requests go to large models. While this improves cost efficiency, it also structurally amplifies the scissors gap: small models carry most of the call volume, while large models retain only a small number of high-value requests.

    Summary: The scissors gap is the result of four overlapping factors:

    • In usage structure, low-cost tasks dominate;
    • In product form, tokens are being commoditized;
    • On the supply side, inference cost keeps falling;
    • In billing, caching and batch discounts dilute the unit price.

    Trying to win share through price cuts alone will only widen the scissors gap further.

    2. Consensus and Controversy: Where Does the High Share Come From

    One interpretation worth being wary of is that the share is merely “trading low price for traffic,” that user loyalty is low, and that users will switch to a cheaper model at any time.

    But for a technology to hold nearly 90% of enterprise open-source model spending, the premise is usually not “cheap” but “stable enough to sustain production.” Low price alone struggles to support large-scale deployment, because enterprises calculate total cost of ownership (TCO), not just unit price:

    • A single malformed output can break a downstream pipeline.
    • A single failed tool call can force an agent task to retry from scratch.
    • Without SLAs, rate limiting, and audit logs, the compliance team will not sign off.

    Open-source models’ ability to occupy a major place in enterprise budgets rests on continuous engineering capability. Completing the same task with less compute, or completing it with fewer tokens, is itself hard power. Call volume can be bought; enterprise budgets cannot. The scissors gap, therefore, is better understood as the combined result of the division-of-labor structure and token commoditization, not as a lack of product competitiveness.

    3. Breakthrough Path One: From Selling Tokens to Selling Delivery

    Since the gap shows up in value distribution, the direction for breaking through is to move the billing object up from the “token” to the “result.”

    Dimension Per-token billing Per-task / result billing Per-seat / subscription Private licensing
    Billing basis Input and output token count Number of successfully completed tasks, business results Number of users and plan tier Deployment license + annual service
    Unit price trend Continuously declines with inference cost Tied to customer value Relatively stable Tied to scale
    Customer perception More usage means higher cost Pay for results Predictable budget Data stays on-premises
    Vendor risk A price cut means lost revenue Must be accountable for results Usage growth is hard to monetize High delivery and operations cost
    Use cases General APIs, developer calls Agents, document processing, quality checks Office-assistant tools Finance, government, and other heavily regulated industries

    1. A layered product portfolio

    • Open-source layer: Keep weights open to win developer mindshare and expand the ecosystem base;
    • Standard API layer: Handle large-scale calls with stability and cost-effectiveness;
    • Enterprise enhancement layer: Dedicated throughput, SLA guarantees, audit logs, fine-tuning, and private deployment, priced at a contract premium.

    This way, the scale that open source brings is converted into sales leads for the enterprise edition, not just call volume.

    2. Move billing up to “results”

    Take contract review again as an example. What enterprises truly care about is “whether the risk clauses in a contract have been fully identified,” not how many tokens were consumed. Billing can then be based on the number of successfully delivered reviews, while the model side uses prompt optimization, structural constraints, and caching to hold down its own costs and capture a profit margin in between.

    3. Monetizing the ecosystem

    Open weights buy de facto standard status: being supported by default across more frameworks, applications, and platforms means distribution channels and integration opportunities. On this foundation, the enterprise edition, hosted services, and industry solutions become the main sources of revenue.

    4. Breakthrough Path Two: Value Engineering with Model Gateways and Inference Optimization

    1. Semantic caching: turn repeated computation into zero cost

    In enterprise settings, requests are highly repetitive: many users ask the same kind of question, and system prompts and knowledge-base snippets stay fixed. Adding semantic caching at the gateway layer lets you hit results before calling the model at all. The trade-off is that semantic similarity is not the same as semantic identity; for questions sensitive to factual freshness, you should set a short cache time-to-live.

    2. Context engineering: make every token more useful

    Token cost depends not only on price but also on how many tokens a task consumes. Common practices include:

    • Place relatively stable system prompts and tool definitions at the front so prefix caching can be reused;
    • Send only genuinely relevant retrieved snippets into the context;
    • Summarize and compress multi-turn dialogues;
    • Use structured output constraints (such as JSON Schema or controlled tool calls) to reduce parsing failures and retries.

    3. Speculative decoding: a new role for a classic algorithm

    The idea of speculative decoding is to use a small model derived from the same source as the target model as a draft model, guess several tokens consecutively, then hand them to the target model to verify in parallel all at once. Accepted tokens are adopted directly; rejected ones are regenerated from the point of divergence.

    The output quality is identical to autoregressive decoding; what improves is decoding speed. In output-intensive scenarios such as long-text generation and code generation, throughput and concurrency often improve markedly. The richer the ecosystem of companion small models for open-source models, the easier this optimization is to adopt.

    4. The key metric: from “price per token” to “cost per successful task”

    This is one of the most important points in this article: the token unit price is not equal to the unit task cost. At the same price, one model may succeed in a single pass, while another may fail and retry three times.

    Cost per successful task = (Input tokens × Input price + Output tokens × Output price + Retry and failure overhead) ÷ Number of successful tasks

    The table below uses illustrative data with prices as relative indices, to illustrate the difference in accounting basis:

    Model tier Token price (index) Avg output tokens First-attempt success rate Cost per successful task
    Light 1.0 1,000 70% 1,429
    Standard 5.0 800 95% 4,211
    Flagship 15.0 600 99% 9,091

    The light model has the lowest unit price, yet its cost per successful task can be about one-third that of the standard model. For tasks where failure is costly—legal, financial, and medical scenarios—the latency, manual review, and compliance risk introduced by retries push the actual cost up further. Selection should be tiered by task:

    • Use light models for fault-tolerant tasks like batch extraction and summarization;
    • Route key decisions and complex reasoning tasks to higher tiers;
    • Use routing so each does what it is best suited for.

    5. Unified governance at the gateway

    Consolidate the above capabilities into a unified model gateway. Similar to an API gateway in a microservices architecture, its common capabilities include:

    • Unified access: Hide the interface differences across model providers;
    • Smart routing: Dynamically select a model by task complexity, budget, and availability;
    • Cache layer: Exact caching and semantic caching;
    • Quota and budget: Set usage caps by team and business line;
    • Observability: Token usage, cache hit rate, P99 latency, failure rate, and cost attribution.

    6. Sample code: a cost-aware router

    1. from dataclasses import dataclass
    2. from datetime import datetime
    3. @dataclass
    4. class Route:
    5. model: str # Logical model name, mapped by the gateway to a specific backend
    6. max_output_tokens: int
    7. unit_price: float # Unit price (relative index, illustrative; replace with real pricing)
    8. SMALL, STANDARD, FLAGSHIP = "light", "standard", "flagship"
    9. ROUTES = {
    10. # Light: fault-tolerant tasks such as batch extraction and classification
    11. "extract": Route(SMALL, 1024, 1.0),
    12. # Standard: routine tasks such as summarization and code completion
    13. "summarize": Route(STANDARD, 2048, 5.0),
    14. # Flagship: complex reasoning and key decisions
    15. "reasoning": Route(FLAGSHIP, 4096, 15.0),
    16. }
    17. def choose_route(task_type: str, est_tokens: int,
    18. spent: float, budget: float) -> Route:
    19. route = ROUTES.get(task_type, ROUTES["summarize"])
    20. # When more than 80% of the budget is spent, auto-downgrade non-critical tasks
    21. if spent > budget * 0.8 and route is not ROUTES["reasoning"]:
    22. route = ROUTES["summarize"]
    23. # Never downgrade critical tasks, to avoid "saving money by causing incidents"
    24. if route.model == SMALL and spent > budget * 0.9:
    25. route = ROUTES["summarize"]
    26. return route
    27. def estimate_cost(route: Route, in_toks: int, cache_ratio: float = 0.0) -> float:
    28. # cache_ratio: cache hit rate on input tokens; cached tokens are billed at 20% of the input price (illustrative)
    29. hit = in_toks * cache_ratio
    30. cost = (hit * 0.2 + (in_toks - hit)) * route.unit_price
    31. cost += route.max_output_tokens * route.unit_price
    32. return cost
    33. if __name__ == "__main__":
    34. spent, budget = 300.0, 400.0 # Illustrative values
    35. r = choose_route("summarize", est_tokens=5000, spent=spent, budget=budget)
    36. print(r.model, round(estimate_cost(r, in_toks=5000, cache_ratio=0.6), 2))
    37. # Output: standard 27200.0

    7. Sample code: accounting and cost calculation

    1. from dataclasses import dataclass, field
    2. from datetime import datetime
    3. import hashlib
    4. import json
    5. @dataclass
    6. class CallRecord:
    7. task_id: str
    8. ts: datetime
    9. model: str
    10. cache_hit: int # Input tokens served from cache
    11. in_toks: int # Input tokens that missed the cache
    12. out_toks: int # Output tokens
    13. unit_cost: float # Cost per token (illustrative)
    14. ok: bool # Whether the output passed validation
    15. class CostLedger:
    16. """Unified accounting: use successful task count, not token count, as the accounting basis"""
    17. def __init__(self):
    18. self.rows: list[CallRecord] = []
    19. def record(self, rec: CallRecord):
    20. self.rows.append(rec)
    21. def report(self, start: datetime, end: datetime):
    22. win = [r for r in self.rows if start <= r.ts < end]
    23. total = sum((r.cache_hit * 0.2 + r.in_toks + r.out_toks)
    24. * r.unit_cost for r in win)
    25. ok = [r for r in win if r.ok]
    26. avg_out = sum(r.out_toks for r in ok) / max(len(ok), 1)
    27. hit = sum(r.cache_hit for r in win)
    28. all_in = sum(r.cache_hit + r.in_toks for r in win)
    29. return {
    30. "total_calls": len(win),
    31. "successful_tasks": len(ok),
    32. "cost_per_successful_task": total / len(ok) if ok else None,
    33. "cache_hit_rate": hit / all_in if all_in else 0,
    34. "avg_output_tokens": avg_out,
    35. }
    36. if __name__ == "__main__":
    37. r = CallRecord("task-001", datetime.now(), "standard",
    38. cache_hit=3000, in_toks=2000, out_toks=900,
    39. unit_cost=5e-6, ok=True)
    40. ledger = CostLedger()
    41. ledger.record(r)
    42. now = datetime.now()
    43. print(ledger.report(now, now))

    This code embodies several practical points:

    • Unified basis: Use “cost per successful task” as the baseline for horizontal comparison and optimization;
    • Quantifiable caching: Account for cache hits separately so that optimization gains can be measured;
    • Governable output: Track average output tokens to constrain verbose prompts and output bloat.

    In a real system, the accounting function can be wrapped as a decorator around the model-call function to collect data non-invasively. Cost accounting should also include the overhead of auxiliary stages such as vector retrieval and embedding calls.

    5. Implementation Architecture: Three-Layer Coordination

    Layer Core responsibility Corresponding cloud capabilities
    Access Unified authentication, model routing, budget control A gateway running on a container service
    Capability Semantic caching, context management, fine-tuning, batch processing High-performance storage, message queue, data processing platform
    Operations Logging, metrics, audit, alerting Log service, monitoring and alerting

    The access layer solves “which model to use and how much to spend”; the capability layer solves “how to complete tasks with fewer tokens”; the operations layer solves “where the money went and how well it performed.” Only when the three layers work together can the pressure of declining unit prices be turned into the drive for greater efficiency.

    6. Selection Checklist

    Model provider (API service provider)

    • Has a premium product line such as enterprise editions or private deployment, rather than relying only on standard APIs?
    • Supports differentiated billing such as caching and batching, and feeds customers’ actual savings back as a reason to renew?
    • Has inference-optimization and elastic-scheduling capabilities to defend margins during price-cut cycles?
    • Evaluates its own competitiveness by “cost per successful task” rather than unit price?

    Model user (enterprise developer)

    • Uses “cost per successful task” rather than token unit price as the selection basis?
    • Applies task-tiered routing with budget thresholds and alerts?
    • Has caching and prefix reuse enabled, and monitors the cache hit rate?
    • Keeps output tokens under control, and tracks the parsing failure rate?
    • For self-hosted inference, has fully accounted for the total cost including operations?

    Conclusion

    Open-source models have firmly established themselves in enterprises’ technology stacks worldwide, but “being used” is not the same as “being priced.” The essence of the scissors gap is this: call volume is growing, but value capture lags behind. The real breakthrough lies not in whether the unit price is high or low, but in whether you can convert tokens into tasks, tasks into results, and results into sustainable revenue. For users, the same shift is also an opportunity: token commoditization makes tokens increasingly like water and electricity, and enterprises will compete on who can more efficiently convert these “compute utilities” into business outcomes.