Open-Source LLMs: Rising Usage, Falling Unit Prices, and How Vendors Can Break Through

    This article examines why open-source LLM usage keeps climbing while unit prices keep falling, and what that scissors gap means for vendors trying to stay profitable. It breaks the gap down into five key causes, then lays out a system for calculating true per-token cost and effective unit price with a worked example. From there it presents three layers of breakthrough — cutting inference costs, restructuring metering and billing toward value-aligned units, and productizing model capability — and closes with a three-step implementation roadmap with concrete acceptance metrics and common pitfalls. Readers will learn how to diagnose margin erosion and move from selling API access to selling business value.

    1. The Symptom: Call Volume Is Rising, Unit Revenue Is Falling

    Open-weight models have significantly lowered the barrier for enterprises to adopt large-model capabilities. Companies can call models through managed services, deploy inference in their own environments, or switch flexibly across multiple providers. As use cases like summarization, extraction, classification, batch processing, and agent loops grow, call volume keeps climbing. But unit token pricing has not risen in step — in some scenarios it is under sustained pressure.

    From the enterprise buying side, spending shows two traits. First, total AI investment continues to grow, but budgets are more cautious, with sharper focus on unit cost, latency, stability, and portability. Second, open models account for a rising share of calls, but their share of spending does not necessarily keep pace — lightweight models in particular take a large portion of high-frequency standard tasks. Note that different institutions use different reporting periods, sample scopes, definitions of “open model,” and token accounting conventions, so exact ratios may vary. Do not read too much into a single number; watch the trend instead: high-frequency workloads are being routed to lower-cost models.

    The economics behind the share shift is straightforward. Open models have lower marginal cost and, at equal capability, lower market prices. At the same time, they mostly handle high-frequency, standardized tasks such as summarization, extraction, classification, and batch processing, so the unit price enterprises are willing to pay is naturally limited. Rising usage and falling unit price are the combined result of cost-structure differences and scenario value density.

    What really determines the ceiling for open-model vendors is the next question: can market share convert into profit, and can call volume convert into revenue? This is not merely a pricing problem — it needs to be broken down systematically across four layers: inference technology, cost accounting, metering and billing, and product form.

    2. Where the Scissors Gap Comes From: Five Key Causes and Their Priority

    To close the scissors gap, first understand how it forms.

    1. The price anchor keeps moving down. Open weights can be run by enterprises at low cost on their own hardware, which gives them a clear price reference. To keep self-hosting customers, managed services must price close to what enterprises would spend building it themselves, or offer differentiated value that self-hosting struggles to match — elasticity, managed operations, security and compliance, stable SLAs. As a result, average selling prices for open models stay under long-term pressure.

    2. Shifting usage mix: agents and batch processing dilute effective unit price. Much of today’s token growth comes from agent loops and batch tasks, and this kind of traffic usually gets pricing discounts. Agents repeatedly read long context, pushing average input length up; large volumes of offline tasks can move to cheap batch channels or run during off-peak business hours; a declining share of conversational interaction weakens the weight of high-margin scenarios. The result: total token volume balloons while the weighted effective unit price is diluted.

    3. Client-side optimization: caching, routing, and trimming. Enterprise inference engineering has improved rapidly, and semantic caching, prefix caching, model routing, and context trimming are now common practice. Every cache hit, every reduction in model size, and every compression of historical context reduces the billable calls on the provider’s invoice.

    4. Pricing model evolution: input/output separation and tiered pricing. Mainstream services price input and output separately and give discounts for cache hits and batch processing. Once discounts stack, a single request that processes thousands of tokens may be billed for only a fraction of the raw token count. The more granular the pricing, the more enterprises optimize their call paths, and the more easily unit revenue is diluted.

    5. Market share is concentrated while revenue is dispersed. Open-model share is highly concentrated. Leading products spread fixed costs across scale and can expand share at lower prices. If smaller vendors keep following the price cuts, they fall into a “the more we sell, the more we lose” cycle.

    To locate your own problem, place the five causes in a priority matrix:

    Priority Key cause Identification signal Response direction
    High Price anchor moving down Customers frequently compare with self-hosting cost Strengthen managed value, avoid pure price competition
    High Rising share of batch and agents Longer average input, more off-peak calls Introduce reserved throughput, off-peak scheduling, task-based billing
    High Client-side caching and routing Rising cache hit rate, falling billable tokens Design cache discount rules, shift to value-based billing
    Medium Stacked pricing discounts Low revenue coefficient per request Make conventions transparent, optimize package structure
    Medium Market concentration and price cuts Leader prices pull the market Deepen scenarios, ecosystem cooperation, differentiated capability

    3. Do the Math: Unit Economics Is the Real Test

    3.1 Break down unit cost

    A basic capability of a model service is to quantify the true cost of each call:

    Unit token cost = (inference compute cost + storage and bandwidth cost + operations and labor cost) / total billable tokens

    Here, inference compute cost is usually GPU-hour cost divided by effective throughput. Storage and bandwidth cost includes object storage, message queues, and other base components, which are also non-negligible at large concurrency. Operations and labor cost includes capacity planning, incident handling, version upgrades, and monitoring and alerting.

    A boundary condition must be added: this formula suits internal management accounting and should not be read as an industry-standard estimation formula. In practice, you need to specify model parameter count, GPU model, concurrency, context length, batching strategy, cache hit rate, billing conventions, tenant cost allocation, and input/output compute coefficients. Otherwise, the same formula can yield completely different conclusions under different deployment conditions.

    The most critical variable is “effective throughput,” not GPU peak compute. Insufficient VRAM limiting batch size, overly long context squeezing KV cache space, and long-prompt queues blocking requests can all make actual throughput fall well short of the theoretical ceiling.

    3.2 Account for effective unit price: an illustrative example

    The following example only illustrates the calculation logic; all parameters are assumed and do not represent any specific product’s data.

    Suppose a request has 3000 input tokens and 200 output tokens, a cache hit rate of 60%, an output compute coefficient of 4, a batch ratio of 30%, and a batch discount of 50%.

    The raw token total is 3200. Effective token amount: 3000 × (1 - 60%) + 200 × 4 = 2000. The average price coefficient is about 2000 / 3200 = 0.625. After stacking the batch discount, the revenue coefficient is about: 0.7 × 0.625 + 0.3 × 0.625 × 0.5 = 0.531. That is, this request’s revenue is roughly equivalent to 53% of full-rate billing on raw tokens.

    This example shows three things. First, stacking cache hit rate and batch discounts significantly lowers single-request revenue. Second, it is not hard for providers to bring costs down to the same level, but if revenue falls in step with cost, gross margin does not necessarily improve. Third, the way out is for the cost curve to drop faster than client-side optimization, while the value unit migrates from tokens to more stable billing objects.

    4. First Breakthrough: Cut Inference Costs So “Low Price” Is Sustainable

    Unit cost sets the price floor. Inference optimization is the foundation for competing on cost-effectiveness. Several mature technical directions already exist in the industry:

    Technique Mechanism Cost impact
    Low-bit quantization Represent weights and activations at lower precision, reducing VRAM usage and memory-access overhead Increases per-GPU batch size and throughput
    Sparsification Compute only activated parameter paths Reduces per-token compute
    Speculative decoding A small model or multiple heads predict several tokens in parallel, and the large model verifies them at once Boosts decoding throughput; requires an acceptance-rate tradeoff
    Continuous batching Dynamically swap requests in and out at each iteration step Significantly raises GPU utilization
    Paged KV cache management Store cache in blocks, reducing fragmentation and redundant copies Supports longer context and larger concurrency
    Chunked prefill Split long prompts into chunks and interleave scheduling with decoding Lowers average first-token latency and tail latency
    Prefill-decode disaggregation Deploy compute-intensive and memory-access-intensive stages separately Eases on-demand scaling
    Cache reuse Reuse system prompts, document prefixes, or repeated question results Directly reduces repeated computation

    When rolling out, build a checklist so optimization does not become guesswork tuning:

    1. Baseline throughput: tokens/s, first-token latency, tail latency, concurrency ceiling.
    2. Per-token cost: broken down by model, scenario, and tenant.
    3. GPU utilization and VRAM utilization: to judge whether the bottleneck is compute, memory, or scheduling.
    4. Cache hit rate: distinguishing prefix cache, semantic cache, and result cache.
    5. Batch ratio and wait time: avoid improving batch throughput while latency worsens.
    6. Rollback strategy: model versions, quantization versions, and routing-policy changes should all keep a rollback path.

    Operational gains are often more immediate: schedule offline tasks off-peak to avoid peak hours; elastic scaling that auto-expands at peak and auto-shrinks off-peak; use long-tail or smaller-spec models to handle low-complexity requests. These actions pay off quickly and make a good first lever for cost reduction.

    5. Second Breakthrough: Restructure Metering and Billing

    Technical optimization lowers cost, but metering and billing determine how much of that cost turns into revenue.

    5.1 How to bill: align billing units with value units

    The closer the billing object is to the business value the customer perceives, the more stable the revenue. Common pricing methods:

    Pricing dimension Suitable scenarios Characteristics
    Per token General API calls Fine-grained and transparent, but vulnerable to client-side optimization
    Subscription and credit packs Development teams, SMBs Predictable revenue, smooths unit-price fluctuations
    Reserved throughput Production systems with steady traffic Customers lock capacity, provider locks revenue
    Per task / per outcome Document processing, batch generation, agent tasks Tied directly to value, lower sensitivity
    Dedicated / private deployment Data-sensitive, high-compliance industries Priced as license or service bundles, avoids pure token comparison

    For high-frequency, standardized scenarios, per-task or per-outcome billing more easily stabilizes revenue. A contract review, a batch of document extraction, or a completed agent task are value units customers find easier to understand. But the prerequisite is that each value unit’s cost can be accurately attributed — otherwise, the more granular the pricing, the more hidden the losses.

    5.2 How to choose a pricing method: four questions

    When a customer asks “how do we bill,” start from four questions:

    1. Is the scenario stable? If call volume fluctuates widely, per-usage billing is more natural; if production tasks are steady, reserved throughput fits better.
    2. What is the customer’s budget cycle? Customers on project budgets suit credit packs or project-based pricing; long-term platform budgets suit subscription or reservation.
    3. Can cost be attributed? If you cannot attribute cost by model, tenant, and scenario, do not rush into per-task or per-outcome billing.
    4. Is it sensitive to client-side optimization? If customers heavily use caching, routing, and trimming, pure token billing is easily diluted and should shift to value units or package locks.

    5.3 Observability of cost and revenue

    The observability system should cover at least three layers:

    • Call-chain tracing: which models a business request passes through, how many retries, whether cache hits occur.
    • Cost accounting: aggregate tokens and compute cost by tenant, app, model, and scenario, and compute per-request gross margin.
    • Quotas and limits: tenant-level quotas, rate limits, and abnormal-usage alerts to prevent a single script task from eroding overall margin.

    Also, billing conventions should be transparent. Separate input/output pricing, cache discounts, batch discounts, minimum fees, and overage unit prices should all be clearly explained to customers. Otherwise, the goodwill from price cuts may be offset by billing disputes.

    6. Third Breakthrough: Productize — Turn “Selling API” into “Selling Capability”

    A token is not a product; it is just a unit for measuring capability. What enterprises are willing to keep paying for is complete capability built around the model that can embed into existing workflows.

    First is the model gateway. Unified access to multiple models, supporting routing, canary release, failover, rate limiting, and auditing, helps customers shed dependence on a single channel. Second is knowledge augmentation, that is, retrieval-augmented generation. Document parsing, chunking, vectorization, hybrid retrieval, reranking, and citation tracing directly affect answer accuracy and trustworthiness. Third is agent and workflow orchestration. Tool calling, task planning, state management, and human-review nodes let models genuinely participate in business processes rather than stopping at Q&A.

    Fourth is parameter-efficient fine-tuning and customization. Methods like low-rank adaptation adapt general models to vertical domains at low cost, with data governance and version management. Fifth is evaluation and continuous iteration. Build evaluation sets covering accuracy, faithfulness, and latency, run regression tests after model or prompt changes, and use data to prove improvements. Sixth is security and compliance. Content safety, data isolation, access control, and operation auditing are often entry conditions for enterprise procurement, not bonus points.

    These capabilities can build on general capabilities of mainstream cloud platforms — object storage, message queues, container platforms, log services, monitoring and alerting, plus model serving, model development tools, and agent development tools. Open-model vendors need not build all the infrastructure themselves; they can focus on the model and inference layer and hand infrastructure and scenario capabilities to ecosystem partners, reaching more paying scenarios while keeping their model advantage.

    7. Implementation Roadmap: Three Steps and Acceptance Metrics

    Step one is to take stock, ideally within 1–2 months. The goal is to establish call-chain tracing and cost accounting, clarifying per-token cost, gross margin, and cache hit rate by model, scenario, and tenant. Acceptance metrics can include: cost-accounting coverage of key scenarios above 90%, a clear share of scenarios where margin can be calculated, an abnormal-call detection mechanism in place, and a formed list of loss-making and high-value scenarios.

    Step two is cost reduction and repricing, ideally over 2–4 months. First apply inference optimization to bottlenecks, then replace part of pure per-usage billing with reserved throughput, credit packs, or per-task billing, and validate revenue changes during a canary period. Acceptance metrics can include: lower per-token cost, higher cache hit rate, optimized batch ratio, controllable revenue change during canary, explainable gross-margin change, stable failure rate, and clear rollback conditions.

    Step three is productization and ecosystem building, to be pursued continuously. Package capabilities into reusable product components, accumulate evaluation baselines, industry templates, and delivery tools, and expand coverage through partner integration and channel distribution. Acceptance metrics can include: template reuse rate, evaluation pass rate, customer task completion rate, delivery cycle, repurchase rate, and ecosystem partner contribution share.

    Also watch for three pitfalls. First is price-only thinking. Cutting prices purely for volume may lose both revenue and margin. Second is parameter-only thinking. Stacking model specs while ignoring inference cost makes unit cost spiral out of control. Third is API-only thinking. Staying at interface supply without going deep into the customer’s business flow leaves you easily replaced by more complete solutions.

    8. Conclusion

    Enterprise practice suggests the trend of rapid open-model adoption is strong. The scissors gap between rising usage and falling unit pricing is less a crisis than the market repricing “the model as infrastructure.”

    For open-model vendors, there are basically three paths: first, use inference engineering to push the cost curve down faster than client-side optimization; second, use metering and billing aligned with business value to stabilize revenue; third, use productization and ecosystem to embed model capability into real processes. All three are indispensable.

    When model capability becomes as ubiquitous as water and electricity, what is truly scarce is no longer tokens, but the ability to reliably, securely, and cheaply turn tokens into business outcomes. This is also the inevitable direction for open-model vendors moving from “selling API” to “selling value.”