LLM Inference Cost Economics: Token Pricing and Private Deployment Trends
This article examines how converging LLM capabilities are turning tokens into standardized commodities, estimates token output per kWh of electricity, and analyzes the implications for API pricing, closed-source business models, and open-source private deployment. It also discusses enterprise break-even considerations for self-hosting and the long-term value of open-source ecosystems.
1. Background: Converging LLM Performance and the Commoditization of Tokens
Over the past two years, benchmark competition in the LLM field has continued to intensify. New flagship models take turns topping various benchmark leaderboards, while the score gap between first and second place keeps shrinking. This sends an important signal: the general capabilities of leading LLMs are converging.
When differences among models in mainstream scenarios such as general Q&A, code generation, and logical reasoning shrink to a level users can barely perceive, competition shifts from “capability level” to “cost level.” In other words, the output of LLM inference services—tokens—is becoming a standardized commodity similar to water, electricity, or network traffic:
- Homogeneous: Models of the same tier from different vendors can substitute for one another on most tasks.
- Metered: Both input and output are billed by token, with transparent and comparable unit prices.
- Strong scale effects: Inference costs continue to decline as cluster scale and utilization increase.
In this landscape, understanding the true cost structure of tokens becomes key to judging the direction of the LLM market. Below, we use a simplified economic model to estimate: how many tokens can one kWh of electricity produce?
2. Estimation Model: From Power Consumption to Token Output
2.1 Step 1: Token Capacity per Unit of Electricity
We make the following key assumptions for a typical LLM inference scenario:
- Effective power per card: A high-end inference accelerator itself consumes about 700W. Including data center cooling, networking, storage, and management nodes (PUE about 1.3), the effective power is calculated at about 900W per card.
- Aggregate output throughput: When a frontier LLM performs large-batch concurrent inference, the aggregate output speed per card is about 1,000 tokens/second (with a conservative range of 500–3,000 tokens/second).
From this, the energy consumption per generated output token is approximately:
900 joules/second ÷ 1,000 tokens/second = 0.9 joules/token
One kWh equals 3.6 million joules, so:
3,600,000 ÷ 0.9 ≈ 4,000,000 output tokens / kWh
That is, one kWh can support the generation of about 4 million output tokens.
2.2 Step 2: Input-Side Energy Consumption Is Significantly Lower
Note that inference consists of two stages:
- Prefill stage: Processes input tokens with highly parallel computation. Throughput is much higher than token-by-token decode output, reaching about 10,000 tokens/second per card, with unit energy consumption of about 0.09 joules/token.
- Decode stage: Generates output token by token, limited by memory bandwidth. Throughput is lower, and this is the main energy-consuming stage.
Based on this estimate, one kWh can process about 40 million input tokens.
2.3 Step 3: Converting to Market Price
Assume typical market pricing of about $50 per million output tokens and about $10 per million input tokens:
| Type | Capacity per kWh | Reference Price | Value per kWh |
|---|---|---|---|
| Output tokens | ~4 million | $50/million | ~$200 |
| Input tokens | ~40 million | $10/million | ~$400 |
Using a mixed input/output ratio of about 3:1 in actual business, the token value corresponding to one kWh is approximately $250–300, or about RMB 1,800–2,200. Considering uncertainties such as throughput level, model size, and batching efficiency, a reasonable range is roughly $100–800 per kWh (about RMB 700–6,000).
3. Market Implications of the Cost Structure
3.1 Long-Term Marginal Cost of Tokens Approaches Electricity Cost
The core conclusion of the above estimate is that when LLMs enter large-scale global application, and hardware depreciation is amortized at scale, the main variable cost of inference is electricity. Industrial electricity costs only a few tenths of RMB per kWh, while current token market prices imply a “value per kWh” of hundreds or even thousands of RMB. The huge spread is essentially the scarcity premium of models, R&D cost amortization, and vendor profit.
This means two things:
- Prices still have significant room to fall: With inference engine optimization, the spread of dedicated inference chips, and intensifying competition, token prices will continue to decline and converge toward the cost line.
- The pure API resale business model is highly fragile: Intermediary services that profit from “reselling token price differences” will struggle to sustain themselves once prices become transparent.
3.2 Structural Challenges for Closed-Source Models
Under an extremely simplified economic model, a closed-source business model that relies solely on selling tokens faces three pressures:
- Free substitutes from open-source models: When open-source models catch up with closed-source models and enterprises can deploy them independently, the premium of closed-source APIs loses support.
- Price war is inevitable: Markets for homogeneous goods ultimately move toward cost competition, while closed-source vendors also carry the burden of recovering huge R&D and training costs.
- Unable to enter the private deployment market: To protect model weights, closed-source vendors find it difficult to provide enterprises with complete local deployment solutions, voluntarily giving up the most profitable part of the enterprise market.
Of course, in reality, closed-source models still have value in compliance assurance, managed operations, and windows of frontier capability. The claim that they are “certain to die” is somewhat extreme. But in the long run, a single revenue structure based only on API subscriptions is indeed difficult to sustain, and diversified monetization is inevitable.
3.3 Private Deployment: The Second Growth Curve for Open-Source Models
In contrast to the closed-source model, the commercialization paths for open-source LLMs are more diverse:
- Private deployment services: Once token usage reaches a certain scale, the inference cost of self-deployment by large enterprises will be significantly lower than pay-as-you-go API calls. Requirements such as data security, intranet compliance, and low latency also naturally point to private deployment.
- Tuning and customization: Engineering services around open-source models, such as domain fine-tuning, distillation and pruning, retrieval augmentation, and inference acceleration, represent a market closed-source vendors cannot enter.
- Channel and managed hosting: For SMEs and individual developers, an intermediary layer that provides hosted inference services based on open-source models can still earn reasonable channel fees.
For enterprises, the private deployment decision can be simplified to a break-even question: when monthly API call costs exceed the total monthly cost of a self-built inference cluster (hardware amortization + electricity + operations labor), switching becomes economically viable. Based on the above cost model, this break-even point is not difficult to reach in high-frequency usage scenarios. Many enterprises in the industry have begun migrating core business workloads back to self-built or dedicated cloud inference environments. General infrastructure capabilities provided by cloud platforms, such as container orchestration, elastic scaling, object storage, and logging and monitoring, also significantly lower the engineering barrier to private deployment.
4. Long-Term Value of the Open-Source Ecosystem
4.1 From “Usable” to “Good to Use”
The significance of open-source LLMs is no longer limited to the early small specialized models of a few hundred MB. Today’s open-source ecosystem covers a complete capability spectrum from general conversation and code generation to multimodal understanding. With mature inference frameworks and quantization tools, enterprises and individual developers can already use open-source solutions to replace the AI capabilities in most commercial software. This “de-locking” value cannot be captured by simple price comparisons alone.
4.2 Ecosystem Spillover of Technology Routes
More broadly, competition in LLMs is not just competition among individual products, but competition over technology routes and ecosystem standards. When an open-source technology route is adopted by more countries and regions, the toolchains, talent systems, and industry solutions formed around it generate strong network effects: the more new users join, the richer the models, plugins, and best practices in the ecosystem become, and the higher the migration cost. Once a vast global market of billions of people connects to an open-source technology system, its commercial value will far exceed the subscription revenue of a single product.
5. Conclusion
Returning to the initial question: how much does topping a benchmark really matter? In an era of converging model capabilities, the marginal value of leaderboard rankings is declining. What truly determines the market landscape is cost structure, business model, and ecosystem openness. The estimate that one kWh produces millions of tokens tells us that tokens will ultimately return to the commodity essence of “electricity + compute.” In this long price convergence, the side with an open-source foundation and the ability to provide private deployment and deep customization services clearly holds more leverage. For developers and enterprise decision-makers, now is the best window to reassess LLM infrastructure choices and build self-controlled inference capabilities.
