LLM Inference Cost Breakdown: From Energy per Token to Business Model Reassessment

1. The Starting Point: Why It’s Worth Calculating How Many Tokens a Kilowatt-Hour Generates

As large language models (LLMs) move into large-scale application, the focus of cost discussions has gradually shifted from the training side to the inference side. The reason is straightforward: training is a one-time investment, while inference costs keep accumulating with user base, call frequency, and context length. For developers, how much compute and how much electricity a single API call actually consumes ultimately determines whether a business can maintain healthy margins as it scales.

“How many tokens can one kilowatt-hour generate” is not a precise constant. It is an engineering range shaped by model size, hardware platform, inference framework, batching strategy, and the distribution of sequence lengths. But precisely because a measurable range exists, we can build a ground-up understanding of inference costs starting from the physical layer.

2. The Basic Breakdown of Inference Energy Consumption

2.1 From Per-Card Power to Effective Power

Take today’s mainstream high-performance AI accelerators: their full-load power draw typically sits in the several-hundred-watt range. Even if a card is rated at 700W, its effective per-card power in a real datacenter is usually higher, for several reasons:

• Datacenter cooling systems consume significant amounts of electricity;
• Network equipment, host CPUs, memory, and storage are all amortized across each accelerator card;
• Power conversion itself introduces losses.

PUE (Power Usage Effectiveness) is the standard measure of total datacenter energy consumption relative to IT equipment energy consumption. With a PUE of roughly 1.3, a 700W per-card power draw is amplified to roughly 900W of effective power. In other words, each card actually consumes about 900 joules of energy every second.

2.2 Energy Cost of Output Tokens

Generating output tokens is a sequential, token-by-token decoding process. In high-throughput batch inference scenarios, the aggregate output speed of a single card can range from several hundred to several thousand tokens per second. Using 1,000 output tokens per second as a neutral assumption, the energy consumed per output token is:

900 J/s ÷ 1,000 tokens/s = 0.9 J/token

One kilowatt-hour equals 3.6 million joules, therefore:

3,600,000 J ÷ 0.9 J/token ≈ 4,000,000 output tokens

This can be read as follows: under moderate throughput and high batching efficiency, one kilowatt-hour can support the generation of roughly 4 million output tokens. The realistic range may span from 2 million to more than 10 million, depending on model size, sequence length, and batch saturation.

2.3 The Energy Difference for Input Tokens

Unlike output generation, the prefill stage on the input side is far more parallelizable. Because an input sequence can be fed into the model and computed in a single parallel pass, per-card throughput is typically much higher than in token-by-token decoding. Using 10,000 tokens per second as a neutral assumption, the energy per input token is:

900 J/s ÷ 10,000 tokens/s = 0.09 J/token

In other words, the per-token energy cost on the input side is roughly one-tenth of the output side. The number of input tokens one kilowatt-hour can process is approximately:

3,600,000 J ÷ 0.09 J/token ≈ 40,000,000 input tokens

This difference also explains why many APIs price input and output tokens asymmetrically: the input side naturally achieves higher compute utilization efficiency.

3. From Energy Consumption to Unit Economics

3.1 Theoretical Revenue Conversion

Setting aside model R&D, platform operations, staffing, and marketing costs, and converting from electricity consumption alone, we can construct an “energy-perspective” unit economics model.

Assume output pricing of $50 per million tokens and input pricing of $10 per million tokens. Then:

• On the output side, one kilowatt-hour corresponds to roughly 4 million tokens, or about $200 in theoretical revenue;
• On the input side, one kilowatt-hour corresponds to roughly 40 million tokens, or about $400 in theoretical revenue.

Blending the two at a typical input-to-output token ratio of 3:1, the combined theoretical revenue per kilowatt-hour works out to roughly $250 to $300.

3.2 Uncertainty and Reasonable Ranges

These calculations depend on multiple assumptions, with throughput being the most sensitive. If per-card output speed falls to 500 tokens per second, the number of output tokens per kilowatt-hour is cut in half; if it rises to 3,000 tokens per second, the result triples. A more reasonable estimate is therefore a theoretical revenue of $100 to $800 per kilowatt-hour.

The point of this range is not to produce a precise number, but to reveal something important: under idealized conditions, the “energy cost” of LLM inference is far below current market pricing. In other words, today’s API prices embed substantial non-energy costs, including R&D amortization, platform construction, continuous optimization, commercial channels, and risk premiums.

4. Implications for Business Models and the Market Landscape

4.1 High Prices Are Not Driven by Electricity Costs

If a single kilowatt-hour can support the generation of millions of tokens even under the most conservative assumptions, then judged purely on electricity costs, the output pricing of many current models sits far above physical cost. What this tells us is that pricing is determined not by power bills, but by the scarcity of model capability, the payback period on R&D investment, and the pace of market competition.

As model capabilities become homogeneous and scarcity fades, prices fall rapidly. The decline is not because energy consumption suddenly dropped; it is because the pricing logic shifts from “covering heavy R&D costs” to “chasing market share and long-term ecosystem lock-in.”

4.2 The Closed-Source vs. Open-Source Competitive Logic Will Change

If the unit economics of inference are already highly elastic, the window in which closed-source models can sustain a premium through technical exclusivity will shorten. The reasons include:

• Capability gaps narrow as training methods converge;
• Cost-sensitive developer use cases exist in large numbers;
• Inference optimization techniques can dramatically lower the cost of serving the same model.

That said, this does not mean closed-source models have no room to survive. In scenarios involving ultra-complex tasks, enterprise-grade compliance, guaranteed service stability, data security, and deep customization, closed-source models may still deliver irreplaceable value.

4.3 Inference Cost Optimization Is Becoming a Key Engineering Capability

From an engineering standpoint, the levers for reducing per-token cost are plentiful:

• Increase batch saturation so a single card serves more concurrent requests;
• Optimize KV cache management to reduce redundant computation;
• Apply quantization, distillation, and sparsification to lower compute volume;
• Route requests by task complexity so large models do not handle simple requests;
• Rebalance hardware resource allocation between the prefill and decoding phases.

The shared goal of all these techniques is to make one kilowatt-hour buy more effective tokens, preserving profit margins under price competition.

5. How Developers Should Build Their Own Cost Models

Rather than relying on vendor-published prices, developers should build an evolvable cost assessment framework grounded in their own business. The key variables to track:

Average input tokens per request
Average output tokens per request
Input-to-output token ratio
Peak QPS and average QPS
Context length distribution
Latency tolerance
Batching feasibility

By building estimation tables along these dimensions, developers can more accurately compare the real cost differences across models, deployment approaches, and providers. The same framework also helps teams make better-informed choices between “tuning the model” and “optimizing the architecture.”

6. Conclusion: Understanding LLM Economics from an Energy Perspective

“How many tokens can one kilowatt-hour generate” looks like simple arithmetic, but behind it lies a deep interplay between inference efficiency, hardware utilization, and commercial pricing strategy. Current LLM API prices are still determined largely by capability scarcity and R&D amortization, not by marginal inference cost.

As inference optimization techniques continue to advance, batching efficiency keeps improving, and capability gaps between models narrow, the marginal cost per token still has room to fall further. For developers and enterprises, what matters is not memorizing a specific number, but mastering a complete analytical framework that runs from energy consumption and throughput through batching to pricing. Only with such a framework can you make more robust technical and business judgments in a fast-changing market.