How Many Tokens Can One Kilowatt-Hour Generate? The Economics of LLM Inference Cost

1. Background: When Performance Converges, Cost Becomes the New Battleground

Over the past two years, the gap between leading large language models on general benchmarks has continued to narrow, with different models trading wins on different niche tasks. When “being smarter” no longer confers a sustained generational advantage, “being cheaper” becomes a quantifiable, tangible form of competitiveness.

Metered tokens are becoming a resource much like electricity or water: users don’t care who operates the power plant—they care about the price per kilowatt-hour and the price per million tokens. The core question of industry competition thus reduces to a simple economic proposition: what determines the marginal cost of intelligence?

The answer ultimately comes down to energy. This article uses “how many tokens can one kilowatt-hour produce” as a starting point to work through the math.

2. Core Calculation: How Many Tokens Can One Kilowatt-Hour Generate?

2.1 Key Assumptions

The calculation targets high-volume inference scenarios (high GPU utilization, requests fully queued) and adopts typical industry-order-of-magnitude parameters:

• Hardware: Data-center-grade GPU accelerator cards, with per-card power draw estimated at around 700W (varies by card type; a conservative representative value is used here);
• System overhead: Beyond the compute card, the data center includes cooling, power distribution losses, networking, CPU/memory, and other overhead. Using a PUE (Power Usage Effectiveness) of ≈1.3, the effective power per card is approximately 700W × 1.3 ≈ 900W;
• Output throughput: Under high-volume inference, aggregate output speed per card is estimated at roughly 1,000 tokens/second (a reasonable industry range is 500–3,000 tokens/second).

2.2 Output Side: Approximately 0.9 Joules per Token

Output tokens are generated one at a time through autoregressive decoding, so the energy calculation is straightforward:

900 joules/second ÷ 1,000 tokens/second = 0.9 joules/token

One kilowatt-hour = 3.6 million joules, therefore:

3,600,000 ÷ 0.9 ≈ 4 million output tokens per kWh

2.3 Input Side: The Energy Advantage of Prefill

The input side (prefill) involves parallel computation: a single forward pass processes the entire prompt, achieving throughput an order of magnitude higher than autoregressive output—roughly 10,000 tokens/second per card. The corresponding energy consumption is only 0.09 joules/token, meaning one kWh can process approximately 40 million input tokens.

2.4 Converting to Revenue: The “Intelligence Value” of One Kilowatt-Hour

Using common pricing tiers for frontier models in the current market (approximately $50 per million output tokens and $10 per million input tokens):

• Output side: 4 million × 50/1 million ≈ $200 per kWh
• Input side: 40 million × 10/1 million ≈ $400 per kWh

Blending by typical input-to-output ratios: a 1:1 mix yields roughly $300 per kWh; a 3:1 mix (conversational, RAG, and other scenarios where input far exceeds output) yields roughly $350 per kWh. Factoring in throughput variation across the 500–3,000 tokens/second range, a reasonable range is approximately $100–$800 per kWh.

3. Code Reproduction: Codifying the Calculation Model

A few dozen lines of Python can fully reproduce the above calculation and allow readers to adjust parameters for their own sensitivity experiments:

# Token output per kilowatt-hour (simplified model)
PUE = 1.3                          # Data center power usage effectiveness
CARD_POWER_W = 700                 # Per-card power draw (W)
SYS_POWER_W = CARD_POWER_W * PUE   # Effective power ~900W
OUT_TOKENS_PER_S = 1000            # Output throughput (tokens/s/card)
IN_TOKENS_PER_S = 10000            # Prefill throughput (tokens/s/card)
JOULE_PER_KWH = 3.6e6              # One kWh = 3.6 million joules
e_out = SYS_POWER_W / OUT_TOKENS_PER_S   # Output energy J/token
e_in  = SYS_POWER_W / IN_TOKENS_PER_S    # Input energy J/token
out_per_kwh = JOULE_PER_KWH / e_out
in_per_kwh  = JOULE_PER_KWH / e_in
# Convert using common pricing tiers (USD per million tokens)
PRICE_OUT, PRICE_IN = 50, 10
v_out = out_per_kwh / 1e6 * PRICE_OUT
v_in  = in_per_kwh / 1e6 * PRICE_IN
def blended(r_in, r_out):
    return (v_in * r_in + v_out * r_out) / (r_in + r_out)
print(f"Output energy {e_out:.2f} J/token, ~{out_per_kwh/1e4:.0f} x 10k output tokens per kWh")
print(f"Input energy {e_in:.3f} J/token, ~{in_per_kwh/1e4:.0f} x 10k input tokens per kWh")
print(f"Output side: ${v_out:.0f}/kWh; Input side: ${v_in:.0f}/kWh")
print(f"1:1 blend: ${blended(1,1):.0f}/kWh; 3:1 blend: ${blended(3,1):.0f}/kWh")

The output matches our manual calculation: one kWh produces approximately 4 million output tokens, worth several hundred dollars.

4. Sensitivity Analysis: Which Parameters Matter Most

• Throughput is the most critical factor: Throughput is inversely proportional to energy consumption. If output throughput is increased to 3,000 tokens/second through larger batch sizes, KV cache reuse, speculative decoding, or low-precision quantization, per-token energy drops to 0.3 joules, tripling the tokens per kWh;
• PUE differences are significant: Air-cooled data centers have a PUE of around 1.5, while advanced liquid cooling can reach as low as 1.1—a difference of over 30% in cost from energy efficiency alone;
• Pricing is a market variable: If price competition drives output pricing down to $10 per million tokens, the output-side converted value drops to just $40 per kWh.

The wide $100–$800 per kWh range comes from the combined fluctuation of throughput and pricing.

5. How Cost Structure Reshapes the Market Landscape

5.1 The Tipping Point for Self-Hosted Deployment

Industrial electricity rates are around $0.08–$0.11 per kWh, while the “intelligence value” of one kWh is worth thousands of dollars. Electricity cost accounts for only a tiny fraction of the selling price—the remainder is filled by hardware depreciation, R&D amortization, idle rates, and corporate profit. When an enterprise’s token consumption is large enough and its workload stable enough, the tipping point arrives where “total API billing > total self-hosted cost (hardware depreciation + electricity + operations).” For large enterprises weighing both data security and cost, self-hosted deployment is almost inevitable.

5.2 Open-Source Models and the “Channel Fee” Business

Aggregation service providers build on open-source models and sell to SMBs and individuals at near-cost prices, earning the money of engineering optimization: improving throughput from 500 tokens/second to 3,000 tokens/second is equivalent to a sixfold increase in gross margin headroom. This “low-cost channel” layer will continue to compress the pricing ceiling of closed-source APIs.

5.3 The Challenge to Closed-Source Premium Pricing

If performance continues to converge, high-priced closed-source APIs offering undifferentiated capabilities will struggle to maintain pricing at multiples of the electricity-converted value. The way out is not “cheaper” but “irreplaceable”: complex reasoning, long context, multimodality, agent workflows, and domain data flywheels. For cloud service providers like Baidu AI Cloud, the real leverage is large-scale compute scheduling, inference acceleration capabilities, and per-token cost—not single-point benchmark scores.

5.4 Power Becomes an Upstream Strategic Resource

When token cost approaches electricity cost, cheap, stable, and green power becomes the core competitive advantage. Data center site selection, liquid cooling retrofits, and green energy trading will escalate from back-office concerns to strategic priorities.

6. Limitations and Corrections to This Calculation

For rigor, it should be noted that this simplified model does not account for: amortization of training costs (training a frontier model consumes tens of millions of kWh), GPU depreciation (often an order of magnitude higher than electricity costs), idle rates due to peak-valley load fluctuations (actual utilization may be only 30%–60%), and premiums for scarce capabilities. A more accurate conclusion is therefore: high-priced closed-source APIs offering undifferentiated capabilities are unsustainable, but models with differentiated capabilities can still command a long-term premium. The absolute claim that “closed-source is doomed” needs to be qualified.

7. Implications for Developers and Enterprises

• Application layer: Build tiered model routing—assign simple tasks to small models and reserve large models for complex tasks; leverage prompt caching and control output length;
• Evaluation metrics: Make decisions based on “total cost of ownership per million tokens” rather than “per-call price”; when volume is large and stable, carefully calculate the self-hosting threshold;
• Engineering efficiency: Every doubling of throughput halves the per-token electricity cost—inference optimization is one of the highest-ROI investments available.

8. Conclusion

One kilowatt-hour equals roughly several million tokens. This number reveals a simple truth: the physical cost of intelligence is not expensive—what is expensive is scarce capability and engineering inefficiency. In the long run, industry competition will shift from “model benchmarks” to a comprehensive contest of “energy, infrastructure, and engineering efficiency.” Tokens are like electricity and water: prices will eventually converge toward cost, and differentiated capabilities are the ultimate source of premium pricing.