LLM Inference Energy Consumption and Token Economics: How a Kilowatt-Hour Reshapes Compute Cost Perception

1. Starting with a Simple Question

When discussing the costs of serving Large Language Models (LLMs), a common question arises: How many tokens can a single kilowatt-hour (kWh) of electricity produce? The value of this question lies not in arriving at an exact figure, but in establishing a mindset that examines inference efficiency from an energy perspective. For developers, platform operators, and enterprise buyers, energy consumption is directly tied to hardware investment, data center construction, service pricing, and gross margins.

To answer this question, several key variables must be defined: hardware power draw, cluster energy efficiency, inference throughput, and the input-output token ratio. Only under a unified set of assumptions can the calculation yield comparable and meaningful results.

2. Power Calculation Framework for Inference Scenarios

2.1 Per-Card Power vs. Total Effective Power

Taking current mainstream high-performance accelerator cards as an example, the power consumption of a single card under full inference load is typically around 700W. However, this is not the complete cost. A real-world data center must also account for:

• Data center cooling system consumption
• Network equipment power
• Host CPU and memory overhead
• Power conversion losses

Introducing the Power Usage Effectiveness (PUE) metric provides a more reasonable estimate of overall energy consumption. A PUE of 1.3 means that for every 1 watt of computing power consumed, the data center actually needs to draw 1.3 watts of total power from the grid. Therefore, the effective power consumption of a 700W accelerator card is approximately 900W.

2.2 Inference Throughput Assumptions

In large-batch inference scenarios, the aggregated output speed per card is the core variable determining token yield per unit of energy. According to publicly available industry data, the output speed of frontier models in well-optimized inference frameworks typically ranges from 500 to 3,000 tokens/second per card. Here, 1,000 tokens/second is used as a conservative baseline, reflecting the average level of real production environments while keeping subsequent calculations robust.

The throughput during the prefill phase is significantly higher than during the token-by-token generation phase. Because prefilling allows for parallel computation of input sequences, per-card throughput can reach around 10,000 tokens/second. This difference heavily impacts the energy conversion calculations for the input and output sides.

3. Deriving Energy Consumption Per Token

3.1 Output-Side Energy Consumption

The formula for calculating the energy consumption per generated output token is:

Energy per Token = Effective Power ÷ Output Throughput

Substituting the baseline values:

900 Joules/second ÷ 1000 Tokens/second = 0.9 Joules/Token

One kilowatt-hour (kWh) equals 3.6 million Joules (3,600,000 J), therefore:

3,600,000 ÷ 0.9 ≈ 4,000,000 Tokens/kWh

This means that under the given assumptions, one kilowatt-hour can support the generation of approximately 4 million output tokens.

3.2 Input-Side Energy Consumption

Due to the high parallelism of the prefill phase, the energy consumption per token is significantly lower:

900 Joules/second ÷ 10,000 Tokens/second = 0.09 Joules/Token

The number of input tokens that one kilowatt-hour can process is:

3,600,000 ÷ 0.09 ≈ 40,000,000 Tokens/kWh

The energy efficiency on the input side is about 10 times that of the output side. This difference stems from distinct computation modes: prefilling involves dense parallel matrix multiplication, whereas token-by-token generation is an autoregressive process with sequential dependencies, leading to naturally lower hardware utilization.

3.3 Comprehensive Conversion

In real-world business scenarios, input and output tokens coexist. Assuming an input-output token ratio of 3:1 (a typical ratio for many conversational AI and Retrieval-Augmented Generation applications), the comprehensive energy consumption calculation is:

Comprehensive Token Yield per kWh = 1 ÷ (Input Energy Ratio + Output Energy Ratio)

Based on the calculations above, the economic value corresponding to one kilowatt-hour falls roughly in the range of $250 to $300 USD. Considering potential fluctuations in throughput, a reasonable interval can be broadened to $100 to $800 USD per kWh.

4. From Energy to Revenue: Reverse Engineering Token Pricing

4.1 Output-Side Revenue Conversion

If output tokens are priced at $50 USD per million, the revenue corresponding to the 4 million output tokens supported by one kWh is:

4,000,000 × $50/1,000,000 = $200/kWh

4.2 Input-Side Revenue Conversion

Input token pricing is typically lower than the output side; assuming $10 USD per million:

40,000,000 × $10/1,000,000 = $400/kWh

The revenue per kWh on the input side is actually higher than on the output side. This occurs because, although the unit price per token is lower, the processing volume is 10 times greater. This result indicates that from an energy efficiency standpoint, the commercial value density of the prefill phase is higher.

4.3 Mixed-Scenario Revenue Model

In a typical workload with a 3:1 input-output ratio, assuming 1 kWh is used to process inputs and outputs in a 3:1 split:

Comprehensive Revenue = 0.75 × $400 + 0.25 × $200 = $350/kWh

If energy allocation is based on actual token processing volumes rather than a simple proportional split, the actual energy proportion on the input side is much lower than its token proportion due to low per-token energy use. Consequently, the comprehensive revenue realistically falls within the $250 to $300 USD range.

5. Implications for Engineering Practices

5.1 The Energy Lever of Batching Strategies

The batching strategy (Batch Size) of an inference service directly impacts throughput. A larger batch size can improve hardware utilization and reduce per-token energy consumption, but it also increases latency. For offline tasks, maximizing throughput is preferable; for online services, a trade-off between latency and energy must be balanced. Understanding this relationship is crucial for developing tiered service strategies.

5.2 Commercial Value of Input-Output Ratio Optimization

Since energy efficiency on the input side is much higher than on the output side, designers should maximize the reuse rate of input information when structuring prompts, managing context, and designing caching strategies. For example, caching system prompts and sharing context prefixes can convert some of the output-side prefilling into parallel computation on the input side, thereby reducing overall energy consumption.

5.3 Hardware Selection and Power Limiting

Different hardware platforms exhibit significant differences in energy efficiency for inference tasks. Certain accelerator cards achieve a higher Token/Joule ratio in specific scenarios by lowering peak frequencies and optimizing memory bandwidth utilization. In procurement decisions, teams should look beyond single-card peak computing power and evaluate energy performance based on actual inference workloads.

5.4 Cost Optimization via Dynamic Scheduling

Real-time monitoring of inference cluster power consumption and throughput—coupled with dynamically adjusting batch sizes, concurrency, and model precision—can significantly reduce per-token energy consumption while maintaining quality of service. For instance, reducing concurrency during off-peak hours saves power, while enabling more aggressive caching during peak hours minimizes redundant computation.

6. Limitations and Boundaries of Applicability

The calculations above rely on several idealized assumptions. Energy consumption in real environments is influenced by:

• Model scale and architectural differences
• Degree of operator optimization in the inference framework
• Quantization precision and sparsification strategies
• Cluster scheduling efficiency
• Cooling methods and geographic location

Therefore, the token yield per kilowatt-hour is not a fixed constant, but a dynamic metric that evolves alongside the technology stack. The value of the framework provided here lies in establishing a complete analytical chain—from power and throughput to economic value. Any team can substitute its own actual parameters to derive conclusions more closely aligned with its business operations.

7. Conclusion: Integrating Energy Efficiency into Engineering Decisions

The question “How many tokens per kilowatt-hour?” essentially transforms ambiguous compute costs into quantifiable, optimizable engineering metrics. For teams focused on cost management and sustainability, this perspective offers insights across three levels:

1. Architectural Level: The energy efficiency disparity between the input and output sides requires system designs to place greater emphasis on context reuse and caching strategies.
2. Operational Level: Energy consumption monitoring should be integrated into metrics systems, forming a holistic service quality evaluation alongside latency and throughput.
3. Business Level: Token pricing strategies must establish a linkage mechanism with underlying energy costs, preventing profit margins from being eroded by energy consumption as scale expands.

As model scales continue to grow and inference demands surge, energy efficiency will no longer be an auxiliary data center metric. Instead, it will become a core determinant of the sustainability of LLM services. Understanding and mastering this analytical framework is an essential competency for technical decision-makers and platform operators in the next phase of industry competition.