LLM Inference Energy Consumption and Token Economics: How Many Tokens Can One Kilowatt-Hour Generate?

5 hr ago0 views

This article presents a reusable framework for estimating the energy cost of large language model (LLM) inference. Starting from a 700 W single-GPU baseline and a data center PUE of 1.3 (about 900 W of effective power per card), it derives roughly 0.9 J per output token at 1,000 tokens/s decode throughput and 0.09 J per input token at 10,000 tokens/s prefill throughput. One kilowatt-hour can therefore support about 4 million output tokens or 40 million input tokens. Mapped against typical token pricing of $10 per million input tokens and $50 per million output tokens, a unit of electricity translates into roughly $200–$400 of revenue, or about $250–$300 for a blended 3:1 input-to-output workload. The article also outlines the framework's boundary conditions and its practical implications for capacity planning, cost optimization, and pricing decisions.

The Question: Is There a Clear Energy Ledger Behind Every Token?

As large language models see rapid adoption, discussions tend to focus on model capability, response latency, and service pricing, while the question “how much electricity does it take to generate one token?” is rarely broken down systematically. For developers, platform operators, and enterprise procurement decision-makers, understanding the relationship between tokens and electricity not only helps evaluate inference costs, but also provides a quantitative basis for capacity planning, model selection, and cost optimization.

This article therefore attempts to build a general, reusable estimation framework for inference energy consumption: starting from single-GPU power draw, factoring in data center infrastructure overhead, and deriving the electricity consumed per output token. It then combines the computational characteristics of input-side prefill to estimate how many tokens one kilowatt-hour can support. Finally, it links the energy figures to common token pricing models to observe the economic value that a unit of electricity can generate.

It should be emphasized that this article does not rely on any specific cloud vendor’s proprietary interfaces or hardware parameters. It is based on common inference deployment patterns and general estimation methods, with the goal of providing a transferable analytical approach.

Estimation Assumptions: Inference Scenario and Hardware Power Baseline

In inference scenarios, LLMs typically run on high-performance GPU clusters. For the sake of calculation, this article takes a typical inference deployment configuration as the baseline:

Single-GPU power draw: ~700 W
Data center PUE (Power Usage Effectiveness): ~1.3
After accounting for CPU, networking, storage, and cooling overhead, effective power per card: ~900 W

PUE is a standard metric for data center energy efficiency. A PUE of 1.3 means that for every joule consumed by actual computation, the data center as a whole consumes roughly 0.3 additional joules for cooling, power distribution losses, and other infrastructure support. This value falls within the common range for mid-to-large AI data centers today.

Adjusting single-GPU power from 700 W to 900 W means the subsequent calculations consider not just the GPU chip itself, but the combined energy consumption of the whole server plus its share of facility overhead. This keeps the estimate closer to real deployment conditions and avoids underestimating electricity costs.

Output Token Energy: Deriving Per-Token Energy from Throughput and Power

Generating output tokens is a classic autoregressive decoding process. Each time the model produces a token, it performs a full forward pass, appends the new token to the input sequence, and continues to the next round. Output throughput is therefore jointly determined by per-GPU compute capacity, memory bandwidth, batch size, and sequence length.

This article assumes a conservative but common aggregate output throughput:

Single-GPU output throughput: ~1,000 tokens/second
Effective power: 900 W, i.e., 900 J/s

The energy per output token is then:

900 J/s ÷ 1,000 tokens/s = 0.9 J/token

Converted to kilowatt-hours:

1 kWh = 3,600,000 J
3,600,000 ÷ 0.9 ≈ 4,000,000 tokens

In other words, under these assumptions, one kilowatt-hour can support the generation of roughly 4 million output tokens.

This result is not an absolute number; it is jointly determined by throughput and power. In real deployments, if output throughput reaches 2,000 tokens/second, per-token energy drops by half; if throughput is only 500 tokens/second, it doubles. The estimate is therefore best treated as a baseline reference rather than a fixed conclusion applicable to all hardware and model configurations.

Input Token Energy: How Prefill Differs

Unlike the output stage, input tokens are typically processed during the prefill stage. Prefill is characterized by high computational density and high parallelism—the model can encode the entire input sequence in parallel at once—so per-token energy is significantly lower than in the output stage.

Under similar hardware conditions, a common throughput assumption on the input side is:

Single-GPU prefill throughput: ~10,000 tokens/second
Effective power still calculated at 900 W

The energy per input token is then:

900 J/s ÷ 10,000 tokens/s = 0.09 J/token

Converted to the number of input tokens one kilowatt-hour can process:

3,600,000 ÷ 0.09 = 40,000,000 tokens

That is, one kilowatt-hour can process roughly 40 million input tokens.

The energy gap between input and output stems mainly from their different computation patterns. Output tokens must be generated one at a time, serially, making full parallelization difficult; input tokens can be processed in batches during prefill, so hardware utilization is higher and unit cost is lower.

From Energy to Economics: The Value of Electricity Under Token Pricing

Linking the energy figures above to common token prices makes it possible to estimate the revenue a unit of electricity can generate.

Assume the following pricing:

Input: $10 per million tokens
Output: $50 per million tokens

The token value corresponding to one kilowatt-hour on the output side is:

4,000,000 tokens × $50 / 1,000,000 tokens = $200

The token value corresponding to one kilowatt-hour on the input side is:

40,000,000 tokens × $10 / 1,000,000 tokens = $400

In real workloads, input and output are usually mixed. Using a 3:1 input-to-output token ratio, the blended revenue is roughly:

Weighted blend of input-side and output-side revenue
≈ $250–300 per kWh

This calculation reveals a noteworthy phenomenon: under token-based pricing, the input side has a lower unit price, but because of its higher computational efficiency, its economic output per unit of electricity can actually exceed that of the output side. This also explains why some model services adopt different pricing for input and output tokens, and why architectures are specifically optimized for long-input scenarios.

Boundaries and Applicability of the Framework

The estimates above depend on several key assumptions; applying them directly without these assumptions can lead to significant deviations. The main boundary conditions include:

Hardware model and power level: GPU cards differ significantly in power draw and throughput. Conclusions drawn from high-end cards cannot be directly transferred to mid-range cards.
Inference workload characteristics: Batch size, sequence length, model parameter count, and quantization strategy all affect real throughput.
PUE and facility conditions: Energy efficiency varies across regions and data centers; PUE can range from 1.2 to 1.6.
Pricing changes: Token prices are influenced by market competition, model positioning, service tiers, and other factors, and are not fixed constants.

A more reasonable approach is to treat the calculation chain in this article as a template, replacing the power, throughput, and pricing parameters with actual values from a specific project to obtain energy and economics data that fits your own scenario.

Implications for Engineering Practice

From an engineering perspective, token energy measurement is not just a financial issue—it directly affects system design. Several points deserve attention:

Inference optimization can significantly reduce unit cost: Techniques such as batching, prefix caching, quantization, and speculative decoding increase per-GPU throughput and directly reduce joules per token.
Evaluate input and output separately: Prefill and decoding have different resource consumption patterns and should be modeled separately in capacity estimates and cost models.
Balance energy efficiency and service quality: Higher parallelism may increase latency, so energy optimization must consider user experience metrics at the same time.
Electricity is a long-term operating variable: For large-scale inference services, the share of electricity cost may rise significantly as call volume grows, and should be factored into decisions between cloud resources and self-built infrastructure.

Conclusion

Token generation in large language models is not a purely digital output—behind it lies real electricity consumption and compute cost. By building a calculation chain of “power → throughput → tokens → kWh → revenue,” it becomes much easier to understand the economic structure of inference services, and the framework provides quantitative support for cost optimization, capacity assessment, and commercial pricing.

In future practice, integrating this kind of energy model with monitoring systems, billing platforms, and scheduling policies can enable more granular cost tracking and resource governance. For developers and enterprise users, mastering this estimation method helps them make more rational decisions across model selection, deployment options, and service pricing.