How Many LLM Tokens Can One Kilowatt-Hour Produce? The Cost Equation Behind the AI Industry

    A practical estimate suggests that one kilowatt-hour of electricity can generate roughly four million output tokens under common inference conditions. By separating prefill and decode workloads, this article examines token energy costs, API economics, falling inference costs, and why the AI industry may shift from selling tokens to delivering measurable business value.

    How Many LLM Tokens Can One Kilowatt-Hour Produce? The Cost Equation Behind the AI Industry

    A newly released large language model has recently delivered impressive benchmark results, reigniting debate over whether the competitive landscape is about to change. But benchmark rankings are not the only—or even the most fundamental—measure of competition.

    At its core, the commercial competition among large language models is a competition over the cost of producing intelligence. One straightforward way to examine that cost is through a physical question: how many tokens can one kilowatt-hour of electricity generate?

    This article breaks down the calculation step by step and considers what it means for the broader AI industry.

    1. Why Start with One Kilowatt-Hour?

    LLM inference essentially converts electricity into computation and computation into tokens. For an inference service provider, the cost structure mainly consists of three components:

    • Depreciation of computing hardware: The amortized cost of accelerators and servers;
    • Power and cooling: Ongoing operating expenses that generally increase with workload;
    • Networking, storage, and operations: Supporting infrastructure and maintenance costs.

    Electricity is the most transparent and easiest variable to quantify. Once we estimate how many joules are required to generate one token, we can derive a lower bound for inference costs and evaluate the sustainability of different business models.

    2. Step One: Estimating Token Output per Kilowatt-Hour

    Key Assumptions for Data Center Inference

    Consider a high-performance inference accelerator commonly used in data centers. We will use the following conservative assumptions:

    1. Effective power consumption of approximately 900 W per accelerator: The accelerator itself has a rated power draw of about 700 W. After accounting for cooling, networking, CPUs, and other infrastructure using a power usage effectiveness (PUE) factor of approximately 1.3, the effective power consumption is estimated at 900 W.
    2. Aggregated output throughput of approximately 1,000 tokens per second per accelerator: For frontier-scale models under high-concurrency inference, aggregated throughput per accelerator may roughly range from 500 to 3,000 tokens per second. We use a moderately conservative midpoint of 1,000 tokens per second.

    Energy Consumption per Token

    Power is energy consumed per unit of time:

    1. 900 W = 900 joules/second

    The energy required to generate one output token is therefore:

    1. 900 joules/second ÷ 1,000 tokens/second = 0.9 joules/token

    One kilowatt-hour equals 3.6 million joules:

    1. 1 kWh = 3,600,000 joules

    The number of output tokens generated by one kilowatt-hour is approximately:

    1. 3,600,000 joules ÷ 0.9 joules/token ≈ 4 million output tokens

    Conclusion 1: Under common inference conditions, one kilowatt-hour can generate approximately four million output tokens.

    3. Step Two: Separating Input and Output Token Economics

    LLM inference has two major stages with very different energy profiles:

    • Prefill: This stage processes the input prompt and involves highly parallel matrix operations. Its throughput is much higher than that of the output stage. A single accelerator may process approximately 10,000 input tokens per second, corresponding to about 0.09 joules per token—or roughly 40 million input tokens per kilowatt-hour.
    • Decode: This stage generates tokens autoregressively, one token at a time. It is constrained primarily by memory bandwidth, resulting in lower throughput and higher energy consumption per token. Under the assumptions above, the cost is approximately 0.9 joules per output token.

    Using representative API pricing levels as an illustrative calculation—not as specific quotations—the economics look like this:

    Item Tokens per kWh Assumed price Revenue per kWh
    Output tokens Approximately 4 million Approximately $50 per million Approximately $200
    Input tokens Approximately 40 million Approximately $10 per million Approximately $400

    Assuming a common business mix of three input tokens for every output token, the blended revenue is approximately $250–$300 per kilowatt-hour, or roughly RMB 1,800–2,200.

    Because throughput varies substantially with model size, concurrency, quantization, and scheduling efficiency, a reasonable broad estimate is approximately $100–$800 per kilowatt-hour, equivalent to roughly RMB 700–6,000.

    4. What Does This Calculation Reveal?

    1. The Energy Cost of Tokens Is Extremely Low

    Compared with residential and commercial electricity prices, which are roughly RMB 0.5–1 per kilowatt-hour in the original reference context, the hundreds of dollars in potential revenue generated by one kilowatt-hour show that electricity itself is not the primary bottleneck.

    The scarce resource is accelerator capacity and the ability to deploy it at scale. Current API pricing reflects hardware depreciation, supply and demand, and provider margins—not merely the marginal cost of electricity.

    2. Inference Costs Are Falling Rapidly

    Energy efficiency per generation of accelerator hardware is improving, while model compression, quantization, speculative decoding, and cache reuse continue to reduce the cost per token.

    Together, these improvements are driving an “inference Moore’s Law” in which token costs decline at a rapid pace. Services priced by the million tokens today could become an order of magnitude cheaper within two or three years.

    3. Premium Closed-Source Models Face Structural Pressure

    As open-weight models approach the capabilities of leading proprietary systems, enterprises can increasingly deploy them on their own infrastructure or through rented compute. This can bring inference costs closer to the combined cost of electricity and hardware depreciation.

    If closed-source APIs continue to charge significant premiums, their defensible advantages may narrow to two areas:

    1. A substantial and sustained capability lead;
    2. Compliance, reliability, and supporting engineering services.

    Once the capability gap becomes small, a business model based primarily on selling tokens becomes highly vulnerable.

    4. Value Will Shift from Selling Tokens to Selling Outcomes

    Future competition will focus less on the price per million tokens and more on who can turn model capabilities into measurable business value.

    Key areas include:

    • Embedding models into concrete workflows such as code generation, customer support, knowledge-base question answering, and document processing;
    • Providing efficient inference infrastructure, including elastic compute scheduling, containerized deployment, monitoring, alerting, and log analysis;
    • Helping customers maximize the number of useful tokens generated per kilowatt-hour.

    5. Implications for Developers and Enterprises

    1. Develop token cost awareness: When designing an architecture, evaluate the input-to-output ratio, context length, and caching strategy. Long prompts may be more economical than long outputs under some pricing and workload conditions.
    2. Evaluate private deployment economics: Once usage reaches a stable scale, operating or hosting an inference cluster may offer a significantly lower unit cost than pay-as-you-go API access.
    3. Prefer open ecosystems where appropriate: Avoid becoming dependent on a single expensive proprietary service and preserve the ability to switch between models.
    4. Treat engineering optimization as a competitive advantage: Batching, quantization, and KV-cache reuse can improve throughput by several multiples, effectively reducing costs without changing the electricity price.

    Conclusion

    One kilowatt-hour can generate approximately four million tokens under common inference assumptions. This figure highlights both the remarkable energy efficiency of LLM inference and the substantial compute premium embedded in current pricing models.

    As hardware improves and open-weight ecosystems mature, the industry may move closer to an era of “selling intelligence at the price of electricity.” The real competitive moat will not be the token itself, but the ability to convert inexpensive intelligence into measurable business value.