LLM Token Cost Economics: From API Pricing Down to the Electricity Floor

    As leading large models converge on capability, competition is shifting from model quality to price-performance, measured in dollars per million tokens. This article breaks down the full cost chain behind that price: why tokens work as a universal unit of account, why the physical floor of a token price is electricity, and how to estimate output and input energy per token. Using standard industry assumptions, it derives roughly 4 million output tokens or 40 million input tokens per kilowatt-hour, converts those figures into sellable value per kWh at typical price points, and explains what that headroom means as prices fall. It closes with the two alternatives cost-sensitive buyers reach for — private enterprise deployment and relay providers serving open-source models — plus a dual framework of technical and cost metrics for evaluating participants in the model era.

    1. Converging performance: competition shifts from capability to price-performance

    Over the past two years, the gap between leading large models on mainstream capabilities — general understanding, code generation, logical reasoning — has closed rapidly. Even when a new-generation model briefly leads on a benchmark, the advantage usually stays confined to a narrow slice of the market and rarely becomes a durable, generation-spanning moat. Once “can it do the job” stops being scarce, “how much does it cost to do the job” becomes the variable that decides market structure.

    The direct result is that the center of gravity in model competition has moved from a capability race to a cost race. The universal yardstick for cost is price per million tokens. The token has been promoted from a technical concept into the unit of account — the de facto currency — of the model era.

    2. Tokens: turning capability into something you can meter

    A token is the basic unit a large model uses to process text, and both input and output are measured in it. Reducing complex model capability to a single token figure effectively compresses “compute + electricity + hardware depreciation + operations” into one scalar that can be compared across vendors. Understanding token pricing is therefore, at bottom, understanding how many physical resources it takes to generate one token.

    This leads to a counterintuitive but defensible conclusion: strip away brand premium and commercial markup, and the physical floor of a token price is electricity. No matter how large the parameter count or how clever the algorithm, inference consumes real electrical energy — and hardware depreciation, data-center construction, networking and cooling all eventually fold into unit energy consumption. Under a radically simplified economic model, electricity is the dominant marginal cost of large-scale, global model serving.

    3. The cost foundation: why scale ultimately runs into electricity

    To see why, separate two categories of spending:

    • Fixed costs: GPU cluster procurement, data-center construction, R&D headcount. These get amortized as scale grows.
    • Marginal costs: the incremental electricity, bandwidth and depreciation added by each additional token served. These grow linearly with usage.

    In a competitive inference market, price oscillates around marginal cost. Whoever drives down energy consumption per token can push price to a level competitors cannot follow. Below is a reproducible quantitative estimate built on assumptions widely used across the industry.

    4. Quantifying the estimate: how many tokens a kilowatt-hour can produce

    The following calculation uses typical inference-scenario assumptions and figures drawn from commonly accepted empirical ranges. They are meant to illustrate orders of magnitude, not to report measured parameters for any specific product.

    Step 1 — effective power per card. A mainstream data-center AI accelerator draws roughly 700W per card at full load. Applying a PUE correction of ≈1.3 for facility cooling, power delivery losses, networking and CPU-side overhead gives an effective figure of about 900W.

    Step 2 — output side (decode) energy. During large-batch inference, aggregate output throughput per card typically falls between hundreds and thousands of tokens per second; taking a conservative 1000 tokens/s:

    1. Energy per token = 900 J/s ÷ 1000 tokens/s = 0.9 J/token
    2. 1 kWh = 3.6 × 10^6 J
    3. Tokens per kWh = 3,600,000 ÷ 0.9 ≈ 4 million output tokens

    In other words, one kilowatt-hour produces roughly 4 million output tokens.

    Step 3 — input side (prefill) energy. Prefill is a highly parallel matrix computation with throughput far above token-by-token decoding. At about 10,000 input tokens/s per card, energy per token drops to about 0.09 J, meaning a single kilowatt-hour can process roughly 40 million input tokens.

    The key insight here: input and output differ by nearly an order of magnitude in energy consumption. Input is a “read” (parallel, cheap); output is a “write” (sequential, expensive). That asymmetry is what makes the pricing logic on the two sides fundamentally different.

    5. From energy to price: cost structure on each side and the blended math

    Turning energy into “sellable value per kilowatt-hour” requires market price assumptions (industry-typical orders of magnitude here):

    • Output side: assume a price of about $50 per million tokens, so 4 million tokens × $50/M ≈ $200/kWh.
    • Input side: assume a price of about $10 per million tokens, so 40 million tokens × $10/M ≈ $400/kWh.
    • Blended: weighted by a real-world input-to-output ratio of 3:1, the combined price lands around $250–300/kWh (roughly 1,800–2,200 yuan).

    Given how widely throughput varies across models, batching strategies and context lengths, a reasonable cost band is roughly $100–800/kWh (about 700–6,000 yuan).

    What these numbers reveal is how much room may exist between today’s prices and physical cost. As competition becomes fully effective and prices keep falling, price will converge toward the “electricity + hardware depreciation” cost line. In that convergence process, hardware efficiency and algorithmic optimization become the core variables that decide whether a company survives.

    6. What the price signal implies: why private deployment becomes inevitable

    Once token prices climb high enough, cost-sensitive users start looking for alternatives on their own. That search usually follows one of two paths:

    1. Private deployment by large enterprises. When call volume is large and data-compliance requirements are strict, a one-time hardware investment amortized over years of electricity and depreciation often beats paying per token indefinitely. Enterprises gain not only a cost advantage but also data sovereignty and customization.
    2. Relay service providers running open-source models. Some intermediaries deploy open-source models on public cloud and charge small and medium customers and individual users a channel fee per call, earning the thin-margin spread on volume.

    It is worth noting that open-source models deliver more than “selling tokens.” Beyond a contest over closed API pricing, enterprises can capture incremental value through private deployment, domain fine-tuning, and distillation or compression — precisely the areas a purely closed API model struggles to cover. Some cloud providers, such as Baidu AI Cloud, already offer private and customized model services to enterprise customers, built on general-purpose capabilities including container platforms, object storage, logging and monitoring/alerting, so that models can be integrated smoothly into an organization’s existing business systems.

    7. Open source and the ecosystem route: value beyond selling tokens

    Alongside the flagship-model race, the open-source ecosystem is a parallel main line. Public repositories host a large number of models ranging from a few hundred MB to several GB in parameter count, plus specialized small models for tasks such as speech recognition, image classification and text embedding. In their narrow domains, these often come close to commercial alternatives. Their value is that they let users replace parts of commercial software with open-source means, reduce dependence on a single vendor, and retain control over deployment and iteration.

    Over a longer time horizon, the developer ecosystem, vertical solutions and regional collaboration networks that form around an open-source technical route have potential commercial and social value that far exceeds the sum of individual token transactions. Once such an ecosystem enters a positive feedback loop, every participant tends to favor the technical path that is cheaper, more controllable and more sustainable to evolve, at every stage from infrastructure to application deployment.

    8. Conclusion: build a dual coordinate system of cost and technology

    Evaluating any participant in the model era requires holding two rulers at once:

    • The technical ruler: capability, reliability, context length, multimodality and inference efficiency.
    • The cost ruler: energy per token, hardware utilization, feasibility of private deployment, and total cost of ownership (TCO).

    As capability convergence becomes the norm, the weight of the cost ruler will keep rising. Whether the choice is a public API, private deployment, or a self-built stack on open-source models, a rational decision starts from a few questions: how much token volume can I absorb, what are my electricity prices and hardware efficiency, and where are my data-compliance boundaries. The answer ultimately comes back to the physical cost line defined by electricity and hardware. Understand that floor, and you understand the real tempo of competition in the model market ahead.