Why High-End Chinese LLMs Are So Expensive: The Math Behind the Price Tag

1 hr ago0 views

High-end Chinese large language models (LLMs) command premium API prices, often at 100 yuan per million output tokens. This article breaks down the economics behind these high price tags, examining whether the cost is justified. Through a detailed analysis of a typical 2.8T-parameter MoE model, we calculate the hardware requirements, actual deployment costs, and break-even pricing. The findings reverse the initial intuition: the high listed price is not a profit grab but a reflection of astronomical compute costs, which might not even be recovered at full utilization. This guide provides a practical framework for enterprises to evaluate model pricing based on total cost of ownership, token efficiency, and workload routing—essential reading for any team selecting or deploying large language models.

From a Skeptical Question: Is It Expensive or Not?

When a recent Chinese flagship model announced its API pricing, discussions around “why dare they price it so high” heated up again. With output token prices reaching the scale of 100 yuan per million tokens—an order of magnitude higher than earlier models—it certainly looks eye-catching at first glance. However, whether the pricing is reasonable cannot be judged by intuition alone. You need to calculate three accounts: capability, deployment, and unit economics. This article does not target any specific vendor or product. Instead, we use the typical specifications of current flagship models as a sample and fully break down the math. Once you work through the numbers, you will find that these seemingly aggressive high prices are likely just making implicit high costs explicit.

The Capability Account: Why Top Models Command a “Capability Premium”

Look at capabilities first. In authoritative benchmarks across code generation, AI agents, general knowledge, reasoning, and vision, the current tier of top models consistently ranks in the top three. Few newer frontier models can outperform them. This position translates into three commercial advantages:

Higher first-attempt pass rates. For multi-turn tool-calling agent tasks, weaker models often require multiple retries and human fallback. Strong models can complete tasks in one run. For the same task, a weaker model may consume several times more tokens and require more human wait time, meaning the “effective output” per token differs fundamentally.
Less context waste. Models with strong instruction-following capabilities do not need to be “coaxed” through repeated prompt adjustments. System prompts can be more concise, indirectly saving input costs.
R&D amortization. Training trillion-parameter models involves massive compute, data engineering, and multiple iterations. These sunk costs are inevitably distributed into inference pricing.

So the first pillar of pricing is the capability premium. But capability alone doesn’t explain everything—the truly “rigid” constraint is the second account: what it actually takes to run such a model.

The Deployment Account: Very Few Machines Can Even Fit It

Take a typical flagship model as a sample: a Mixture-of-Experts (MoE) architecture with a total parameter count of around 2.8T. Each token activates only 16 routed experts plus shared experts (from a pool of 896 experts), with native support for million-level (1024K) context.

First, let’s calculate memory requirements. Stored in common low-precision numerical formats, the 2.8T parameters alone take up about 2.8TB of memory. Adding KV cache, activation values, and inference framework overhead, the total memory requirement for a single node exceeds 3TB. Comparing to hardware realities:

A previous-generation flagship accelerator card with 80GB per card and an 8-card node gives about 640GB of total memory—not even enough to fit the weights. This is why the previous flagship 8-card nodes cannot handle it.
Only current-generation top-tier accelerator cards with 256GB to 288GB per card, when networked in 8-card configurations, can achieve single-node deployment. Today, such products can be counted on one hand.

Being able to fit the model is only the first step. The decode phase is a classic memory-bound bottleneck. How fast it runs depends on memory bandwidth and inter-card interconnect bandwidth. Whether the system can handle concurrency depends on how much KV cache fits in the remaining memory after the weights are loaded. The most common engineering optimization is speculative decoding: a small model drafts candidates, and the large model verifies them. Public benchmark data shows single-request decode speed can improve from about 111 tok/s to 331 tok/s—nearly a 3x improvement.

The Throughput Account: Fast Single Requests Don’t Equal Profit

Online services cannot pursue single-request speed alone. The machine must be kept busy with concurrency—this is the value of continuous batching. Requests dynamically enter and leave batches, so GPUs almost never idle. Referencing public single-node test data (an 8-card flagship node):

At 20 to 30 concurrent requests, single-user decode speed drops from 331 tok/s to about 16 to 20 tok/s.
However, total node throughput increases to about 672 to 754 tok/s.

16 to 20 tok/s is roughly equivalent to a human’s fast reading speed, still usable for interactive products. This is the classic trade-off of “sacrificing individual latency for overall throughput.” Based on this, we can estimate peak monthly output:

Node throughput T ≈ 672 ~ 754 tok/s
Monthly output    = T × 86,400 seconds × 30 days
                 ≈ 1.74 ~ 1.95 billion tokens/month

Even running 7×24 non-stop, a single node’s maximum monthly output is only 1.7 to 2 billion tokens. This number is the denominator for all subsequent cost calculations—keep it in mind.

The Cost Account: How High Is the Break-Even Line at Full Load?

Acquiring such computing power typically follows four paths (prices are approximate public market rates, converted to RMB at recent exchange rates):

Overseas public cloud rental for a flagship 8-card node: approximately 550,000 RMB/month.
Domestic compute rental market for the same spec: approximately 265,000 RMB/month.
Overseas channel full-machine purchase: approximately 3.9 million RMB one-time investment.
Domestic scarce channel purchase: according to public reports, transaction prices can reach 7 to 8 million RMB or higher.

Taking the most favorable scenario—domestic monthly rental at 265,000 RMB, monthly output of 2 billion tokens, counting only output tokens, only rental fees, with electricity and bandwidth free—the break-even price is:

Monthly output: 2 billion tokens = 2,000 M tokens
Break-even price = 265,000 RMB ÷ 2,000 M ≈ 132.5 RMB/M tokens

There is a common order-of-magnitude pitfall here: 1 billion = 100 M, so 2 billion = 2,000 M, not 20,000 M. If you add an extra zero to the denominator, you will mistakenly get a break-even line of 13.25 RMB/M—off by a factor of 10. Whenever you see any cost calculation, the first step should be to recalculate it yourself.

When the math is done correctly, the conclusion actually “reverses”: assuming output pricing at 100 RMB/M tokens, full-load monthly revenue = 2,000 M × 100 RMB = 200,000 RMB, which is lower than the 265,000 RMB monthly rental. In other words, even if the machine never rests, electricity is free, and input tokens are not charged, pure rental mode still loses money at this price.

The Real Cost: Even Higher Than 132.5 RMB/M

The 132.5 RMB/M figure is only the “theoretical lower bound” under idealized assumptions. Real-world operations face several additional costs that are often overlooked:

Prefill overhead for input tokens. In real business scenarios, input volume is often several times the output volume, especially in long-context scenarios. These consume the same compute and time on the machine.
KV cache squeezes concurrency limits. With million-level context, a single long session can consume tens of GB of memory, directly reducing the number of concurrent requests the system can serve.
Utilization can never reach 100%. Traffic has peaks and valleys. During late-night low-demand periods, the machine idles. Sustained effective utilization of 40% to 60% is already considered good. Sensitivity analysis on the break-even line:

Break-even price = Monthly rental ÷ (Monthly output × Utilization)
Utilization 100% → approximately 132.5 RMB/M
Utilization  60% → approximately 220.8 RMB/M
Utilization  40% → approximately 331.3 RMB/M

Electricity, cooling, and operations. A flagship 8-card node draws power at the tens-of-kilowatts level. Running 7×24 for a month, electricity plus cooling costs reach tens of thousands of RMB. Additionally, there are ongoing expenses for on-call operations, hardware failure replacement, and network bandwidth.
Procurement mode depreciation and value loss. If you invest 3.9 to 8 million RMB in hardware, depreciating over 3 to 5 years means a monthly depreciation of about 70,000 to 200,000 RMB. Moreover, when newer hardware launches, the residual value of older cards drops sharply, creating significant capital risk.

Adding all these items back, the true unit break-even cost rises significantly above 132.5 RMB/M. This also explains why top-tier models commonly adopt a pricing structure of “high list price, with discounts and caching handled separately”: the list price anchors to full-load costs and R&D amortization, while actual transaction prices are adjusted downward through volume discounts, context caching, and off-peak scheduling.

Conclusion and Selection Recommendations

Returning to the question posed in the title: why dare to price this high? Because the pricing is anchored not to the “marginal cost of electricity per token,” but to the sum of three factors: the compute cost that may not break even even at full load, the supply constraint of scarce hardware, and the R&D amortization from capability leadership. It looks expensive, but it is actually making hidden costs explicit.

For users and decision-makers, this analysis offers four actionable takeaways:

Evaluate based on “cost per completed task,” not “price per token.” A strong model passes on first attempt; a weaker model needs repeated retries. The final total bill could be completely reversed.
Leverage common cost-reduction tools: context caching, batch offline inference, concise system prompts, and controlled conversation history length can all directly compress your bill.
Use layered routing: assign high-complexity tasks to flagship models, routine tasks to smaller models, or distill a low-cost specialized model from the flagship.
Calculate total cost of ownership (TCO) before building your own infrastructure: include utilization, depreciation, electricity, and operations, then compare against managed services. Whether you choose to build your own inference cluster or use managed LLM services from cloud platforms (such as Baidu AI Cloud), it is recommended to use the same framework as this article to first calculate two key parameters—the “full-load break-even line” and “actual utilization”—before making a decision.

Finally, one trend observation: unit token prices will decline over the long term. Hardware generational improvements, quantization and decoding optimizations, and scheduling efficiency gains are all continuously pushing today’s “astronomical” prices down. Understanding the arithmetic behind the pricing allows you to understand both why it is expensive today and predict how it will drop tomorrow.