Why Flagship LLM APIs Are Priced High: Deployment and Cost Breakdown
This article explains why flagship large-model APIs command high prices by examining capability positioning, hardware requirements for trillion-parameter MoE deployment, inference throughput, and per-million-token cost. It also outlines how to evaluate total cost of ownership and match workloads to the right model tier.
1. Start With Capability Positioning
Before discussing price, it is necessary to establish the model’s capability tier. According to public evaluation information, this flagship model ranks among the leaders in coding, agents, knowledge reasoning, and visual understanding. Specific leaderboard positions, evaluation dates, and evaluation criteria should follow the official release and the original evaluation pages, and each item should be verified when cited.
Pricing usually benchmarks against models in the same tier, not merely models from the same country. If a model’s capabilities are close to the global top tier, its price reference point moves up accordingly. Understanding the price therefore starts with understanding which capability tier it intends to compete in.
2. Deployment Threshold: Hardware Requirements at Scale
According to public information, the model uses an ultra-large-scale mixture-of-experts (MoE) architecture with total parameters on the order of about 2.8 trillion (2.8T). Each token activates sixteen routed experts, plus shared experts. The model supports an ultra-long context of about 1 million tokens.
This design has two direct implications.
- Large VRAM requirements. Even loading only the weights at common precision requires multi-TB-level VRAM. Ultra-long context also requires additional KV cache, further increasing VRAM usage.
- Limited hardware options. Under FP16 or FP8 without quantization, with a target of million-token context, and without CPU offloading, eight previous-generation accelerators with 80GB VRAM each typically cannot complete a single-node deployment. The conclusion may differ with quantization, memory offloading, or multi-node approaches. Therefore, hardware selection for private deployment needs to be judged against the specific precision and context target.
3. Inference Performance: Single-Request Speed and Whole-Node Throughput
Performance data needs to distinguish between two types of metrics.
- Single-request speed: The output speed when one user has exclusive inference resources. The original article cites about 111 tokens per second, and about 331 tokens per second with speculative decoding enabled.
- Whole-node throughput: The total output speed when one node serves multiple requests simultaneously. With continuous batching handling 20 to 30 concurrent requests, per-user speed drops to 16 to 20 tokens per second, while total whole-node throughput is about 670 to 750 tokens per second.
These numbers come from a single source. The article does not fully list the accelerator card version, inference framework version, batch size, input/output length, and quantization conditions. Before applying them directly to your own environment, you should retest under equivalent conditions.
Based on the upper bound of whole-node throughput, running at full load for 24 hours gives a single-node monthly output of about 1.7 billion to 1.9 billion tokens. This number is the denominator for the cost calculation later.
Full load is an ideal assumption. Real workloads have clear peak and valley fluctuations, and capacity is often idle during low-demand periods at night. Every halving of utilization doubles the amortized cost per million tokens.
4. Cost Accounting: Break-Even Price per Million Tokens
The formula is:
Cost per million tokens = total monthly cost ÷ monthly effective output (in million tokens)
The prices in the table below are order-of-magnitude figures compiled from public market information. The original article converts at USD 1 = CNY 6.79. Price collection dates, exchange-rate dates, tax inclusion, and electricity inclusion are not stated, so the figures are for reference only.
| Acquisition method | Cost magnitude |
|---|---|
| Overseas cloud provider monthly whole-node instance rental | About CNY 550,000/month |
| Domestic IDC public monthly rental | About CNY 265,000/month |
| Overseas channel whole-node purchase | About CNY 3.9 million one-time investment |
| Domestic market whole-node spot | Supply-constrained; price clearly higher than official channels |
Using the domestic public monthly rental option as an example, assume full-load monthly output of 2 billion tokens, or 2,000 million-token units, and count only output-token revenue:
CNY 265,000 ÷ 2,000 = CNY 132.5 per million tokens
If monthly output is calculated at 1.7 billion to 1.9 billion tokens, the result is about CNY 137 to 156 per million tokens. In other words, just covering rental cost requires a price of more than about CNY 130 per million output tokens. If utilization is only 50%, the cost per million tokens under the same rent is about CNY 265.
This estimate does not yet include the following costs:
- Compute consumption for input tokens. Input also consumes compute. If only output is charged, the cost of the input portion must be covered by other means.
- Electricity and data center cooling.
- Capacity loss caused by insufficient utilization.
- R&D, operations and maintenance personnel, network bandwidth, and security compliance.
- A reasonable profit margin for the business.
If whole-node purchase is chosen, CNY 3.9 million amortized over 36 months is about CNY 108,000 per month, before electricity and operations and maintenance. In a full-production month, hardware amortization alone is about CNY 54 per million tokens. Hardware cost still accounts for a large share of the long-term path.
At a charge of CNY 100 per million tokens, rental cost alone is already higher than revenue. Even at full production, the operation would lose money, let alone cover electricity, cooling, operations and maintenance, and utilization losses. This shows that flagship-model inference is a heavy-asset business, and gross margins are usually not wide.
5. How to Evaluate Pricing Rationally
- When estimating total cost of ownership (TCO), include rental or amortization, electricity, operations and maintenance, labor, utilization, and the ratio of input to output tokens.
- The input-to-output ratio changes the cost structure. Long-document analysis has more input and less output; code generation and long-form writing have a higher output ratio. The two types of tasks have different unit costs and should be calculated separately.
- Peak concurrency determines the number of nodes required. Configuring for peak load causes idle capacity during low periods, while configuring for average load causes queuing during peaks. Elastic scaling or hybrid scheduling can be considered.
- For high-frequency but moderately difficult tasks, small and medium models or model routing are usually more cost-effective. For complex reasoning, long context, and agent orchestration, the per-token value of a flagship model may be more reasonable.
The above is an order-of-magnitude estimate and does not constitute procurement or pricing advice. Actual costs depend on regional electricity prices, utilization, inference framework optimization, and commercial discounts. It is advisable to rely on your own actual workload and quotes.
6. Conclusion
There are three main layers behind the high price of flagship models. First, the capability tier determines the pricing reference. Second, the trillion-parameter MoE architecture and ultra-long context raise the hardware threshold. Third, inference is a business that depends on large amounts of high-end hardware. Even at full load, hardware amortization per million tokens reaches the CNY 100 level, and price pressure increases as utilization declines.
Understanding these costs helps determine whether the price is commensurate with capability and usage. When selecting a model, beyond the unit price, you should also calculate your actual token mix and total cost of ownership.
