How to Choose a Chinese LLM: Architecture, Pricing, and Open vs. Closed Source Compared
This article provides a practical framework for selecting large language models (LLMs) from Chinese vendors. It focuses on three critical questions before evaluating model size or benchmark scores: contextual and reasoning needs, cost per call, and data residency constraints. It explains dense versus mixture-of-experts (MoE) architectures, how to calculate API pricing, open-source versus closed-source trade-offs, a selection checklist, and a step-by-step evaluation process for real business tasks.
Start with These Three Questions
When choosing a Chinese LLM, do not begin by comparing parameter counts or public benchmark rankings. Instead, answer three questions first:
- What context length and reasoning capability does your task require?
- Can you afford the cost per call at your expected volume?
- Can your data leave your own environment?
The rest of this article expands on these questions and provides an actionable selection process.
Dense Models vs. MoE: How Architecture Affects Cost
Two main architecture families dominate the current LLM landscape.
Dense Models
Dense models use all their parameters for every inference. Their structure is simpler, and training and deployment are less complex. However, the more parameters a dense model has, the more computation is required per token.
Mixture-of-Experts (MoE) Models
MoE models split the feed-forward layers into multiple “experts.” Each layer includes a gating network (router) that computes a score for each token across experts, selects the highest-scoring experts for computation, and leaves the rest idle. This allows the total parameter count to be large while the actual number of activated parameters per token remains relatively small.
Two important caveats when using MoE:
- Experts are not fixed by task type or domain. The routing is learned during training and is not externally interpretable.
- A large total parameter count does not automatically mean cheap inference. All parameters must still reside in GPU memory, multi-GPU deployment introduces communication overhead, and uneven routing can make certain devices bottlenecks. The main advantage of MoE is the capability obtained per unit of computation; memory footprint and deployment costs must be evaluated separately.
| Dimension | Dense Models | MoE Models |
|---|---|---|
| Computation per token | Proportional to all parameters | Proportional to activated parameters |
| GPU memory footprint | Roughly equal to parameter count | Based on total parameter count; usually higher |
| Deployment complexity | Lower | Higher; need to manage parallelization and routing load |
| Best suited for | Small-to-medium private deployments with simple workflows | Large-scale cloud services where capability and cost must be balanced |
How to Calculate API Pricing
Most providers charge per million tokens, with separate rates for input and output. The formula is:
Single-call cost = (input tokens ÷ 1,000,000) × input price + (output tokens ÷ 1,000,000) × output price
Example scenario: A customer service system handles 10,000 requests per day. Each request contains 2,000 input tokens and 500 output tokens. Suppose a model charges ¥2 per million input tokens and ¥8 per million output tokens. (These are illustrative only and do not represent any actual vendor’s pricing.)
- Daily input: 10,000 × 2,000 = 20,000,000 tokens → 20 × ¥2 = ¥40
- Daily output: 10,000 × 500 = 5,000,000 tokens → 5 × ¥8 = ¥40
- Total daily cost: ¥80
When estimating costs, consider four additional variables:
- Reasoning models may produce a hidden chain-of-thought, and those tokens are usually billed as output. The actual output volume can be much higher than the final answer.
- Repeated system prompts or knowledge-base content that hits a cache may be billed at a significantly lower rate than regular input.
- Batch APIs or off-peak discounts may reduce the unit price, but latency may increase.
- Price tiers can change once a certain context length is exceeded.
Always verify current pricing from the official pricing page, and note the date you checked. Prices change frequently; include the URL in your internal documentation.
Open Source vs. Closed Source
The choice between open and closed source is not binary. When evaluating, check the following points:
- Weight availability: Can you download the model files and deploy them locally?
- License type: Permissive licenses (e.g., Apache 2.0, MIT) usually allow commercial use. Custom licenses may restrict user scale, industry, or redistribution. Read the full terms carefully.
- Data compliance: For data involving personal information, trade secrets, or industry regulation, prefer private deployment or a compliant dedicated cloud environment.
- Operations capability: Self-hosting requires GPU resources, inference framework tuning, monitoring, and upgrades. The operational overhead is often underestimated.
- Version control: Cloud APIs may update or retire model versions. Lock to a specific version and keep a rollback plan.
A common pattern: sensitive data goes to a local open-source model, while general or peak traffic uses a cloud API. Both can be routed through a unified gateway for easy switching.
Selection Checklist
Before finalizing a choice, fill in the following items:
- Task type: Classification, information extraction, Q&A, code generation, long-document analysis, or multi-step reasoning
- Context length: Maximum input and output length for a single request
- Output format: Do you need strict JSON or a fixed template? Does the model support structured output?
- Latency and concurrency: Acceptable time to first token; peak requests per second
- Data compliance: Can data leave your domain? Does it contain sensitive fields?
- Cost ceiling: Maximum cost per call and total monthly budget
- Fallback plan: Which model do you switch to if the primary becomes unavailable or degrades in quality?
Implementation Steps
- Collect 50–200 real business samples covering normal requests, edge cases, and failure scenarios. Label them with ground truth answers or scoring rules.
- Run the same prompts with the same temperature parameter on two or three candidate models. Record the exact model versions.
- Measure four categories: task accuracy, output format compliance rate, time to first token and total latency, and cost per request.
- Assign weightings to these metrics based on business priority. For example, in compliance-heavy scenarios, output format compliance and safety should carry higher weight.
- Launch a small-scale canary deployment, then continuously monitor quality and cost. Re-run your evaluation set after every model version update.
Benchmark Scores Are Not a Substitute for Business Evaluation
Scores from public benchmarks are influenced by the test set distribution, prompt templates, evaluation scripts, and model versions. They may not reflect real-world business performance. To evaluate long-context capability, embed several key pieces of information in your own long documents and check whether the model retrieves them accurately. For reasoning ability, use real complex problems from your business rather than looking at math competition examples.
For high-risk domains such as legal, medical, or finance, model outputs should only be used as supplementary reference. Critical conclusions require human review.
