Chinese LLM Technical Routes: A Four-Dimensional Breakdown of Architecture, Context, and Multimodality
Chinese large language models have shifted from a shared "parameter race" to distinct technical routes. This article breaks those routes down across four dimensions—architecture, context, modality, and alignment—and explains the design principles and tradeoffs behind each. It compares dense models with sparse MoE, walks through long-context engineering paths and their real cost, surveys multimodal fusion strategies, and contrasts alignment paradigms. It then covers why code capability matters, how to translate model advantages into verifiable engineering metrics, and how to build multi-model orchestration with observability—so you can map each route onto your own constraints rather than betting on a single vendor.
Introduction: From the “parameter race” to divergent routes
After the Transformer architecture was introduced in 2017, research teams inside and outside China entered the paradigm almost simultaneously. Early on, Chinese vendors largely followed a “reproduce and catch up” path. Only around 2023, as compute supply, the open-source ecosystem, and engineering experience accumulated, did different teams begin to make their own tradeoffs across four technical axes: architecture, context, modality, and alignment.
As a result, the question “what is vendor X’s core advantage” has limited value on its own—model versions iterate on a cycle of a few months, and advantages drift. The more engineerable question is: what are the design principles and applicability boundaries of these technical routes, and how do I map them onto my own business constraints? The four dimensions are unpacked below.
I. Architecture: rebalancing cost between dense scaling and sparse MoE
1.1 Dense models: a high capability ceiling, but a rigid inference cost
In a dense architecture, every token passes through all parameters. The upside is stable training, mature engineering, and a deterministic inference path. The cost is that per-token cost scales roughly linearly with parameter count, which makes cost optimization difficult at serving time.
1.2 MoE: trading “parameter capacity” for “activated compute”
The core idea of Mixture of Experts (MoE) is specialization: split the feed-forward layer into several expert subnetworks, and route each token to only the few most relevant experts. Total parameters can be made very large (large capacity, broad knowledge coverage), while the parameters actually activated per token stay small (controlled compute).
Using a common order of magnitude from industry designs: roughly two hundred billion total parameters, with on the order of tens of billions activated per token—capacity about three to four times the activated amount, a representative ratio range in current public MoE designs.
The gating network that drives routing is essentially a classifier:
import torchimport torch.nn.functional as Fdef moe_forward(x, experts, router, top_k=2, temperature=1.0):# x: [batch, seq, hidden]logits = router(x) / temperature # temperature controls the sharpness of the routing distributionprobs = F.softmax(logits, dim=-1)w, idx = probs.topk(top_k, dim=-1) # keep only the top-k expertsw = w / w.sum(-1, keepdim=True) # normalize to keep the output scale stableout = torch.zeros_like(x)for k in range(top_k):ek = experts[idx[:, :, k]] # gathered expert weightsout = out + w[:, :, k].unsqueeze(-1) * ek(x)return out
Several details determine whether a route is good or bad:
- Temperature coefficient: at high temperature the softmax distribution flattens, tending to activate more experts; routing is more “even,” load balancing is better but specialization drops. At low temperature the distribution sharpens, top-k almost always hits the same fixed experts—strong specialization but a risk of routing collapse (most tokens crowd toward a few experts while the rest are “starved”).
- Load-balancing auxiliary loss: to prevent collapse, training usually adds an auxiliary balancing loss that penalizes variance in expert load.
- Expert capacity factor: a per-expert token cap is set; tokens that overflow are passed to the next-best expert or routed through a residual connection, avoiding memory and communication spikes.
1.3 The real cost of MoE (easiest to overlook during selection)
- Memory and bandwidth: small activated parameters ≠ small memory footprint. At inference time all expert weights must be resident (or swapped in and out on demand), so the memory threshold is set by total parameters.
- Communication overhead: all-to-all scheduling under expert parallelism is sensitive to network bandwidth and easily becomes a bottleneck in cross-machine deployments.
- Training stability: the gradient sparsity from discretized routing requires z-loss, balancing losses, and warmup strategies.
- Fine-tuning difficulty: instruction-tuning on small datasets naturally produces uneven expert load, and some experts may simply not participate in updates.
Conclusion: MoE’s advantage lies in decoupling inference cost from capacity, which suits high-concurrency, cost-sensitive online services. If a business is memory-constrained or needs fine-grained domain fine-tuning on small data, a small-to-mid dense model is often the lower-maintenance choice.
II. Context: is long context a matter of “capability” or “parameters”
Long context has been the most contested direction of the past two years, but two concepts must be separated: window length and effective utilization.
2.1 Three typical engineering paths
| Path | Approach | Advantage | Cost |
|---|---|---|---|
| Positional-encoding extrapolation | RoPE base scaling, position interpolation, NTK-aware scaling | Small change, reuses existing corpora | Attention quality degrades in the extrapolation range |
| Attention sparsification | Sliding window + global tokens, block-sparse attention | Controllable inference cost | Degraded long-range precise retrieval |
| Data-side construction | Long-document corpora, multi-hop QA, synthetic long-range dependency data | Directly improves effective utilization | High data-engineering cost |
In practice these are usually combined: local-global hybrid attention in the architecture to control cost, positional-encoding adaptation in training, and long-range dependency injection in the data. The KV Cache memory estimate is worth computing by hand first:
KV Cache (bytes) ≈ 2 × layers × kv_heads × head_dim × seq_len × batch × bytes_per_elem
With layers=32, kv_heads=8, head_dim=128, seq_len=128K, batch=1, bf16:
2 × 32 × 8 × 128 × 131072 × 1 × 2 ≈ 17.2 GB (single sequence)
This is why grouped-query attention (GQA), KV Cache quantization (int8/int4), chunked prefill, and PagedAttention-style memory management are necessities in this direction—they are not nice-to-haves but prerequisites for turning “long context” from a demo into a service.
2.2 Window length ≠ effective utilization
What really determines the experience is: consistency in long-document summarization, cross-paragraph coreference resolution, multi-hop localization, and adherence to long-range constraints. Two layers of testing are recommended:
- Needle-in-a-haystack single-point retrieval (a lower bound on localization ability);
- Multi-hop, multi-constraint tasks that require global induction (the real business distribution).
If a model can only reliably use the first 10K of a 100K window, its effective capability is that of a “10K model,” not a “100K model.”
2.3 Dividing labor between long context and retrieval augmentation
The two are not substitutes but have different cost curves:
- Long context: pack all material into the window; cost rises linearly (or worse) with tokens, but there’s no need to chunk and semantic relationships stay intact.
- Retrieval augmentation (RAG): fetch only relevant fragments; low cost and easy to update, but subject to recall failures, chunking that breaks semantics, and conflicts between fragments.
A sensible architecture is layered: stable domain knowledge goes through knowledge-base retrieval, while ad-hoc tasks that need a global view go through long context—with both calibrated against the same evaluation set.
III. Modality: engineering tradeoffs across three fusion routes
Multimodal capability can’t be judged only by “can it read images”—fusion method and data quality matter.
- Concatenative (encoder + adapter + language model): the vision encoder’s output is projected to align with the language model’s hidden space, usually freezing one side or using two-stage training (align first, instruction-tune later). The most controllable and currently the mainstream route.
- Natively unified (discrete tokens / early fusion): visual information is quantized into tokens and processed isomorphically with text tokens; theoretically more natural cross-modal reasoning, but it places very high demands on paired-data scale and training stability.
- Tool-based (model + external vision toolchain): the model handles planning and reasoning, while specific recognition is delegated to specialized models. Fast to implement and accurate at individual points, but the chain is long and latency and failure rates compound.
What sets the ceiling on multimodal experience is data: the granularity and quality of image-text pairs, OCR and structured documents such as charts and tables, fine-grained localization annotations, and video temporal alignment. Evaluation should also be designed around the business distribution—document understanding, receipt/table extraction, fine-grained judgment in industrial quality inspection, video temporal QA—rather than just generic image recognition.
IV. Controllability: alignment paradigms and the specifics of the Chinese context
One representative approach abroad constrains model behavior with an explicit list of principles—giving the model a written set of guidelines, having it self-criticize and revise against those guidelines, then training on preference data. Many teams in China take a different path: deep localization at the data and knowledge layer. The difference between the two can be understood as follows:
| Dimension | Principle-list-driven | Data-and-knowledge-driven |
|---|---|---|
| Constraint entry point | Written rules inside the model | Training corpora and knowledge augmentation |
| Primary methods | Self-criticism/revision + preference learning | Corpus cleaning and debiasing, knowledge-graph augmentation, domain alignment |
| Strengths | Auditable rules, interpretable behavior | Fits local context, injects deterministic knowledge |
| Weaknesses | Hard to adjudicate when rules conflict | Implicit, hard to audit |
The two are actually complementary. A more practical layered approach for implementation:
Input-side filtering (block sensitive / unauthorized requests)→ Model-side alignment (behavioral tendencies from preference optimization)→ Output-side moderation (classifiers + rules + human fallback)→ Full-chain logging and traceability
A few easily overlooked practices:
- Over-refusal is also an alignment failure. Safety evaluation must be done alongside usefulness evaluation, or the model becomes “afraid to answer anything.”
- Reward-model bias propagates to the main model. The annotation consistency (inter-annotator agreement) of preference data matters more than its volume.
- The value of knowledge augmentation: the probabilistic generation of language models is inherently unreliable in scenarios that demand determinism (policies and regulations, internal corporate procedures, numerical definitions). Constraining with a knowledge graph or rule base is essentially using deterministic knowledge to set boundaries around probabilistic generation.
V. Capability spillover: why code capability deserves a separate look
Code capability is often treated as a proxy for general reasoning ability, for three reasons:
- Strong verifiability: compilation, unit tests, and static checks all give automatic correct/incorrect signals, so high-quality synthetic training data can be constructed at scale—an advantage that pure-text tasks can’t easily replicate.
- Long-range structural dependencies: a function call or a type definition may span thousands of lines, naturally training long-context and global-dependency modeling.
- Tool use and planning: real development tasks require reading and writing files, executing commands, inspecting results, and iterating—exactly the core form of agentic capability.
For enterprise adoption, repository-level context is usually supported by “retrieval + symbol indexing + call graph” working together: vector retrieval handles semantic similarity, and symbol indexing (classes, functions, reference relationships) handles precise navigation; the two are complementary. Evaluation should look at end-to-end task completion—given an issue description, can the model locate, modify, and make the tests pass—rather than only isolated function completion.
VI. Engineering selection: translating “advantages” into verifiable metrics
Don’t ask “which model is strongest”; ask “which route matches my constraints.” Work through the following checklist:
- Build a business evaluation set: no fewer than a few hundred real samples, covering the long tail and adversarial cases; keep both automatic metrics (accuracy, format compliance) and double-blind human preference scoring.
- Latency metrics: time to first token (TTFT), end-to-end P50/P95, output throughput (tokens/s). Note that average latency hides the long tail—P95 is what sets the user’s experience floor.
- Cost model: account separately for input tokens, output tokens, cache hits, and batch discounts; compute cost per thousand requests rather than cost per thousand tokens—output-length distributions differ enormously across models.
- Stability and capacity: rate-limiting strategy, peak elasticity, and a failure-degradation path (automatic primary/backup model switching).
- Compliance and deployment form: public cloud API, dedicated instances, private deployment—each with different data boundaries and audit requirements.
- Version governance: model upgrades cause behavioral drift, so a regression evaluation set must be kept and staged (canary) rollouts used.
VII. Deployment architecture: multi-model orchestration and observability
Mature enterprise solutions rarely bind to a single model; instead they build a model-serving gateway layer:
def route(request):if request.need_long_context and request.latency_slo < 2:return long_context_model # long-document / cross-file tasksif request.task_type == "code":return code_specialized_model # dev-focusedif request.task_type == "vision_doc":return multimodal_model # image-and-table documentsreturn default_model # general fallback
Recommended companion capabilities:
- Semantic and prefix caching: high-frequency Q&A and shared system prompts can significantly cut latency and cost;
- Tiered routing: simple requests go to small models, complex requests escalate to large models, balancing cost and quality;
- Observability: call logs, token consumption, refusal rates, and collected human feedback, all fed into unified logging and monitoring/alerting for cost attribution and quality regression;
- Data pipeline: unstructured material goes into object storage, the retrieval layer uses vector search, and the evaluation set and version assets are managed separately.
In domestic cloud environments, the above capabilities are typically carried by general-purpose product categories: model hosting and inference services, elastic container platforms, message queues for asynchronous batch processing, object storage and vector search, content-safety moderation, and logging and monitoring/alerting. For example, Baidu AI Cloud positions itself in these scenarios as providing integrated support from model invocation and knowledge-base retrieval through to content safety and runtime monitoring, letting teams focus their effort on business logic and the evaluation system rather than rebuilding infrastructure.
VIII. Conclusion: the endpoint of the route debate is your constraints
Condensing the four technical axes into one decision checklist:
| Your core constraint | Preferred route |
|---|---|
| High concurrency, cost-sensitive | Sparse MoE + prefix/semantic caching + tiered routing |
| Long documents, cross-file reasoning | Long-context optimization (GQA + KV quantization) + hybrid attention |
| Knowledge-intensive, needs traceability | Retrieval augmentation + knowledge-graph constraints + citation tracing |
| Images/tables/quality inspection | Multimodal (encoder-adapter route) + document parsing up front |
| Strong compliance, needs auditing | Private/dedicated deployment + layered content safety + full-chain logging |
Back to the opening question: the real “core advantage” of these teams lies not in any single feature but in the combination of architectural tradeoffs, data engineering, and engineering capability. For developers, the right approach is not to bet on one vendor but to build a reusable evaluation set and a swappable model-access layer—because technical routes will keep diverging, and your evaluation system and engineering architecture are the long-term assets.
