How to Compare Domestic Chinese LLMs: Technical Approaches and Model Selection

    This article explains why leading Chinese large language models often share similar foundations yet differ in practice. It compares MoE architectures, training efficiency, broad capability, and deployment trade-offs, then outlines how to evaluate models against business tasks, performance metrics, and operating costs.

    1. The Main Differences Come Down to Engineering Trade-Offs

    Public technical information and industry trends suggest that many Chinese large language models share core design choices. Most are based on the Transformer architecture, and many are moving toward mixture-of-experts (MoE) designs. Rather than activating every parameter for every inference request, an MoE model routes each input to a small subset of experts. This allows it to maintain substantial overall capacity while reducing the computation required for an individual request.

    For this reason, model comparisons should look beyond total parameter counts and headline model sizes. More useful factors include:

    1. The ratio of total parameters to activated parameters;
    2. The number of experts and the design of the routing mechanism;
    3. Whether the attention architecture is optimized for long contexts and inference efficiency;
    4. Training-data quality and data utilization;
    5. Whether the optimizer can reduce training costs;
    6. The model’s strengths in areas such as coding, mathematics, reasoning, long-context tasks, and tool use;
    7. How open the model is, how complex it is to deploy, and how much effort it takes to integrate into an enterprise environment.

    In other words, models may share a similar architectural foundation while developing different strengths through their training strategies, data mixtures, engineering choices, and intended use cases.

    2. A Shared Foundation: From Dense Models to Mixture of Experts

    A traditional dense model uses all its parameters at every layer. As the model grows, the memory, compute, and communication bandwidth required for training and inference increase rapidly. An MoE model divides some of its feed-forward layers into multiple expert modules, activating only a small subset for each input.

    The process can be summarized as follows:

    1. Input token
    2. ↓
    3. Router selects the most suitable experts for the token
    4. ↓
    5. A small subset of experts processes the token
    6. ↓
    7. Expert outputs are combined
    8. ↓
    9. The result moves to the next layer

    This design has two main benefits.

    First, a model can have very large overall capacity. Different experts can specialize in coding, mathematics, natural language, domain knowledge, or linguistic patterns in different languages. Second, because only some parameters are used for each inference pass, the actual cost is driven more by the number of activated parameters than by the total parameter count.

    However, adding more experts does not automatically improve an MoE model. The experts must develop useful specializations, routing must distribute work effectively, and communication across devices must remain efficient. If too many tokens are routed to a small number of experts, those experts can become overloaded while others remain underused.

    The key design considerations therefore include load-balancing losses, routing strategies, expert capacity limits, and distributed communication efficiency—not just the number of parameters.

    3. Approach One: Emphasizing Scale and Inference Throughput

    One approach uses extremely large sparse architectures, with total parameter counts reaching or approaching the trillion scale while activating only billions of parameters per token. The goal is to retain broad knowledge capacity and task coverage while keeping the computation for each inference pass relatively low.

    These models often increase the number of experts to support finer-grained specialization. Some experts may focus on code generation, others on mathematical reasoning, and others on general language understanding. More experts can increase the model’s expressive capacity in theory, but they also make routing, memory management, and cross-device communication more challenging.

    For attention, this approach may use cache-efficient designs, such as compressing keys and values into a latent space to reduce KV cache usage during long-context inference. KV cache is an important factor in online serving capacity: as context length and concurrent request volume grow, cache pressure increases. Compressing the attention representation can therefore improve deployment efficiency.

    Typical strengths include:

    • Suitability for complex reasoning, code generation, and multi-step tasks;
    • Potential for high throughput under large-scale concurrent workloads;
    • Lower per-request inference cost through sparse activation;
    • Broad coverage across domains due to large model capacity.

    Deployment can be more demanding, however. These models may require a more complex distributed environment, with high-speed interconnects, substantial memory, and compatible inference frameworks. Small and midsize organizations should assess their hardware and operational capacity rather than relying on total parameter counts alone.

    4. Approach Two: Prioritizing Training Efficiency and Token Utilization

    A second approach focuses not only on model scale but also on how to achieve better results from less—and higher-quality—training data.

    The supply of high-quality human-labeled data is limited, making it increasingly difficult to improve models simply by expanding dataset size. Training efficiency has therefore become an important competitive factor. Token efficiency describes how much useful capability a model gains from a given number of training tokens.

    Engineering teams may improve training efficiency in several ways.

    1. Use optimizers suited to large-scale training

    Traditional optimizers can require substantial additional state and place pressure on memory and communication bandwidth when training very large models. Newer optimization methods may aim to reduce optimizer state, improve the direction of parameter updates, or increase the capability gained per training token.

    More aggressive optimization methods can introduce stability risks. In particular, if the numerical magnitudes of query and key vectors in attention layers continue to grow, attention distributions may become overly concentrated, potentially causing abnormal gradients or training divergence.

    2. Constrain attention computation for numerical stability

    One common approach is to clip or otherwise constrain the magnitudes of queries and keys, keeping attention scores within a more stable range. These measures do not directly add model parameters, but they can make training more stable.

    Stability improvements do not automatically increase model capability. Their main value is reducing the likelihood of failed training runs, increasing the number of effective training steps, and lowering the cost of reruns and troubleshooting.

    3. Add high-quality synthetic data

    Synthetic data can expand the supply of mathematics problems, code samples, reasoning traces, and domain-specific knowledge. More synthetic data is not necessarily better: inaccurate, repetitive, or overly uniform samples can teach the model unwanted biases.

    A more robust workflow typically includes:

    1. Clean source data
    2. ↓
    3. Generate candidate samples
    4. ↓
    5. Validate with rules and model-based review
    6. ↓
    7. Deduplicate, check for contamination, and assign quality tiers
    8. ↓
    9. Mix into pretraining or instruction-tuning data

    The central advantage of this approach is the ability to improve training outcomes when data is limited. It can be useful for teams seeking to control training costs and shorten iteration cycles. Its technical challenges lie not only in model architecture, but also in data engineering, training stability, and evaluation.

    5. Approach Three: Balancing General Capability and Practical Applications

    A third approach often uses MoE techniques similar to those described above, while placing greater emphasis on balancing general capability with practical use. Rather than focusing exclusively on maximum model scale, it aims to perform well across general question answering, coding, mathematics, tool use, long-context understanding, and complex task planning.

    These models may invest in areas such as:

    • Multi-stage pretraining to broaden general knowledge coverage;
    • Instruction tuning to improve interaction quality;
    • Reasoning data to strengthen performance on complex problems;
    • Tool-use training to improve task execution;
    • Safety data and refusal policies to reduce high-risk outputs;
    • Long-context training to improve document analysis.

    This approach can produce a versatile foundation for enterprise use. However, broad capability does not mean the model will lead on every individual task. Organizations should evaluate models against their own workloads. For customer service, priorities may include factual consistency and appropriate refusal behavior. For coding, they may include compilation and test pass rates. For knowledge-base question answering, citation accuracy after retrieval-augmented generation (RAG) may be more important.

    6. Why Total Parameter Count Is Not a Measure of Capability

    A common mistake in model comparisons is to use total parameter count as a direct proxy for model quality.

    For an MoE model, total parameters indicate overall capacity, while activated parameters are more closely related to the computation required for an individual inference pass. Two models with similar total parameter counts can have very different deployment costs if their activated parameter counts, layer counts, context lengths, or attention architectures differ.

    At a minimum, enterprise evaluations should distinguish between:

    • Total parameters: The model’s overall capacity;
    • Activated parameters: The scale of computation for an individual inference pass;
    • Memory usage: A factor in whether the model can be deployed;
    • KV cache size: A factor affecting long-context concurrency;
    • Time to first token: An indicator of interactive responsiveness;
    • Generation throughput: A measure of how efficiently the model handles batches of requests;
    • Cost per request: A factor in commercial viability;
    • Task accuracy: A measure of business value.

    Model evaluation should use data representative of real business workloads rather than relying only on public benchmarks.

    7. Mapping Technical Strengths to Enterprise Use Cases

    Organizations focused on code generation, complex analysis, or multi-step automation should prioritize reasoning, tool use, and long-context performance.

    Organizations more concerned with training costs and private deployment should examine activated parameter counts, quantization results, inference-framework compatibility, and training-data efficiency.

    Organizations looking for one model to support customer service, knowledge bases, workplace collaboration, and content generation may prefer a balanced model with strong ecosystem compatibility and flexible deployment options.

    In practice, organizations can also use general capabilities such as compute resources, object storage, container platforms, logging, and monitoring from providers such as Baidu AI Cloud to build a model development and runtime environment. Cloud infrastructure is only one part of a successful deployment, however. Results also depend on data quality, prompt design, retrieval workflows, evaluation criteria, and integration with business systems.

    8. Conclusion: The Real Differentiator Is Architecture Combined with Engineering

    Chinese large language models may share substantial similarities in their underlying architectures. MoE, sparse activation, attention compression, and large-scale distributed training have all become prominent design directions.

    Their differences are more likely to appear in areas such as:

    1. Emphasis on very large total capacity;
    2. Focus on training efficiency and data utilization;
    3. Balance between general capability and practical applications;
    4. Suitability for high-throughput online inference;
    5. Suitability for cost-efficient training and private deployment.

    For developers and enterprise users, the most reliable approach is not to chase parameter counts but to evaluate models across the full chain of task, metrics, cost, and deployment. A model’s practical value becomes clear only when its capabilities are assessed against real business workflows.