China's LLM Crunch: From Distillation and Borrowed Compute to Autonomous Breakthrough

    The capability gap between China's leading LLMs and their global counterparts may widen over the coming period. This article explains why: export controls are restricting access to high-end accelerator chips and overseas compute, while leading proprietary models are closing off the distillation shortcut. It then lays out a roadmap for building sustainable capability, covering data engineering, pretraining fundamentals, post-training and reinforcement learning, judge-based evaluation, and inference efficiency. Finally, it shows how graph-level abstraction, backend alignment baselines and open-source community co-development can reduce dependence on a single compute ecosystem.

    China’s LLM Crunch: From Distillation and Borrowed Compute to Autonomous Breakthrough

    Introduction: The Pressure Comes From Two Borrowed Shortcuts Closing at Once

    The capability gap between leading large models may widen, at least for a while, and possibly in stages. The reason isn’t mysterious: the two “borrowed” channels Chinese LLM teams have relied on are narrowing at the same time.

    • The compute channel: Export controls have clearly tightened access to high-end accelerator chips and overseas compute resources. Any further scale-up in training is now constrained by hardware supply.
    • The knowledge channel: Some leading closed-source models no longer offer downloads, and sampling output from their hosted services is restricted. On top of that, providers commonly prohibit using their outputs to train competing models. As a result, using distillation to borrow capability has far less room to work.

    The consequence is straightforward: every step a frontier team takes leaves the followers with one fewer shortcut. What really determines a team’s long-term position isn’t “how fast you catch up,” but “how strong your fundamentals are.” This article breaks that judgment into three parts:

    1. Risk breakdown: Why the dividend of the borrowed approach is running out, and how the gap forms;
    2. Catch-up checklist: What it takes to build self-sustaining capability;
    3. Breakthrough roadmap: How to reduce dependence on a single compute ecosystem and co-build with China’s domestic hardware and software stack.

    I. Why the Gap Forms: From the “Distillation Dividend” to a Capability Deficit

    Distillation Can Teach a Model to “Look Like” the Teacher, Not to “Know How”

    At its core, distillation is behavioral cloning. A student model is trained to fit the output distribution of a teacher, using data the teacher generates. It’s efficient, cheap and fast to show results. But it hits two hard ceilings:

    • The teacher sets the capability ceiling. A student can only approximate its teacher, not surpass it.
    • The teacher’s data sets the generalization boundary. Once a task falls outside the teacher’s output distribution, performance drops sharply. Examples include a new language, a niche domain, or a complex task that requires multi-step self-correction.

    The reasoning capability problem is even harder to spot. A student can learn the form of reasoning, such as breaking things into steps, writing “first / next,” and ending with an answer. That doesn’t mean it has learned the substance, such as verification, backtracking and error correction. Change the prompt slightly, or give it a task type the teacher never demonstrated, and performance can fall off a cliff.

    “Distillation + coding specialization” is especially fragile. Coding capability depends heavily on how well the training code distribution is covered. When a student meets libraries, framework versions or edge-case syntax the teacher never saw, it struggles to fill in the gaps on its own.

    It’s worth noting that even top labs with deep resources and clear research directions can misjudge a small sub-area and take a long time to correct course. The tolerance window for technical-path choices is far narrower than people imagine.

    When Compute Tightens, Engineering Inefficiency Gets Amplified

    When training scale is capped by hardware supply, efficiency becomes a hard requirement. Common sources of waste include:

    • Low arithmetic-intensity operators that create memory-bandwidth bottlenecks, leaving compute idle;
    • Pipeline bubbles that are hard to eliminate;
    • Communication and computation running one after the other instead of overlapping;
    • Fault recovery that takes hours, sometimes even days.

    When supply is ample, these losses can be masked by simply “buying more cards.” When supply tightens, they translate directly into slower progress.

    Inference works the same way. Output speed, time to first token, and cost per token determine how many real users a product can serve.

    “Benchmark Chasing” Borrows Against Long-Term Capability

    When R&D goals are defined by a single leaderboard or a single task, teams tend to fall into several habits:

    • Data mixes drift toward the evaluation sets;
    • Contamination of evaluation sets goes unnoticed;
    • Reward design gets gamed (reward hacking);
    • Training resources concentrate on a few easily measured tasks.

    These practices pay off in the short term, but they’re risky over the long run. They shape a model’s capabilities into something “narrow and pointy.” Long-tail robustness, tool use, instruction following and the ability to recognize what it should refuse are all drawn down in the process. Repurposing (“hacking”) a general-purpose LLM into a coding-specialized model may look like differentiation, but in practice it narrows the model’s capabilities even further.

    How the Gap Plays Out When All Four Constraints Hit at Once

    In reality, constraints rarely arrive all at once. They tend to stack up:

    1. First come restrictions on the most advanced chips;
    2. Then tighter limits on compute leasing;
    3. Then restrictions on frontier model outputs.

    Add in teams dependent on “distillation + specialization” shortcuts, and the gap opens in a compounding way.

    Pressure periods are therefore a true test, and the pressure propagates layer by layer through the capability stack:

    1. Compute: Training scale is forced to shrink;
    2. Data: The supply of data generated by teacher models is cut off;
    3. Post-training: If core pieces such as reward design and the judge model (judge) are under-invested, their weaknesses show up directly;
    4. Inference: Cost and latency disadvantages show up in user retention and word of mouth.

    II. The “Self-Sustaining” Catch-Up Checklist for Pressure Periods

    Data Engineering: It Sets the Capability Floor

    Data quality sets the capability floor. Algorithms only determine how quickly you approach that floor.

    • Fine-grained cleaning: Work through the pipeline in order: exact deduplication, near-duplicate removal, quality-based filtering, and contamination checks against evaluation sets.
    • Mix and curriculum: The ratio of code, math, multilingual and domain corpora directly shapes a model’s capability profile. It’s common practice to build strong representations early in training with general, high-quality, thoroughly deduplicated data. The share of high-value data such as reasoning and code is then raised in later stages.
    • Verifiable data: Tasks where answers can be checked by a program, such as math, code and formal logic, are the scarcest resource in post-training.
    • Synthetic data: It suits long-tail scenarios like low-resource languages and specialized domains. Keep an eye on diversity collapse caused by generating data from the same source. It works best when combined with rule-based generation or cross-model validation.
    • Governance: Record data sources, licenses and versions, so every change in capability can be traced back to a specific data change.

    Pretraining Fundamentals: Architecture, Stability and Efficiency

    • Architecture choices:
      • Sparse mixture-of-experts (MoE) structures offer greater parameter capacity at a relatively low inference cost. They also demand more of routing balance and expert-parallel communication.
      • Attention variants need to balance long-context efficiency against the KV cache overhead they add on the inference side.
      • Positional encodings need to be assessed for their ability to extrapolate to longer sequences.
    • Training stability: Activation recomputation, gradient checkpointing, optimizer-state sharding, and loss scaling under mixed precision are all standard techniques. They reduce per-GPU memory pressure and prevent numerical overflow.
    • Parallelism strategies: Data, tensor, pipeline and sequence parallelism need joint tuning. The main goals are eliminating pipeline bubbles and reducing cross-node communication.
    • Stability operations: Loss spikes are a common problem in large-scale training. Establish gradient-norm monitoring, parameter rollback and step-skipping mechanisms. These keep training under control when hardware supply is limited and runs can’t be restarted at will.

    Post-Training and Reinforcement Learning: From “Imitating Answers” to “Learning by Trial and Error”

    Through interaction with an environment and reward signals, reinforcement learning lets a model discover strategies its teacher never demonstrated on its own. This is the main path to surpassing the teacher.

    • Verifiable rewards: On tasks where answers can be checked, such as math and code, use rule-based checks or sandbox execution results as the reward (RLVR). This cuts off room for reward hacking at the source.
    • Process rewards: Give feedback on intermediate steps in a reasoning chain to ease the problem of sparse rewards.
    • Multi-turn environment feedback: Let models iterate on real feedback from tool calls and code execution, instead of only imitating the teacher’s reasoning trajectories.
    1. # Illustrative code: hybrid reward combining rule verification, process reward, and judge scores
    2. # The rule-based score carries the highest weight; the judge score covers only subjective dimensions that cannot be verified automatically
    3. W_RULE, W_STEP, W_JUDGE = 0.6, 0.2, 0.2
    4. def reward(rollout, task, env):
    5. # 1. Rule verification: run unit tests on code in a sandbox; compare math answers against the ground truth
    6. rule = 1.0 if env.verify(task, rollout.final_answer) else 0.0
    7. # 2. Process reward: hit rate of intermediate steps that can be checked (e.g., intermediate results, theorem citations)
    8. steps = task.verifiable_steps or []
    9. step = sum(1 for s in steps if s.check(rollout.trace)) / max(len(steps), 1)
    10. # 3. Judge scores: subjective dimensions such as writing quality and readability; rate-limited and spot-checked separately
    11. judge = judge_model.score(task.rubric, rollout) if task.need_judge else 0.0
    12. return W_RULE * rule + W_STEP * step + W_JUDGE * judge

    With a hybrid reward, watch out for the judge score becoming a channel that can be gamed. Fixing that is exactly what the next section is about.

    Evaluation and the Judge: The Most Undervalued Piece of Core Infrastructure

    Automated evaluation is key to iteration speed, and judge quality directly determines whether optimization is heading in the right direction. Known biases include at least:

    • Position bias: A preference for options listed first or last;
    • Length bias: A preference for longer responses;
    • Self-enhancement bias: A preference for outputs stylistically similar to its own;
    • Same-origin favoritism: Models trained from the same source as the judge tend to score higher.

    If a judge has systematic biases, reinforcement learning will push the model toward what “the judge likes but users don’t.” Building a reliable judge takes at least the following steps:

    1. Randomize positions: Swap the A/B order in pairwise comparisons;
    2. Anchor to references: Use human-confirmed reference answers as anchors;
    3. Aggregate multiple samples: Average several scoring passes to reduce the variance of a single score;
    4. Align with humans: Calibrate the judge on a set of human-labeled data and keep tracking the agreement rate;
    5. Prefer rules: Keep tasks that can be verified automatically away from the judge.

    Inference Infrastructure: The “Efficiency Moat” for Pressure Periods

    • Memory management: Paged KV cache management sharply reduces memory fragmentation, so GPU memory can hold more concurrent sequences.
    • Scheduling: Continuous batching lets new requests join at any time, so they don’t have to wait for the entire batch to finish.
    • Cache reuse: A prefix cache hit skips the repeated prefill of a shared system prompt.
    • Speculative decoding: A small model drafts tokens and the large model verifies them, which raises decoding throughput.
    • Disaggregated deployment: Run prefill and decoding in separate resource pools and tune each for its own workload, whether compute-intensive or memory-bandwidth-intensive.
    • Low-bit quantization: Inference often uses 8-bit or even lower-bit quantization of weights and activations. Quantized and full-precision models need to be evaluated together. Watch for accuracy loss being amplified on long-chain reasoning tasks.

    III. Breaking Path Dependence: From One-Off Porting to Ecosystem Co-Building

    The Real Risk of the Comfort Zone

    Teams that have long developed within mature GPU programming models and operator library ecosystems tend to build up habits:

    • Highly customized kernels;
    • Reliance on specific vendor profiling tools;
    • Collective communication libraries and frameworks tightly coupled to one another.

    In their usual environment, this stack is extremely efficient. On a different chip architecture, though, many carefully optimized kernels can’t be reused, and porting amounts to a rewrite.

    An even more realistic risk is “dual-track maintenance.” The team maintains one codebase for mainstream international chips and another for domestic chips. Team effort is split across both, and optimization on each side stays shallow.

    Graph-Level Abstraction: Confine Backend Differences to One Place

    The key to decoupling is separating model logic from hardware implementation:

    • The upper layer describes computation in general tensor operators.
    • The lower layer has different backends decide how to schedule those operators onto specific chips.
    1. # Illustrative code: compute graph with pluggable backends; model code never depends directly on any chip vendor's APIs
    2. from dataclasses import dataclass
    3. @dataclass
    4. class OpSpec:
    5. name: str # Operator name, e.g. rmsnorm / flash_attn / moe
    6. dtype: str # Data type, e.g. fp16 / bf16
    7. meta: dict # Shape, causal masking, tensor-parallel sharding scheme, etc.
    8. class Backend:
    9. def compile(self, graph: list[OpSpec]) -> callable: ... # Graph compilation and operator fusion
    10. def collectives(self) -> dict: ... # all-reduce / all-gather / reduce-scatter
    11. def memory_hints(self) -> dict: ... # Memory hierarchy, cache capacity and bandwidth
    12. class GpuBackend(Backend): ... # Mainstream GPUs: mature kernel libraries + graph compiler
    13. class DomesticAiChipBackend(Backend): ... # Domestic AI chips: vendor compiler + open-source operator ecosystem
    14. BACKENDS = {"gpu": GpuBackend, "domestic": DomesticAiChipBackend}
    15. # At deployment time, only the backend needs to be switched; the model architecture, training scripts, and business code stay unchanged
    16. runtime = BACKENDS["domestic"]().compile(model.to_op_graph())

    With this design, migration shifts from “rewriting model code” to “filling in backend capabilities.” From an engineering standpoint, only one compute graph definition and one set of training scripts need to be maintained.

    Build Backend “Alignment Baselines” and Let Data Drive Migration

    • Numerical consistency: Feed the same fixed inputs to every backend and compare outputs layer by layer. This pinpoints the layers where precision deviations appear.
    • Convergence consistency: Compare loss curves across backends on small-scale models to make sure training dynamics are the same.
    • Performance baselines: Build benchmarks that cover each operator and parallelism configuration, and run regression tests on a regular basis.
    • Continuous verification: Add multi-backend compilation and smoke tests to the CI pipeline, so no commit breaks the usability of any backend.

    From One-Off Porting to Community Co-Building

    No single company can cover every operator and every use case. A practical approach is to contribute the operator implementations, precision fixes and performance tuning experience gained while adapting to domestic chips back into open-source toolchains. Chip vendors, framework communities and other users can then build on them together.

    This way, many participants gradually fill the gaps in the ecosystem, and teams that come later don’t have to repeat the same mistakes.

    Design for Portability at the Architecture Level

    • Operator selection: Prefer “common operators” that have mature implementations on every backend. Keep a list of scarce or unsupported ones and evaluate alternatives.
    • Communication topology: The model-parallel sharding scheme needs to take into account the interconnect topology and bandwidth characteristics of domestic clusters.
    • Inference works the same way: Quantization schemes, attention kernels and communication libraries need to be adapted for each backend, each with its own accuracy baseline.

    IV. The Whole Supply Chain: A Compute Breakthrough Takes More Than One Chip

    China’s domestic compute ecosystem will only mature through the joint evolution of many layers. These include:

    • Chip design;
    • Advanced-node manufacturing;
    • Packaging;
    • High-bandwidth memory;
    • Equipment and materials;
    • Electronic design automation (EDA) tools;
    • Compilers and operator libraries;
    • Frameworks;
    • Models;
    • Applications.

    A weak link anywhere becomes a bottleneck for the whole.

    Iteration speed matters just as much. The shorter the feedback loop between chips, compilers, operator libraries, models and applications, the faster the ecosystem matures. When the feedback loop is long, even a single breakthrough is hard to turn into usable compute.

    So, more than asking “has a given chip caught up,” it’s worth paying attention to:

    • The pace of chip releases and the maturity of the software stack;
    • Operator coverage and end-to-end performance on common models;
    • Developer migration costs and how friendly the toolchain is.

    As a trend, with the supply of high-end single chips constrained, an increasingly realistic engineering path is emerging. It means organizing large amounts of relatively ordinary compute through multi-card interconnect, high-speed networks and efficient parallelism strategies to run training and inference. The ability to do this is itself a core competitive strength.

    V. Seeing the Gap Clearly: Pressure Periods Are Also Capability Windows

    During pressure periods, it helps to distinguish three types of gaps:

    Gap type Manifestations Response priority
    Measurable gaps General reasoning, code generation, multilingual capability High: catch up through data and reinforcement learning
    Ecosystem gaps Toolchain maturity, operator library coverage, developer habits High: sustained investment, with compounding returns
    Time-based gaps New directions and paradigms that leaders have already invested in Medium: keep tracking and run small-scale validations; avoid blindly following trends

    History shows that competitive landscapes are rarely linear. Even top labs can go quiet for a long time after a wrong turn. Resources don’t decide everything.

    What really opens up the gap usually comes down to three things:

    • Data quality;
    • Iteration speed;
    • Engineering efficiency.

    None of these depends heavily on the most advanced hardware. That’s exactly why they’re what teams should focus on during pressure periods.

    Conclusion

    Pressure periods are unavoidable, but “under pressure” is not the same as “standing still.”

    In the short term, real capability gaps between leading models do exist. They may widen as compute supply and the flow of knowledge become more constrained.

    In the medium term, the gap can narrow step by step once teams do two things:

    • Shore up the fundamentals: data, pretraining, reinforcement learning, evaluation and inference infrastructure;
    • Reduce dependence on a single compute ecosystem through graph-level abstraction and community co-development, as China’s domestic hardware and software stack matures along the chip → toolchain → model → application flywheel.

    For R&D teams, the most pragmatic strategy right now comes down to three things:

    1. Reduce reliance on distillation and benchmark chasing, and invest resources in verifiable data and reinforcement learning;
    2. Plan for multi-backend portability from the architecture design stage;
    3. Get involved in open-source communities and the domestic chip ecosystem, turning one-off porting into collective co-building.

    Teams that weather this round of tests will no longer be held back by external supply. They’ll have built a technical moat that is genuinely their own.