Gray-Box Competition: What the "Distillation Narrative" Claims About LLM Capabilities and How to Counter It

    The "distillation narrative"—the claim that a follower model's capabilities come entirely from distilling a leader's, so restricting the upstream will stall it—spreads easily but is costly to refute. This article breaks the claim into its logical premises, separates three often-conflated ideas (distillation, capability inheritance, and autonomous training), and examines three overlooked asymmetries in cost, time, and objectives. It then lays out practical, reproducible checks—from config fingerprints and vocabulary comparison to behavioral and representation tests—for judging a model's provenance, and closes with a path for enterprises to move from merely calling external models to owning and sustaining their own capabilities.

    In debates about where a model’s capabilities come from, the “distillation narrative” is a claim that spreads easily but is expensive to refute. It conflates “using distillation techniques” with “relying entirely on distillation,” and from there concludes that “if you restrict the upstream, followers will stall.” This article separates three ideas—distillation, capability inheritance, and autonomous training—and tests each one’s hidden assumptions. It then offers practical checks for judging a model’s provenance, along with a path for enterprises to move from “being able to call” a model to “owning and sustaining” their own capabilities.

    1. Breaking Down the Problem: What Does the “Distillation Narrative” Actually Claim?

    The so-called “distillation narrative” is a popular claim that a follower’s LLM capabilities come entirely from distilling a leader’s model, so restricting the upstream will stall the follower. To respond, the first step is to break down the logic chain of the assertion rather than getting stuck on whether distillation ever happened.

    1. Capability-source claim: all or core capabilities come from distilling another model’s outputs.
    2. Dependency claim: this capability supply is one-way and sustainably depended upon; once the upstream is restricted, the follower cannot obtain it.
    3. Inference claim: therefore restricting upstream development is enough to stall the follower’s capability growth.

    Two key premises need testing: first, the specific form “distillation” refers to; second, whether the factual judgment “capabilities depend on the upstream” actually holds.

    It also relies on gray-box argumentation. Black-box reasoning judges by input/output; white-box reasoning judges by the full parameters and training process; gray-box reasoning cannot open the model’s interior yet claims to have seen everything. The conclusion “cut off supply → stall” is built on this gray-box narrative that cannot be fully verified, which itself weakens its force.

    More notably, arguments for the “supply cutoff” theory are often over-extrapolated. For example, training the strongest models requires massive compute—that is a fact—but there is no necessary link between “limited compute scale” and “capabilities must stall.” A more reasonable statement is that constraints change the shape and center of gravity of the growth curve—for instance, placing more emphasis on inference efficiency, domain adaptation, and cost control—rather than simply drawing a “stall line.”

    2. Clarifying Concepts: Distillation, Inheritance, and Autonomous Training Are Not the Same

    2.1 The Strict Definition of Knowledge Distillation

    “Distillation” was originally a chemistry term, later borrowed into machine learning, referring to using a stronger, more complex “teacher model” to guide the training of a smaller “student model.” The classic approach:

    • Objective: make the student’s probability outputs approximate the teacher’s output distribution. The loss is typically a weighted sum of cross-entropy loss and distillation loss (usually KL divergence), with a temperature coefficient T scaling the logits.
    • Dark knowledge: the temperature-softened probability distribution carries the teacher’s judgment about the similarity relationships between options—information that hard labels on the “correct answer” cannot express.
    • Common uses: model compression and acceleration, synthetic data construction, and multi-task unification.

    Implementations differ between black-box mode (text outputs only) and white-box mode (vocabulary-level probabilities), and the difference is large: black-box distillation mostly yields “selection preferences,” while white-box distillation also includes the “probability distribution.”

    Distillation can also be understood as transferring “procedural knowledge”: the teacher gives the student not a set of rules but a “default behavior” formed over large inputs. This is the key thread for the later discussion of inheritance.

    2.2 Five Distinct Forms of “Distillation” That Must Be Distinguished

    Online, “distillation” is often used as an all-purpose label. In fact, it maps to at least five different practices:

    1. Academic model compression: the student inherits only partially due to capacity or compute limits—a typical “trade quality for quantity.”
    2. Black-box API outputs as synthetic corpus: batch-generating data with an external model for training; quality depends on prompt design, sampling scale, and filtering strategy.
    3. Directly reusing open weights for continued training: continued pre-training or post-training on released parameters. This is the most direct form of inheritance, but it is conditional on the license permitting it.
    4. Using similar-model outputs as preference data: for example, using a stronger model to rank responses and generate substitute preference data.
    5. Reproducing the training pipeline from technical reports: retraining based on publicly disclosed data composition, hyperparameters, and training strategy. This is method reproduction, not parameter inheritance.

    Lumping these five together as “distillation” is like treating all “using public literature” as “plagiarizing papers”—it confuses the nature and the degree of the behavior.

    2.3 The Ceiling of Distillation: What Is Inherited Is “Behavior,” Not an Irreproducible Core

    The student inherits the teacher’s behavioral tendencies on a set of tasks. Once it leaves the teacher’s coverage or hits out-of-distribution problems that the teacher itself gets wrong, its capabilities cannot increase out of thin air. So distillation does not create a one-way “cut off supply → useless” dependency: as long as the student gains enough independent training and autonomous iteration, it can follow an evolutionary path different from the teacher’s.

    Conversely, when multiple models share highly similar training corpora, their outputs and representations will be highly alike even if they never distilled from each other directly. This means similarity is “weak evidence,” and asserting distillation from similarity alone is methodologically unsound.

    3. Debunking “Distillation-Only, Cut-Off-Stalls”: Three Overlooked Asymmetries

    3.1 Cost Asymmetry: Followers Benefit More from “Standing on Shoulders”

    Using “whether it can run” to prove “who came first” is backwards reasoning. Those accumulating knowledge face massive unknowns and trial-and-error; those standing on shoulders only absorb settled experience and can skip a great deal of low-value exploration. In technical products, followers often overtake leaders through lower cost, faster iteration, and closer fit to scenarios. LLMs follow the same pattern—three examples:

    • Data efficiency: reusing public data and high-quality synthetic data lets followers reach a similar level with fewer tokens, then reinvest the saved resources into data quality and post-training.
    • Efficiency optimization: sparse mixture-of-experts, attention approximations, quantization-aware training, and KV-cache management are general techniques that do not depend on any vendor’s proprietary technology, yet they cut training and inference costs significantly. The compute freed up can then go into more targeted domain data and post-training.
    • Scenario depth: vertical-industry corpora, knowledge, and feedback data naturally reside with local industry participants, independent of whether the upstream model is open.

    3.2 Time Asymmetry: Constraints Change the Curve but Rarely Reverse the Trend

    Attributing all currently accumulated capability to distillation is like staring only at someone’s existing wealth while ignoring how they obtained it. Likewise, once an important capability appears, public research, open weights, and talent mobility usually let the “prototype capability” rapidly reappear and evolve in different teams’ own ways. Restricting upstream growth can at most slow how fast the upstream “grows taller”; it almost never makes already-spread capabilities disappear again. Meanwhile, followers sit under different resource constraints and tend to trade architecture and training-efficiency gains for parity in real performance with the upstream.

    The argument need not stay at the level of pure logic; two technical facts support it:

    • Openness: most capability diffusion today happens atop public research, open weights, and general toolchains. If two capabilities both come from public information, claiming that one is unique while the other is copied simply does not hold.
    • Convergence of generational gaps common in tech catch-up: after a new architecture appears, different teams gradually achieve similar results under their own constraints. If followers could “only distill,” such parallel breakthroughs based on public ideas could not occur.

    So a more reasonable judgment is that constraints create friction but rarely a “permanent rift.” The real watershed is whether the follower has completed the switch from “inheriting capability” to “autonomous iteration.”

    3.3 Objective Asymmetry: Defenders Struggle; Followers Often Win on Lower Cost

    This matches common experience in business competition: incumbents must hold an absolute lead in every key scenario, while followers only need to be “good enough and better” in some core scenarios. Cost structure and scenario fit often decide business outcomes more than absolute capability. The reasons come at three levels:

    • Technically: post-training and alignment largely depend on verifiable signals. Tasks such as math, coding, and tool-calling can be automatically checked by programs, so reward signals do not need human annotation. “Self-play”–style training does not rely on an external teacher—internal verifiable signals alone can keep improving capability. Once a model has this self-generate, self-verify, self-iterate loop, its development no longer depends on whether the upstream is open.
    • Engineering-wise: inference-side quantization, distillation, and sparsification optimizations can be tailored to specific scenarios to reach a “good enough” result at lower cost.
    • Ecosystem-wise: feedback data settles into local toolchains, industry data, and user feedback, forming a “better with use” positive feedback loop.

    This is the “supply cutoff” theory’s biggest blind spot: it assumes capability travels only along a one-way road from the upstream, ignoring an autonomous evolution channel that does not depend on the upstream.

    4. Judging Model Provenance: Making the Gray-Box Narrative Testable

    The “supply cutoff” theory resists falsification precisely because of the gray-box state. To turn the gray box into verifiable checkpoints, examine four layers:

    1. Documentation and config: are technical reports, data mixtures, and training details public; are config-file hashes identical; are the hyperparameters alike.
    2. Weights and vocabulary: does it directly reuse upstream weights or a tokenizer; do vocabulary intersections and rare-token distributions match; is the vocabulary size consistent with the hidden dimension.
    3. Behavior: do outputs overlap heavily; are probability distributions, error patterns, and confidence calibration consistent under identical inputs; is the handling of rare languages and obscure symbols the same.
    4. Representation: are the similarity structures of hidden states (for example, linear CKA) notably high—far above independent models of the same size and corpus.

    These methods align with common practice in the model-theft detection field, and vocabulary fingerprints and representation similarity are especially hard to fake.

    4.1 From “What It Can Do” to “How It Fails”: Distillation Inherits Failure Modes, Not Correct Answers

    An independently trained model may answer a question correctly yet still err differently, because the error distribution depends on the data distribution and training dynamics. A distilled student copies the teacher’s characteristic error patterns and shows confidence-distribution biases such as being “confidently wrong.” So comparing error types, confidence, and calibration curves on hard sample sets is often more discriminative than comparing accuracy.

    4.2 A Black-Box Technique for “Reading” the Model: Vocabulary Fingerprints

    Vocabulary is a relatively static, high-disclosure layer in LLMs, which makes it well suited for initial screening. A tokenizer vocabulary often has tens of thousands of entries; if two models’ vocabularies are nearly identical, they most likely share the same tokenization scheme. When the hidden dimension, attention-head count, and other hyperparameters also match exactly, the likelihood of an inheritance relationship rises sharply.

    The sample code below shows how to compare two config files with the standard library and output structural similarity metrics:

    1. import hashlib
    2. import json
    3. import math
    4. # Structural fingerprint fields: model weights are not read directly,
    5. # only public architecture configs are compared.
    6. FIELDS = [
    7. "vocab_size", "n_layers", "n_heads", "hidden_size",
    8. "intermediate_size", "max_position_embeddings",
    9. "layer_norm_eps", "rms_norm_eps",
    10. "rope_theta", "torch_dtype", "model_type",
    11. ]
    12. def fingerprint(model_dir: str) -> dict:
    13. with open(f"{model_dir}/config.json", "r", encoding="utf-8") as f:
    14. cfg = json.load(f)
    15. fp = {}
    16. for key in FIELDS:
    17. if key in cfg:
    18. raw = json.dumps(cfg[key], sort_keys=True, ensure_ascii=False)
    19. fp[key] = hashlib.sha256(raw.encode()).hexdigest()[:8]
    20. return fp
    21. def compare(fp_a: dict, fp_b: dict) -> dict:
    22. common = set(fp_a) & set(fp_b)
    23. # Share of fields with identical values: a preliminary signal of structural consistency
    24. same = [k for k in common if fp_a[k] == fp_b[k]]
    25. return {
    26. "matched_fields": len(same),
    27. "compared_fields": len(common),
    28. "identical_rate": round(len(same) / max(len(common), 1), 4),
    29. }
    30. if __name__ == "__main__":
    31. a = fingerprint("/path/to/model_a")
    32. b = fingerprint("/path/to/model_b")
    33. print(compare(a, b))

    This code is only an example. A real comparison must also cover the vocabulary content itself (not just its size), the tokenizer type and special tokens, and whether the training hyperparameters match. In addition, “high structural similarity” does not equal “capabilities come from distillation”: teams working on the same problem in the same period may well make highly similar architectural choices. The role of structural fingerprints is to narrow the search and generate hypotheses—not to deliver conclusions.

    4.3 Reproducible Experiments at the Behavior Layer

    Beyond config comparison, the core is a set of reproducible behavioral control experiments:

    1. Zero-temperature consistency: on a fixed prompt set, measure the fraction of identical outputs; for generative tasks, use metrics such as edit distance and semantic similarity to gauge closeness.
    2. Teacher-coverage test: test the student on capabilities the teacher never saw (for example, specific domain knowledge or special languages). If the student unaccountably inherits them, the evidence of an inheritance relationship strengthens markedly.
    3. Error-pattern comparison: compare error types, confidence distributions, and calibration curves on the hard set.
    4. Representation similarity: compute the hidden-state CKA. This opens another window onto the “gray box”—without knowing the weights themselves, you can judge whether the two share an origin.

    4.4 A Necessary Caveat: Don’t Treat “Same Origin” as “Plagiarism”

    “Structural similarity” only indicates that the two models may share an origin; it cannot distinguish open-source inheritance, legitimate fine-tuning, method reproduction, and unauthorized distillation. Judging legality must return to licenses, terms of use, and data compliance. Technical detection answers “same origin or not,” while legal and ethical judgment answers “legitimate or not”—the two cannot substitute for each other. This is the step enterprises most easily overlook in model selection and compliance audits.

    5. Enterprise Practice: From “Being Able to Call” to “Owning and Sustaining”

    Having broken down the three premises of the “supply cutoff” theory, from the enterprise’s perspective it ultimately points to an engineering problem: how to reduce single-source dependence on external model capabilities and turn “borrowed capability” into “own assets.” Whatever technical route is chosen, the full engineering pipeline is unavoidable: data preparation, training, evaluation, deployment, and continuous iteration. Data, compute, algorithms, and organizational coordination are all indispensable.

    5.1 Defining Inheritance as “Learning From”: Avoid Reinventing the Wheel

    Enterprises need not reject inheritance; they should confine it to a legal, reproducible scope and treat public research as a foundation to learn from, rather than something to copy word for word. Section 2.2 also offers another path: under a permitting license, directly continue training on open-weight models. This is usually more controllable and lower in cost than training from scratch, and it runs entirely on the enterprise’s own technology stack and data.

    5.2 Building Enterprise-Owned Capability Assets and Positive Feedback

    1. Data assets: turn business data into trainable corpora, and build pipelines for collection, cleaning, deduplication, and quality assessment—while protecting privacy and compliance boundaries.
    2. Evaluation system: build evaluation sets covering general ability, domain ability, and safety/compliance, serving as a yardstick for continuous iteration.
    3. Post-training: use instruction tuning, preference optimization, and reinforcement learning on verifiable tasks to fit the model to business scenarios.
    4. Deployment and operations: watch inference cost, latency, and stability, and establish monitoring, alerting, and version management.

    These capabilities can be supported by general cloud services on Baidu AI Cloud. Data governance can draw on object storage and data-processing platforms; elastic compute can come from container platforms; and the reliability of evaluation and inference services can rest on logging services and monitoring/alerting. These are general cloud-infrastructure capabilities that can be combined as needed.

    5.3 Making the “Borrow vs. Build” Ratio a Tunable Parameter

    Enterprises can treat “external-capability share” and “autonomous-iteration capability” as two tunable parameters: when external dependence is high, direct resources into data governance, evaluation, and scenario fine-tuning; once autonomous capability matures, gradually raise the in-house share. This captures the efficiency dividend of inheritance while avoiding being led astray by the “supply cutoff” narrative.

    6. Breaking Through at the Communication Level: Turn “Rote Repetition” into “Evidence”

    Back to communication. The word “distillation” takes only a moment to repeat, while refuting it takes lengthy argument—which is exactly why it wins on spread. A more effective strategy is not to get stuck in back-and-forth debate, but to let verifiable facts speak:

    1. Definition first: distinguish “has used distillation” from “only knows distillation,” pulling the discussion from a clash of stances back to a contest of facts.
    2. Provide evidence: publish technical reports, vocabulary and config notes, evaluation results, and version records—turning the gray box into a checkable white box.
    3. Provide commitments: give clear, verifiable policies on open weights, license terms, and data compliance—more persuasive than any debate.
    4. Provide time: prove that “autonomous evolution” genuinely exists through continuously iterating versions, rather than through a one-off capability demonstration.

    In summary, distillation is a mature technical tool, and using it does not mean it is the entire source of capability; the “cut off supply → stall” inference ignores the real channel by which followers achieve autonomous evolution through inheritance, efficiency optimization, and scenario depth. Rather than arguing “is it distillation or not,” it is better to focus energy on “can it keep iterating.” For enterprises, rather than worrying “is someone else distilling,” it is better to invest resources in data, evaluation, and scenario capabilities—assets that will not disappear when external models open or close.