From L0 to L1: The Layered Logic and Practical Path from Foundation Models to Domain Models

    A practical breakdown of how large models are layered into L0 foundation models, L1 domain models, and L2 scenario-specific capabilities. It explains what truly defines a foundation model, how continued training and alignment turn a general base into a domain model, and how enterprises can cut through "model" marketing hype to choose the right layer for their needs—covering corpus governance, data mixing, evaluation loops, cloud deployment architecture, and common pitfalls like catastrophic forgetting.

    1. A Sober Look Amid Concept Inflation

    Ever since general-purpose conversational models took the market by storm, the term “large model” has fallen into rapid inflation. New labels appear constantly—financial models, industrial models, weather models, government-affairs models—as if any team that plugs into general capabilities can claim to own a “某某 large model.” This inflation troubles not only end users but also the teams and decision-makers who are genuinely investing in foundational R&D.

    A widely cited judgment holds that the teams actually building a foundation model can be “counted on one hand.” Why is there such a vast gap between the appearance of a “hundred-model war” and the reality of a small minority? The root cause is the lack of a clear definition for what “foundation model” actually means. Once you introduce an L0/L1 layering framework, the chaotic market picture becomes clear almost immediately: the vast majority of so-called “large models” on the market are domain models built on top of someone else’s base, while teams that train a general-purpose base from scratch are exceedingly rare.

    Understanding this layering is not merely a matter of definitions—it directly affects an enterprise’s technology choices, cost structure, and long-term evolution strategy.

    2. The L0 Layer: What Is a Foundation Model

    A foundation model (the L0 layer) refers to a general-purpose base model obtained through large-scale self-supervised pre-training on massive, general-purpose, unlabeled or weakly labeled corpora. It targets no single industry; instead, through general objectives such as next-token prediction, it learns the general patterns of language and even multiple modalities. Its core characteristics can be summarized in four points:

    • Scale-driven: Parameter count, training data volume, and compute investment all reach extremely high magnitudes, making it a typically capital- and engineering-intensive product.
    • Generality first: The goal is not to solve a specific business problem but to form transferable general representations and reasoning capabilities.
    • Emergent capabilities: Once scale crosses a certain threshold, the model exhibits abilities that were never explicitly trained—few-shot learning, instruction following, and multi-step reasoning.
    • High barrier and high cost: The compute clusters, data governance, distributed training frameworks, and top-tier talent required for pre-training are all indispensable.

    Precisely because the barrier is so high, the players at the L0 layer are naturally few. This is also the most easily misunderstood part of the “hundred-model war” narrative—most participants are not competing at this layer at all.

    3. The L1 Layer: Domain Models and Continued Training

    The L1 layer (domain/industry models) refers to models built on top of an L0 base by injecting industry data and domain knowledge. The key technique is continued training, which follows two common paths:

    The first is continued pre-training. On top of the general base, you continue self-supervised training using large-scale domain corpora (such as a vertical field’s literature, tickets, procedures, and code) so the model acquires that field’s distinctive terminology, expression conventions, and knowledge structures. This step emphasizes “adding knowledge.”

    The second is alignment and fine-tuning. After continued training, instruction tuning teaches the model to follow task instructions, and then preference alignment based on human feedback makes outputs conform more closely to business norms and user expectations. This step emphasizes “changing behavior.”

    It is worth stressing that the value of an L1 model depends heavily on the quality of its L0 base. The stronger the base’s general reasoning ability, the more pronounced the “standing on the shoulders of giants” effect for the domain model; conversely, if the base is underpowered, no amount of domain data can fully compensate. Therefore, choosing which L0 base to build on is often the single most critical first decision when constructing an L1 model.

    4. Beyond the Layers: L2 Scenario-Specific Capabilities

    In practice, many needs do not require going from L0 all the way to L1. For scenarios with frequently updated knowledge, requirements for traceability, or data that is unsuitable for large-scale training, a lighter approach is to layer retrieval-augmented generation (RAG), prompt engineering, and lightweight fine-tuning (such as parameter-efficient fine-tuning) on top of L1—or even directly on L0. This layer can be understood as an L2 capability layer aimed at specific scenarios. Mapping the three together:

    Layer Positioning Main Techniques Typical Cost
    L0 General foundation base Large-scale self-supervised pre-training Very high
    L1 Domain/industry model Continued pre-training + alignment fine-tuning Medium to high
    L2 Scenario/application capability RAG, prompt engineering, lightweight fine-tuning Low to medium

    What most enterprises actually need is a combination of “reusing L0 + building L1/L2 themselves,” rather than training a base from scratch.

    5. Enterprise Selection: How to Tell Whether It’s a Foundation Model

    Faced with the endless stream of “large model” labels on the market, technical decision-makers can quickly screen candidates with three questions:

    1. Is it pre-trained from scratch? If a model’s capabilities clearly depend on some general base, it is essentially an L1 model, not an L0.
    2. Are its training data sources credible and sustainable? The performance ceiling of a domain model is determined by data quality, not by slogans.
    3. Does it have publicly evaluable benchmarks? Products that lack comparative evaluation and offer only marketing talk should be treated with caution.

    The purpose of this judgment is not to disparage L1 but to help enterprises do the right things at the right layer: assess L0 supply when base-level capability is needed, and focus on L1/L2 construction when business deployment is the goal.

    6. A Typical Technical Path from L0 to L1

    Taking the construction of a domain model as an example, the general engineering workflow can be abstracted as follows (pseudocode that illustrates the process only, without referencing any specific product interface):

    1. # General workflow for building a domain model
    2. base_model = load_foundation_model() # Choose an L0 base
    3. corpus = curate_domain_corpus() # Domain corpus cleaning and deduplication
    4. model = continued_pretrain(base_model, corpus) # Continued pre-training from L0 -> L1
    5. model = instruction_tune(model, task_pairs) # Instruction alignment; learn to follow tasks
    6. model = align_preference(model, feedback) # Preference alignment; conform to business norms
    7. # Combine retrieval augmentation on the inference side to keep knowledge updatable and traceable
    8. def answer(query):
    9. context = retrieve(query) # Retrieve relevant business knowledge
    10. return model.generate(query, context) # Generate output with citations

    Key points for each stage:

    • Corpus governance: Deduplication, denoising, privacy de-identification, and copyright compliance are the lifeline of continued pre-training; dirty data directly corrupts model behavior.
    • Data mixing: The ratio of general data to domain data needs repeated tuning; an imbalanced ratio leads to “learning the new while forgetting the old” (catastrophic forgetting).
    • Evaluation loop: Build a domain-specific evaluation set covering accuracy, safety, format compliance, and other dimensions, and run continuous regression testing.
    • Cost control: Compute is the biggest variable; budgeting and scheduling should account for training scale, GPU memory usage, and throughput.

    7. Cloud Deployment Architecture

    To build and operate the models above in the cloud, you can usually assemble a complete closed loop by reusing a few general-purpose cloud capabilities:

    • Object storage: Holds massive training corpora, model weights, and dataset version management.
    • Container platforms and compute scheduling: Manage elastic resources for training and inference tasks, improving GPU utilization.
    • Message queues: Decouple pipeline stages such as data preprocessing, training, and evaluation, ensuring high throughput and retryability.
    • Logging services and monitoring/alerting: Track training metrics, inference latency, error rates, and cost to surface problems promptly.

    For example, general-purpose capabilities offered by cloud providers such as Baidu AI Cloud—compute, storage, containers, data processing, and monitoring—can serve as one of the infrastructure options for hosting this kind of layered model engineering, helping enterprises focus their energy on data and business rather than low-level operations.

    8. Common Challenges and Best Practices

    • Challenge 1: Confused concepts lead to wrong choices. Best practice: use the L0/L1/L2 layering as your yardstick and clarify which layer you actually sit on.
    • Challenge 2: Catastrophic forgetting. During continued training, retain a certain proportion of general corpora and run regression evaluations.
    • Challenge 3: Knowledge freshness and hallucination. For volatile knowledge, prefer RAG over training injection to ensure traceability.
    • Challenge 4: Imbalance between cost and effect. First validate value with lightweight means (prompts, RAG, lightweight fine-tuning), then decide whether to invest in higher-order continued training.
    • Challenge 5: Security and compliance. Both the training and inference ends require content-safety and data-compliance governance.

    9. Conclusion

    Beneath the noise of the “hundred-model war,” what remains truly scarce is always the L0 foundation model, while the vast majority of commercial value is actually born from the scenario-specific construction of L1 and L2. Once the layering logic is clear, enterprises no longer get swept along by concepts when making choices: when base-level capability is needed, evaluate the supply rationally; when deployment is needed, do the data, training, and evaluation work solidly. Technology evolves quickly, but a clear understanding of the layers is the underlying capability for cutting through concept inflation and making correct engineering decisions.