L0 vs. L1: How to Build from Foundation Models to Industry Models
As AI-generated terminology multiplies—finance models, industrial models, weather models, government models—the word "large model" has become overloaded, obscuring the fact that most products labeled this way sit at very different technical levels. This article uses an L0/L1 layering perspective to separate foundation models from industry models. It defines each layer, outlines the core traits that make a model a true L0 (data breadth, self-supervised objectives, compute scale, transferability), and explains two main paths for building L1 domain models on top of an L0 base: continued pre-training and supervised fine-tuning with alignment. It also covers the end-to-end engineering loop of compute, data pipelines, evaluation, and serving, and closes with practical guidance for anyone making technical build-versus-adopt decisions.
1. Concept Inflation: When “Large Model” Becomes an Overused Label
Discussions around AI large models have stayed intensely popular, and a new term seems to appear every few weeks: finance large model, industrial large model, weather large model, government large model, and so on. On one hand, this proliferation shows that the technology is rapidly penetrating every industry. On the other, it has created serious conceptual confusion.
For end users, this inflation makes it hard to tell what a given model can actually do. For engineering teams genuinely working on foundation model R&D, it hides a critical technical fact: most products marketed as “large models” do not operate at the same technical level. Clarifying these levels is the prerequisite for understanding the industry landscape—and for making the right technical choices.
This article centers on one question: how to use an L0/L1 layering view to clearly distinguish “foundation models” from “industry models,” and, on that basis, to explain how to build genuinely usable industry capabilities from a base model.
2. L0 and L1: A Clear Definition of the Model Hierarchy
A widely adopted approach in the industry divides the model stack into two abstraction levels.
L0 layer: Foundation Model
L0 refers to a general-purpose base model obtained through large-scale self-supervised pre-training on massive, general-purpose, task-agnostic corpora. It does not target any specific business scenario directly; instead, it pursues broad language understanding, knowledge storage, and generation capabilities. It is the source of capability for the entire stack.
L1 layer: Domain / Industry Model
L1 refers to a model that is further trained and aligned on top of an L0 base for a specific industry or task, so that it shows higher quality and stronger specialization in a particular vertical. An L1 model must depend on an L0 to exist and cannot be defined in isolation from its base.
To summarize the relationship in one sentence: L0 is the “foundation,” and L1 is the “floors” built on that foundation. An industry can have many L1 models, but the number of true L0 providers is very small. This also explains why, in a strict technical sense, only a handful of teams can genuinely claim to be “building foundation models.”
3. Core Characteristics of an L0 Foundation Model
You can evaluate whether a model is truly L0 along the following dimensions:
- Breadth and scale of training data: L0 relies on general corpora (public text, code, multimodal data, etc.) rather than single-industry data. Data scale typically reaches the trillions-of-tokens level.
- Self-supervised training objective: The core training goal of L0 is self-supervised learning (e.g., predicting the next token), without depending on manually labeled task tags.
- Order of magnitude of compute: Fully training an L0 requires a GPU cluster of a thousand cards or more, running for weeks or even months. This is an important threshold for separating a “real L0” from a wrapper.
- Transferability and generality: L0 must be able to transfer in zero-shot/few-shot fashion to many downstream tasks, rather than being optimized for a single task.
It is worth emphasizing that what determines whether a team can build an L0 is not only algorithms, but the combined capability of compute, data engineering, and training stability. Many products that claim in-house large models are actually interface wrappers around an existing base. That belongs to the application layer, not L0 R&D.
4. How to Build an L1 Industry Model: Two Technical Paths
Once you have (or have trained) an L0 base, there are two main technical paths for building an L1 industry model.
Path 1: Continued Pre-training
Continued pre-training is the key means of letting L0 absorb domain knowledge. The approach is to continue self-supervised training on top of the base model using large-scale unlabeled domain corpora, so the model internalizes industry terminology, specialized knowledge, and domain conventions.
# Continued pre-training example: injecting domain knowledge into the base model# Note: the following is framework-agnostic pseudocode illustrating the training# logic; it is not a specific framework API# 1. Load the general-purpose foundation modelmodel = load_foundation_model(base_checkpoint)# 2. Prepare the domain corpus (no manual labeling needed, only large-scale domain text)domain_dataset = build_corpus(domain_raw_text)# 3. Continue training with the same self-supervised objectivetrainer = ContinuePretrainTrainer(model=model,dataset=domain_dataset,learning_rate=small_lr, # Continued pre-training usually uses a smaller learning# rate to avoid catastrophic forgetting)trainer.train()
The key challenge in continued pre-training is preventing catastrophic forgetting: a learning rate that is too high, or domain data that is too narrow, can damage the base model’s original general capabilities. In engineering practice, this usually requires mixing in a certain proportion of general corpus and running strict capability regression tests.
Path 2: Supervised Fine-tuning & Alignment
When an industry model needs to follow a specific instruction format, produce structured output, or align with business conventions, you fine-tune and align it after continued pre-training. This typically includes:
- Supervised Fine-tuning (SFT): Using paired “instruction–response” data so the model learns to respond in the expected format.
- Preference alignment (e.g., RLHF/DPO-style approaches): Making the model’s outputs better match human preferences and business values.
The comparison looks like this:
| Dimension | Continued Pre-training | Fine-tuning / Alignment |
|---|---|---|
| Data form | Large-scale unlabeled domain corpus | High-quality instruction/preference-labeled data |
| Objective | Inject domain knowledge | Align behavior, format, and conventions |
| Typical scale | Tens of billions to hundreds of billions of tokens | Thousands to millions of samples |
| Main risks | Catastrophic forgetting, training instability | Overfitting, degraded general capability |
5. Engineering in Practice: The Complete Loop of Compute, Data, and Operations
Building L0→L1 is not a one-off training task but an engineering system that must run continuously. Every one of the following elements is essential:
Compute and training frameworks: Large-model training depends heavily on high-performance GPU clusters and distributed training frameworks (data parallelism, tensor parallelism, pipeline parallelism, etc.). For teams without in-house L0 capability, a more pragmatic approach is to build L1 on top of mature L0 base capabilities rather than duplicating the investment.
Data engineering: Collecting, cleaning, deduplicating, anonymizing, and mixing high-quality industry data often determines success more than the algorithm itself. It is advisable to establish a data quality pipeline and version management to keep corpora traceable and reproducible.
Evaluation system: You need a dual evaluation mechanism—both domain-specific quality evaluation (e.g., industry Q&A accuracy) and general-capability regression tests—to ensure the L1 does not lose basic capabilities while becoming more “specialized.”
Model serving and operations: After training is complete, models must be deployed elastically on a container platform, with model weights and training artifacts managed in object storage, online requests handled through message queues and API gateways, and full-stack observability provided by logging and monitoring/alerting services. For example, general cloud product categories such as container platforms, object storage, logging, and monitoring/alerting can support the complete engineering chain from training to inference, so a team does not have to build the infrastructure from scratch. (Availability and scope vary by region; consult the official documentation for the specific services you need.)
6. Practical Advice for Decision-Makers
Based on the analysis above, here are a few actionable recommendations:
- Know which layer you are on: If your goal is simply to solve a specific business problem, prioritize building L1 on a mature L0 rather than blindly building your own base.
- Beware the “wrapper” trap: Calling general capabilities through an interface does not create a real industry moat; genuine differentiation comes from domain data and continuous iteration.
- Move in small steps—fine-tune before pre-training: In most scenarios, high-quality fine-tuning is enough. Only when domain knowledge differs dramatically should you consider the costly investment of continued pre-training.
- Treat evaluation as a first-class citizen: Without a solid evaluation system, you cannot objectively judge whether an L1 is truly better than a general-purpose L0.
Conclusion
The inflation of the term “large model” is, at its core, a case of market narrative running ahead of technical definition. The L0/L1 layering view gives us a ruler for re-understanding the industry: foundation models are a scarce foundation, and industry models are pragmatic buildings. For developers, understanding this technical path from L0 to L1 is what lets you find your place amid the proliferation of concepts and invest limited resources where they genuinely create value.
