Engineering Generative AI in Production: Foundation Models vs. Domain-Specific Approaches

    This article examines the key technical decisions behind deploying generative AI in production. It explains the L0 (foundation model) and L1 (domain model) layering, clarifies why the term "large model" has become inflated across the market, and breaks down the dual cost pressures of training and inference along with optimization techniques such as quantization, distillation, and PEFT/LoRA. It also addresses the "last mile" delivery challenge solved through RAG, prompt engineering, and business system integration, and weighs the strategic trade-offs between B2C and B2B. Readers will gain practical principles for choosing the right layer to enter, controlling real costs, and delivering stable business value.

    Introduction: From “Model Emergence” to “Implementation Anxiety”

    Since large-scale pre-trained models entered the public eye, the term “large model” has gone through rapid inflation. New concepts keep appearing—financial large models, industrial large models, weather large models, government large models, and so on. This conceptual expansion has a direct consequence: not only do end users struggle to judge the real capability boundaries of different products, but even upstream foundation model developers face a practical question—how to get their models accurately recognized in a crowded market and matched with the right deployment scenarios.

    One fact is worth clarifying: the teams that actually build foundation models from scratch are very few. Many products labeled as “industry large models” are, in essence, application-level outputs produced by doing domain-specific continued training on top of an existing foundation model. Understanding this is the premise for viewing today’s competitive landscape and deployment paths rationally.

    L0 and L1 Layering: The Boundary between Foundation and Domain Models

    The industry often uses a layered framework to distinguish different levels of model capability:

    • L0 (Foundation layer / Foundation Model): Models pre-trained from scratch on massive, general-purpose corpora, possessing broad language understanding, generation, and reasoning capabilities. This is the base for everything above it.
    • L1 (Domain layer / Domain Model): Built on top of L0, using domain data for continued pre-training or fine-tuning so that the model has more specialized knowledge and expression in a specific field (such as healthcare, law, or finance).
    • Application layer: On top of L1, model capabilities are packaged into deliverable business systems through techniques such as prompt engineering, retrieval-augmented generation (RAG), tool calling, and agent orchestration.

    A common misconception is to treat L1 domain models as peers of L0. In reality, building L1 depends heavily on the quality of L0: the stronger the foundation model’s general capabilities, the lower the marginal cost of domain adaptation and the higher the performance ceiling. This is also why investment in the foundation layer has such a strong “leverage effect”—a single high-quality pre-training run can be reused by countless upper-layer applications.

    Why Terminology Inflation Troubles the Market

    The essence of terminology confusion is a lack of transparency around capability boundaries. When every product calls itself a “large model,” buyers cannot tell whether a solution is an in-house foundation model or a fine-tuned wrapper, nor can they evaluate its real performance in specific scenarios. For technical teams, this chaos means having to make a resource trade-off between “following” and “building in-house”:

    1. Follow strategy: Build domain adaptation and products on top of mature foundation models. Low investment, fast results, but limited differentiation and a tendency toward homogeneous competition.
    2. In-house foundation strategy: Start from L0 to build a long-term moat, but this requires enormous compute, data, and talent investment, with a long payback period.

    The rational approach is to first clarify what the target scenario requires in terms of capability, then decide which layer of the stack to enter—rather than blindly chasing concepts.

    The Cost Problem: Dual Pressure of Training and Inference

    One of the core constraints on deploying generative AI is cost, which runs through the entire chain from training to serving.

    Training cost: Pre-training a foundation model from scratch requires a massive GPU cluster, vast amounts of high-quality corpora, and lengthy tuning and iteration—a classic heavy-asset investment. This is also the fundamental reason why “only a handful of teams truly build L0.”

    Inference cost: Once a model goes live, every request consumes compute. Inference cost is affected by input/output length, concurrency, context window, and other factors. An easily overlooked point is the KV Cache—during autoregressive generation, computed key-value pairs must be cached for reuse, and their GPU memory footprint grows linearly with context length, directly determining how much concurrency a single card can support.

    Optimization techniques: To control costs, engineering commonly uses the following techniques:

    • Quantization: Compressing weights from FP16 to INT8 or even INT4, significantly reducing memory footprint and compute overhead, at the cost of a small loss in accuracy.
    • Distillation: Using a large model (teacher) to guide the training of a small model (student), so the small model approximates the large model’s performance on specific tasks, thereby lowering inference cost.
    • Parameter-efficient fine-tuning (PEFT): Such as LoRA, which trains only a small number of low-rank incremental parameters, greatly reducing the compute and storage required for fine-tuning.

    A sample LoRA fine-tuning configuration:

    1. # LoRA (parameter-efficient fine-tuning) configuration example
    2. lora_config = {
    3. "r": 8, # Rank of the low-rank matrices
    4. "lora_alpha": 16, # Scaling factor
    5. "target_modules": [ # Inject trainable parameters only into attention projection layers
    6. "q_proj", "k_proj", "v_proj", "o_proj"
    7. ],
    8. "lora_dropout": 0.05, # Regularization to mitigate overfitting
    9. "bias": "none"
    10. }
    11. # Compared with full fine-tuning, the number of trainable parameters
    12. # typically drops by one to two orders of magnitude, allowing
    13. # post-training and experiment iteration on a single card or a few cards

    The “Last Mile” Delivery Challenge

    Even when a model’s capabilities are strong enough, serving each customer to the “last inch” still requires a lot of manual effort. The reason is that general-purpose models output probabilistic text, while enterprise customers need stable, controllable results that fit business rules. The gap in between often has to be bridged by the following engineering:

    • Retrieval-augmented generation (RAG): Connect the model to a private enterprise knowledge base so it answers based on real-time, accurate data, mitigating hallucination.
    • Prompt engineering and structured output constraints: Through system prompts, few-shot examples, JSON Schema constraints, and similar methods, make outputs controllable and parseable.
    • Tool calling and business system integration: Embedding the model into enterprise systems such as CRM, ticketing, and approval workflows is what truly produces business value.

    A minimal RAG pipeline:

    1. User question
    2. ↓
    3. Query rewrite / vectorization
    4. ↓
    5. Vector retrieval → Top-K relevant document chunks
    6. ↓
    7. Assemble prompt (question + context + constraints)
    8. ↓
    9. Large model generation → result validation / citation traceability
    10. ↓
    11. Return structured answer

    It is precisely this characteristic—“every customer and every scenario needs to be re-polished”—that makes purely B2C or purely standardized products hard-pressed to cover deep needs, and keeps the manual cost of delivery high.

    Strategic Trade-offs between B2C and B2B

    One noteworthy phenomenon: for many teams, the primary purpose of building a B2C product is actually to demonstrate model capabilities to B2B customers. B2C applications have a low barrier, reach users fast, and are easy to turn into intuitive demonstrations—an effective vehicle for validating and showcasing capability—while the real commercial revenue and long-term value often come from deep B2B integration and continuous service.

    In the B2B market, the core challenges technical teams must face include:

    1. Data compliance and privacy security: Keeping enterprise data within its domain, auditable, and controllable is a hard gate for procurement.
    2. Effect stability: Enterprise scenarios demand far higher accuracy and explainability than consumer products.
    3. System integration cost: Models need to be seamlessly embedded into existing IT architectures, which depends on the coordination of general infrastructure capabilities such as object storage, message queues, container platforms, logging services, and monitoring/alerting.

    B2C and B2B are not an either-or choice; they can form a closed loop of “capability demonstration—reputation validation—deep delivery”: use B2C to validate and refine capability, and use B2B to achieve commercial closure and scale.

    Practical Recommendations for Engineering Implementation

    Based on the above constraints, here are several actionable principles:

    1. Layer first, then select: Clarify whether the target sits at L0, L1, or the application layer, avoiding resource waste caused by mismatched concepts.
    2. Reuse rather than rebuild: Prioritize reusing mature foundation models, concentrating resources on high-value areas such as domain data, RAG, and scenario refinement.
    3. Evaluate cost upfront: Estimate training and inference compute costs at the design stage, and leave room for optimization through techniques like quantization, distillation, and caching.
    4. Use retrieval to supplement knowledge, engineering to supplement stability: Compensate for the model’s knowledge-timeliness and output-stability shortcomings through RAG and structured constraints.
    5. Validate with B2C, monetize with B2B: Form a positive feedback loop between capability demonstration and commercial delivery.

    Conclusion

    Which scenarios generative AI can be used well in still lacks industry consensus, and deployment remains in a rapidly evolving, chaotic phase. But one thing is certain: rationally viewing the L0/L1 layering, facing up to the dual costs of training and inference, and facing up to the weight of “last mile” delivery are must-answer questions for every technical team. Rather than chasing endless new concepts, it is better to return the focus to the fundamentals of engineering: choose the right layer to enter, control real costs, and deliver stable value. This is the reliable path for cutting through the chaos and truly putting large models to practical use.