Vertical LLMs vs General-Purpose Models: Task Fit, Training Strategy, and Cost

    This article explains how to decide between a vertical-domain large language model and a general-purpose model, and why that choice is fundamentally a task problem rather than a parameter-count problem. It covers what supervised fine-tuning does and does not teach, how reinforcement learning improves strategy on verifiable tasks, why a small model can outperform a larger general model on a narrow task, and how to structure the training system so decoding no longer bottlenecks iteration. It also provides a minimal single-GPU workflow, practical guidance on rewards, parsing, and group-relative advantage, and a four-question checklist for making the selection. Readers will learn how to evaluate vertical models on correctness, stability, and unit cost instead of picking a side by model size.

    First, Separate “Domain Adaptation” from “Capability Ceiling”

    When comparing vertical-domain LLMs with general-purpose LLMs, don’t look only at parameter count or whether the corpus is “industry-specific.” A more useful distinction is whether the model has to answer broad questions in an open world, or solve well-bounded, verifiable, repeatable professional tasks.

    The strength of a general model is coverage. It has seen diverse tasks, writing styles, and fragments of knowledge, so it fits the exploration phase, situations where requirements change frequently, and cases where evaluation criteria are fuzzy—for example, general retrieval, cross-topic writing, and prototype validation. The cost is this: to stay general, both parameter count and inference cost stay high, and on any one narrow task the extra general capability doesn’t necessarily translate into a higher accuracy.

    The goal of a vertical model is not to be “a little good at everything” but to be reliably correct on a limited distribution. Mathematical derivation, code repair, receipt/invoice extraction, procedural Q&A, and ticket triage all have relatively clear inputs and outputs and results that can be verified automatically or semi-automatically. At that point, concentrating capacity on the task distribution, the solution steps, and the reward signal is often more effective than piling on more general corpus.

    So “vertical or general” is first a question about the task, and only second a question about model size. If the task is undecidable, the evaluation is highly subjective, and knowledge changes so fast that no rules can be distilled, the payoff from verticalization is limited. If you can construct a gold answer, run executable verification, or score with rules, a small model with targeted training has a chance to beat a larger general model on that specific point while keeping training and inference costs within an acceptable range.

    What Supervised Fine-Tuning Usually Learns

    In industry practice, the most common verticalization path is: collect Q&A, manuals, tickets, or expert-revised text, then run supervised fine-tuning (SFT). This step delivers real value, but its boundaries are also clear.

    SFT mainly aligns the conditional distribution: given an input similar to the examples, the output looks more like the demonstration text. It can quickly learn terminology, formatting, refusal conventions, citation habits, and safety boundaries, substantially reducing the “doesn’t sound like a domain expert” problem. For format extraction, standardized phrasing, and short-text classification, it is often already enough.

    What SFT struggles to break through on its own is multi-step logic. Demonstration data usually shows only one good trajectory, so the model sees “what the final answer should look like” rather than “which intermediate steps were rewarded and which fluent-looking steps actually count as errors.” When errors come from missed conditions, arithmetic mistakes, or wrong tool-call ordering—not from tone—simply imitating a gold-standard answer produces three typical phenomena.

    First, surface fitting. The model recites solution boilerplate and still follows the wrong template after key numbers or conditions are swapped. Second, out-of-distribution fragility. When the wording of training problems differs slightly from exam problems, accuracy drops noticeably. Third, the ceiling is set by demonstration quality. Experts rarely write out contrasting wrong answers and comparison reasoning, so the model lacks a “relatively better” gradient signal.

    Therefore, equating a vertical model with “one round of SFT on industry data” overestimates tone alignment and underestimates strategy improvement. SFT is a good fit for cold start and format alignment; but if the goal is accuracy, pass rate, or executable success rate, you also need a training signal that can compare multiple candidate outputs.

    Reinforcement Learning Adds Strategy, Not Another Round of Rote Learning

    On scorable tasks, reinforcement learning (RL) treats generation as a policy: sample multiple answers to the same question, use rewards to separate good from bad, then update the generation policy. Compared with fitting a single gold answer, it directly optimizes ranking and selection.

    This class of methods has a few prerequisites; otherwise it tends to collapse or waste compute.

    The reward must align with the business goal and be as verifiable as possible. For math problems, extract the final number and compare it; for code problems, run the tests; for structured extraction, use field-level matching; for customer-service scenarios, be careful—a pure fluency score rewards long, empty responses. When you can use rules, an executor, or a deterministic checker, prefer not to use another LLM as the judge, so you don’t also learn the judge’s bias.

    Sampling must be diverse. If the temperature is too low or the prompt locks the model down, the multiple answers are nearly identical and the advantage estimate carries no information. If diversity is too high and the reward is sparse, most samples score zero and gradient noise dominates the update.

    The update must constrain step size. When the new and old policies differ too much, high-reward samples push the model into an unstable region, showing up as format collapse, repetitive output, or reward hacking—the model learns to game the scoring rules instead of solving the problem. Common practice is to limit the probability ratio, mix in a divergence penalty against the reference distribution, and keep a validation set for early stopping.

    At the algorithm level, the classic Proximal Policy Optimization (PPO) learns both a policy and a value network, using the value network to estimate how good a state is before computing the advantage. The value network is itself a large-model-scale burden: one more set of parameters, an optimizer, and activations, so both memory footprint and implementation complexity go up. For tasks that only care about “which answer is better within the same question,” you can replace the absolute value with a group-relative advantage: generate a group of answers to the same question and use the mean or standardized result of the group’s rewards as a baseline. Answers above the group average get positive advantage, and those below get negative advantage. This removes one critic model to maintain, usually cuts memory noticeably, and shortens the engineering pipeline. The cost is that the advantage is only comparable within a group; the reward scale across questions still needs normalization, and too small a group means high variance.

    The illustrative loss below only expresses the “probability ratio times advantage” structure; a real system also needs clipping, masking, and regularization:

    1. def grpo_like_loss(new_logp, old_logp, advantage):
    2. ratio = (new_logp - old_logp).exp()
    3. return -(ratio * advantage).mean()

    It shows the optimization direction but is not a complete objective function ready for production. At the sequence level, you also need to mask the valid tokens so that padding and prompt positions don’t participate in the update.

    Why a Small Model Can Overtake on a Narrow Task

    Parameter count determines the ceiling of representable capability and the breadth of pretraining coverage, not the current policy quality on a given distribution. A large general model spreads its capacity across many capabilities that a specific business rarely uses; a small model, if its pretraining base is decent, can concentrate updates on a narrow strategy, and at inference each step is cheaper, so with the same budget it can sample more candidates and run more verification rounds.

    The cost difference usually comes from three places. During training, optimizer states and activations grow roughly linearly—or faster—with parameter count. During inference, autoregressive generation dominates the overall RL time, and a small model can raise throughput more easily. At deployment, concurrency and latency budgets let a vertical model run resident, whereas a large general model can often only be called per request. The result is a common pattern: on one-step math reasoning, patch generation under unit-test constraints, and information extraction with fixed patterns, a small model optimized with rewards achieves higher accuracy or pass rate than a general model without targeted optimization, while the per-request cost is an order of magnitude or more lower. The exact magnitude depends on the base model, data difficulty, whether the reward can be gamed, and whether the verification leaks—so you shouldn’t treat one experiment’s numbers as a universal law.

    The failure modes of a small model are equally clear. When the pretraining base is insufficient, RL cannot conjure missing knowledge out of nothing; it only reinforces wrong shortcuts. When the task requires long-horizon tool use, open-ended retrieval, or cross-domain analogy, capacity and knowledge coverage still matter, and the vertical small model should serve as a specialized component rather than replacing the entire general assistant.

    A more robust division of labor is: the general model handles cold-start data drafting, hard-case explanation, and low-frequency long-tail cases; the vertical model handles the high-concurrency, scorable, latency-sensitive main path. Compare the two on the same validation set, rather than concluding from a subjective “does it sound like an expert.”

    Training System: Split Out the Slowest Stage

    Large-model RL is often slow not because the loss formula is complex but because generation, scoring, and updating compete for the same resources. Generation needs high throughput and continuous batching; updating needs to hold gradients, optimizer states, and perform parameter synchronization. If everything is crammed into the training process, a lot of time is spent on autoregressive decoding, and memory is filled up by generation caches.

    A cleaner structure has four layers, each described with general capabilities rather than tied to a specific vendor’s implementation.

    The data layer reads samples, builds prompts, and records the problem ID along with the gold answer or test cases. Columnar files make it easy to scan in batches and to keep the raw problem, the analysis, and the labels separate, avoiding leaking evaluation labels into the prompt.

    The inference layer does only policy sampling: load the current weights, generate multiple completions for each prompt, and return the text and the old-policy log probabilities. It should support continuous batching, paged KV cache, and hot weight updates. This layer pursues throughput and does not compute gradients.

    The reward layer consumes the completions. Rule-based rewards parse the final answer, run tests, or compute field overlap; model-based rewards are used only where rules don’t cover, and they must be on the same scale as the rule-based rewards. The reward layer can scale horizontally, but it must be versioned to avoid silently changing the criteria mid-training.

    The training layer updates the policy based on old log probabilities, new log probabilities, and advantages, commonly using parameter sharding and gradient accumulation to fit single-machine memory. It is not responsible for large-scale decoding. A distributed scheduling layer places these roles in different processes or machines and handles queues, failure retries, and weight broadcasting.

    The data flow can be condensed into five steps: read a batch of problems; the inference side generates multiple answers; the reward side scores and either discards unparseable samples or records them as zero; use group standardization to obtain advantages and let the training side update; evaluate on a fixed validation set and save a checkpoint. Validation must use problems unseen during training, and the generation config must match production—otherwise the offline gains won’t transfer.

    Compared with the “synchronous decoding inside the training process” approach, handing decoding to a high-throughput inference engine commonly speeds up the overall iteration loop by a factor of several, not a few percent. The exact multiple depends on sequence length, group size, batch size, and network bandwidth, so confirm it with your own profiling data rather than copying someone else’s ratio.

    Rewards, Parsing, and Advantages: What Makes Training Actually Work

    Take verifiable math Q&A as an example: the reward doesn’t need to be complex. Extract the final number from the answer, compare it with the gold answer, give a positive score if correct and zero or negative if wrong. Parsing should tolerate thousands separators, plus/minus signs, and a little explanatory text, but should reject ambiguities like “the text contains multiple conflicting numbers and you take the last one,” otherwise the model learns to append the correct answer behind irrelevant calculations.

    The group advantage can be the reward minus the group mean, then divided by the group standard deviation plus a small constant. Groups that are all correct or all wrong have advantages near zero and contribute little to the update—that’s normal; but if many groups are all wrong, the problems are too hard or parsing is failing, and you should check the data first rather than raising the learning rate.

    The policy update acts only on answer tokens. The log probabilities of the prompt part don’t participate in the loss. Samples whose new/old probability ratio exceeds the clipping range should be constrained to prevent a single high-score trajectory from dominating. At the same time, log the reward mean, parse success rate, average length, clipping ratio, and validation accuracy. When length rises but accuracy stays flat, the model is usually piling on reasoning boilerplate; when parse success rate drops, either the format reward is insufficient or the sampling temperature is too high.

    For code generation, you can use the number of passing unit tests as the reward, but the tests must be hidden and time-limited, and the model must be forbidden from modifying the tests themselves. For information extraction, use exact field matching plus type checking. Open-ended writing lacks a hard reward, so it is not suitable as the main evidence for “a small model beating a large model” and is at most good for style alignment.

    A Minimal Loop That Also Runs on a Single GPU

    When resources are limited, you don’t need to chase multi-machine setups first. A reproducible minimal path is: pick a small base that already has basic instruction-following ability; prepare a training set and a strictly isolated validation set; use rule-based rewards to wire up “generation—parsing—scoring”; start with a medium group size, for example a few to a dozen-plus answers per problem, bounded by memory; use a small learning rate and short training runs, and first check whether the validation curve beats the SFT baseline.

    On a single high-memory GPU, the keys to saving memory are: don’t train a separate value network; time-share memory between inference and training; use gradient checkpointing; use a reasonable micro-batch size; and backpropagate only on the answer segment. If it’s still not enough, reduce group size and sequence length first, rather than turning off advantage normalization.

    Pitfalls can be summed up in a few points.

    Don’t mix validation problems into the training sampling pool. Don’t leak solving hints beyond the answer format into the prompt while giving the baseline no such hints in a comparison experiment. Don’t use a subjective judge that drifts as the model updates as the sole reward. Don’t silently treat reward parsing failures as zero without monitoring the ratio—that optimizes “format collapse” into a local optimum. Don’t report only training reward without independent validation. Don’t expect RL to become strong out of nowhere when the base can’t do the task at all; in that case, first do SFT with a small amount of high-quality trajectories, then turn on RL.

    For checkpoint strategy, select the model by validation metrics rather than training reward, and keep the reward-function version and generation config. Otherwise you can’t explain whether a later regression came from the data, the parsing, or the random seed.

    How to Make a Selection Instead of Picking a Side

    You can decide with four questions.

    Can the task result be judged automatically? If not, a general model with retrieval and human review is more realistic, and vertical RL easily misaligns the reward.

    Which is bigger—the cost of errors or the call volume? High-frequency, high-cost steps that can be regression-tested are worth training a vertical policy for; low-frequency exploration should keep using the general model.

    Can the base already get it right occasionally? If correct trajectories already appear in your samples, RL is amplifying rare correct behavior and is likely to succeed; if there are never any correct samples, you should first add data or switch to a stronger base.

    Are latency and cost blocking the business? If so, let a vertical small model carry the main path and the general model serve as fallback and hard-case handling; if not, and requirements change every week, verticalizing too early creates data and evaluation debt.

    An evaluation report should at least include: a comparison against SFT alone; a comparison against the general model under the same prompt and the same judging script; the parse failure rate; average latency; and per-call cost. A claim of “better results” without the same judging script has no reference value.

    Summary

    A vertical-domain LLM is not a shrunk-down copy of a general model; it aims capacity, rewards, and engineering budget at one verifiable distribution. SFT solves “does it sound right and follow the conventions”; RL solves “which trajectory should be sampled more” when a reliable reward exists. Using group-relative advantage to drop the value network, separating inference from training to raise decoding throughput, and using rule-based rewards to hold correctness in check, a small model has a chance to beat an unoptimized large general model on specific tasks and keep costs within what the business can afford.

    Conversely, in open domains, when rewards are ambiguous, or when knowledge coverage is insufficient, a general model remains the more reasonable default. The selection criteria should be validation-set accuracy, stability, and unit cost—not parameter count or whether some trendy training framework was used.