Reassessing Scaling Laws: How Many Parameters Can Post-Training Replace?

    Scaling laws are often interpreted as a simple race to larger models, but real-world performance also depends on data quality, post-training, inference cost, model architecture, and verifiable execution environments. This article examines how reinforcement learning and extended reasoning can improve smaller or unchanged base models, why the gains have clear cost trade-offs, and why high-quality executable environments may become the next major bottleneck for coding and security-focused AI systems. It also provides a practical framework for evaluating models based on both quality and lifecycle cost.

    1. What Scaling Laws Actually Say—and How They Are Commonly Misread

    The core idea behind scaling laws is straightforward: under a given compute budget, model size, training data volume measured in tokens, and total computation follow predictable power-law relationships. Model capability generally improves as these variables scale together. Later work on compute-optimal training further suggested that, for a fixed compute budget, parameter count and token count should grow in roughly balanced proportions rather than maximizing parameters while underfeeding the model with data.

    The industry initially made mistakes on both fronts. Early practice prioritized larger models because the marginal gains from adding parameters appeared steeper. At the same time, training data was constrained by the limited supply of high-quality text, resulting in too few tokens.

    A subtler bias came from looking at cost too narrowly. Teams optimized the cost of a single training run without including the cost of every future inference request in the model design objective. Once a conversational product goes into production, inference spending grows with request volume and often becomes one of the largest long-term items on the cloud bill.

    One asymmetry is worth remembering:

    Training is a one-time expense; inference is a recurring expense.

    If a model saves compute during training but requires three times as much computation per output token and produces much longer responses, the initial “saving” often disappears within months in a high-concurrency environment. Evaluation scores therefore cannot be used alone for model selection. They must be considered alongside amortized lifecycle cost.

    2. Four Variables That Shape Intelligence—and What the Pareto Frontier Means

    Model capability is influenced by at least four variables. Their relative importance changes depending on the task.

    Variable How it affects capability or cost Common misconception
    Training data Defines the knowledge boundary and the density of learnable signals for coding and reasoning Looking only at total volume instead of the proportion of high-quality, verifiable examples
    Training duration, especially post-training and RL Determines how effectively pretrained capabilities are activated and optimized Assuming that post-training only handles alignment and formatting
    Inference cost Compute per output token × average output length × request volume Looking only at the listed price per million tokens while ignoring the amplification caused by reasoning tokens
    Model architecture Determines whether total parameters and active parameters are coupled, as in dense models, or decoupled, as in MoE models Treating total parameter count as a direct measure of compute requirements

    Mixture-of-Experts architectures are important because they decouple knowledge capacity, represented by total parameters, from per-token computation, represented by active parameters. An MoE model can have a very large total parameter count while activating only a subset of experts for each token. This allows it to support greater knowledge capacity at a similar inference compute budget.

    The trade-offs include higher memory requirements, uneven expert utilization, and engineering complexity related to routing stability.

    When model capability scores and per-request costs are plotted together, the meaningful candidates are those on the Pareto frontier: no other model is better in both dimensions. A model that remains on the frontier at a relatively low cost can be more valuable for most enterprise workloads than a model that leads on a maximum score but costs several times more.

    In recent years, the most intense competition has concentrated in the middle and upper-middle capability range—the boundary between models that are capable enough to perform useful work and models that are too expensive to deploy at scale.

    3. A Controlled Experiment: How Far Can a Fixed Base Model Go?

    A recent iteration of an open-weight model family demonstrated a particularly clean experimental design. It was close to a textbook controlled-variable experiment.

    The base model remained unchanged. The architecture, total parameter count, and active parameter count also remained unchanged. The only variable was the intensity of post-training and reinforcement learning, applied over roughly a month.

    The result was that the new version tied with models in the same category that had substantially more parameters on an intelligence index published by an independent third-party evaluator. It entered the top tier of open-weight models.

    The result is informative because it removes parameter growth as a confounding factor. The industry has traditionally attributed most capability gains to larger-scale pretraining, while treating post-training as a stage for alignment and output formatting.

    This experiment suggests that, at the 700B total-parameter scale, pretraining has already embedded a substantial amount of usable capability into the weights. Post-training and RL do more than polish the model’s style. They reshape its behavior distribution: capabilities that the model could previously demonstrate only occasionally become behaviors that it invokes more reliably and preferentially.

    The role of reinforcement learning at this stage should not be exaggerated. RL does not create new knowledge from nothing; that knowledge still comes from pretraining and supervised fine-tuning data. Instead, reward signals increase the probability of high-value behaviors and reduce ineffective exploration.

    As a result, RL gains on a fixed base model typically rise sharply at first and then flatten into a plateau. The experiment suggests that, at least at the 700B scale, the plateau had not yet been reached.

    4. Chain-of-Thought Length: Trading Time for Effective Compute

    To understand whether a model has “enough parameters,” it is first necessary to understand how smaller models compensate for limited parameter capacity.

    A Transformer’s serial computation depth in a single forward pass is bounded by its number of layers and attention structure. A 20B-scale model cannot physically perform the same amount of computation on one token as a much larger model.

    It has another way to compensate, however. Each reasoning step can be written back into the context, after which the model performs another forward pass over the expanded context. This wraps additional autoregressive serial computation around a model with fixed forward-pass depth. In effect, the model produces several more drafts:

    It trades inference latency for effective computation depth.

    An intelligence benchmark once showed a 27B-scale model matching a larger model in the same general category. The apparent secret was that it “thought longer.”

    The costs are equally clear:

    • Output token counts increase substantially. Because most inference pricing is based on output tokens, costs can grow superlinearly.
    • As the context grows, the quadratic complexity of attention becomes more significant, worsening tail latency.
    • Long reasoning chains increase inference-cache pressure. Under concurrency, both memory usage and throughput are affected.
    • Longer chains are not always beneficial. If the model makes an incorrect decision early, later tokens may amplify the mistake rather than correct it.

    Coding tasks are especially suitable for this type of long-chain compensation because they are naturally verifiable: code can compile, pass unit tests, and satisfy static checks. A model can repeatedly inspect and revise its output, exchanging additional computation for more drafting rounds until its accuracy approaches that of a larger model.

    This helps explain why several models with noticeably long reasoning times still achieved strong benchmark results.

    The same experiment also produced an important counter-signal. Although the new version matched the capability of larger models, its reasoning time did not increase. If its performance depended mainly on generating longer drafts, its reasoning-token count should also have risen.

    Because that did not happen, the capability was unlikely to be supported primarily by a longer reasoning chain. Instead, the base model may already have internalized more of the required computation into its forward process.

    This leads to a more significant conclusion:

    At the 700B scale, parameter capacity may already be close to sufficient for coding tasks.

    The remaining gap may come less from how much computation the model can perform per token and more from how many high-quality, verifiable training environments can be produced.

    5. The New Bottleneck: Production Capacity for Executable Environments

    If model training is viewed as a data flywheel, the fuel for post-training is not ordinary text. It is the executable environment.

    A minimal executable environment requires:

    1. Reproducible builds and locked dependencies so that the environment can be recreated from scratch at any time;
    2. Runnable tests and assertions that provide objective success signals;
    3. Sandbox isolation to prevent generated code from gaining unauthorized access, making external connections, or abusing resources;
    4. Clear termination conditions and scoring functions to reduce reward hacking;
    5. Controllable, tiered difficulty so that RL sampling includes enough tasks that are solvable but not trivial.

    The difficulty is that publicly available code is much more common than publicly available environments. Repositories are easy to collect, but the proportion that can build inside a clean container, run a complete test suite, and produce trustworthy results is far smaller.

    The next stage of competition will therefore depend heavily on who can convert real development scenarios into high-quality executable environments at scale, then distill verifiable reasoning trajectories from them.

    This also explains why some teams can close the gap through post-training without changing the base model. Their advantage may not be additional compute. It may be their ability to produce environments.

    6. How to Interpret a Benchmark Tie Scientifically

    Aggregated intelligence indexes are useful for coarse positioning, but they should not be used directly for production model selection. At least four issues must be considered:

    • Number of samples: The gap between pass@1 and pass@k can be substantial. Models with long reasoning chains often benefit more from pass@k, but production systems typically receive only one opportunity to produce an answer.
    • Capability breakdown: Aggregate scores can mix mathematics, reasoning, coding, and instruction following. Always inspect the individual categories.
    • Contamination and overfitting: Frequently used public benchmarks may be exposed to training-data leakage. Dynamic or private evaluations are generally more informative.
    • Cost weighting: Scores should be compared against average output tokens multiplied by the per-token price.

    In practice, the most reliable approach is still to build a small but demanding private evaluation set for the target business. The following minimal evaluation skeleton records both quality and cost:

    1. # Evaluation skeleton: compare models or configurations on the same task set
    2. tasks = load_private_tasks("my-repo-regression") # Private, human-labeled, with executable assertions
    3. def run_eval(model_client, tasks, max_samples=1):
    4. total_score, total_out_tokens = 0.0, 0
    5. for t in tasks:
    6. resp = model_client.chat(
    7. messages=t.prompt,
    8. max_output_tokens=t.budget, # Cap the reasoning budget to control costs
    9. )
    10. # Prefer objective signals: successful compilation, tests, and static checks
    11. score = verifier(t, resp.text) # Returns 0.0 to 1.0
    12. total_score += score
    13. total_out_tokens += resp.usage.output_tokens
    14. n = len(tasks)
    15. return {
    16. "quality": total_score / n, # Capability dimension
    17. "avg_out_tokens": total_out_tokens / n, # Cost dimension; do not omit it
    18. "pareto_point": (total_score / n, total_out_tokens / n),
    19. }
    20. results = {name: run_eval(client(name), tasks) for name in candidates}
    21. # Keep only candidates on the Pareto frontier before business trials

    The most valuable metric in this table is often avg_out_tokens. It makes the hidden cost of “thinking longer” explicit. If two candidates deliver the same quality but one produces twice as many output tokens, their long-term bill can also be roughly twice as high.

    7. Capability Depth: Why Security-Oriented Models Are Becoming a New Battleground

    A recent wave of open-model releases has placed substantial emphasis on cybersecurity capabilities. This is not accidental; it reflects a clear commercial logic.

    First, capability supply is relatively limited in this area. General-purpose conversational AI is already highly competitive, with vendors repeatedly competing on price-performance, user experience, and long-context capabilities. Differentiation is becoming harder.

    Security analysis—including log tracing, vulnerability explanation, attack-chain reconstruction, code auditing, and detection-rule generation—requires dense domain knowledge. High-quality public examples are scarce. A team that develops strong performance in this area can create meaningful differentiation beyond general benchmark scores.

    Second, the security market has a procurement structure that is well suited to vendor-led delivery. Buyers are typically governments and large enterprises. Procurement involves compliance processes, dedicated budgets, and relatively high renewal value. These are use cases where customers require implementation and professional support, rather than a purely volume-driven API business.

    Third, integration friction can be lower. Major model vendors may already have technical partnerships and joint-solution foundations with security vendors and systems integrators. A model can be embedded as an engine inside an existing security product ecosystem instead of requiring a new channel, implementation organization, and service model to be built from scratch.

    Technically, however, several challenges remain when general-purpose models are applied to security:

    • Knowledge freshness: Vulnerabilities and attack techniques evolve quickly. Model weights become outdated, so retrieval augmentation and external intelligence sources are essential. Purely parametric memory is not sufficiently reliable.
    • Asymmetric error costs: A missed detection can be far more damaging than a false positive. This means models should generally support analyst judgment rather than act as fully autonomous enforcement systems.
    • Evidence-chain requirements: Conclusions must be traceable to original logs and configurations. Outputs need citations and precise locations, not only natural-language judgments.
    • Auditability: Enterprise and government environments require operation logs, permission isolation, and data-residency controls. Model services may therefore need private deployments or dedicated instances.

    The corresponding technical stack generally includes log ingestion and retrieval, structured processing of large log volumes, monitoring and alerting, streaming pipelines for real-time detection rules, key and credential management, and complete operational audit trails.

    These are broadly applicable cloud infrastructure capabilities. Any private security deployment will depend on them, and the product maturity of a cloud provider across these categories can serve as an objective dimension for comparison.

    8. A Practical Checklist for Developers

    The analysis above leads to the following decision path:

    1. Do not judge models by parameter count alone. Total parameters represent knowledge capacity, while active parameters are more relevant to compute requirements. For MoE models, total parameters and actual inference cost are not proportional.
    2. Calculate lifecycle cost rather than relying on list price. A basic estimate is:

      monthly cost ≈ request volume × (input tokens × input price + average output tokens × output price)

      Average output tokens can differ by several times for models with long reasoning chains.

    3. Set an explicit reasoning budget. In production, cap the maximum output tokens and define fallback behavior for requests that exceed time limits. This prevents tail latency from damaging the overall SLA.
    4. Use pass@1 rather than pass@k for most deployment decisions. If the task allows multiple attempts and verifier-based selection, pass@k is meaningful. Otherwise, it is primarily a laboratory metric.
    5. Build a private evaluation set with executable assertions. This is the most reliable signal that is not contaminated by public leaderboards and that reflects actual business value.
    6. Choose models near the Pareto frontier rather than chasing the maximum score. Models in the middle-to-high capability range often provide a better balance of cost, latency, and controllability.
    7. Invest in environment assets for post-training. If the team plans to iterate on models continuously, executable-environment production may provide more leverage than additional compute.

    Returning to the original question—how many parameters can post-training replace?—this experiment suggests that post-training can replace at least one generational step in model scale.

    Two broader conclusions matter even more.

    First, at the 700B scale, the parameter bottleneck for coding tasks may already be largely alleviated. The bottleneck is shifting toward the production capacity of high-quality executable environments.

    Second, cost structure deserves to be a first-principles consideration in model selection, often more than parameter count itself. This is especially true when reasoning-chain length becomes the dominant factor in the inference bill.

    A benchmark tie is only a snapshot. Long-term return on investment will be determined by the quality of your business evaluation set, the verifiability of your execution environments, and your ability to measure lifecycle cost.