How Far Can Post-Training Push an Open-Weight Model? A Controlled Experiment on Reasoning Cost vs. Capability

The Question: Can “Not Changing the Base” Still Reach the Top Tier?

A recent open-weight model achieved a top-tier ranking in a third-party intelligence index, tying with leading open-weight models in the same category. What matters here is not the score itself, but the path taken to reach it: the base model remained unchanged, the network architecture was untouched, and both total and active parameter counts stayed identical. The only variable was a longer investment in post-training and reinforcement learning (RL)—on the order of one month.

This is essentially a clean controlled experiment. It attempts to answer a long-debated industry question: given a fixed base model, how much additional capability can be extracted purely through post-training and RL? If the gains are significant, then scaling pre-training is not the only route to better models. Engineering teams could achieve meaningful capability jumps by iterating on the post-training pipeline alone, without rebuilding the foundation model.

The results suggest the experiment was effective: the model matched competitors at the same tier and landed on a Pareto frontier—a point where no other configuration dominates it in both capability and cost. Notably, this occurred in the highly competitive mid-to-high capability range. The model is neither “cheap but weak” nor “strong but expensive”; it occupies a competitive middle ground.

Deconstructing Scaling Laws: Parameters Are Not the Only Variable

A more nuanced interpretation of scaling laws is worth exploring. The core argument can be summarized as follows: model capability is not a function of parameter count alone. It is a joint function of training data scale, training duration, inference cost, and architecture type (dense vs. MoE).

Breaking this down:

1. Training data: Parameters and data must be balanced. Early industry practices sometimes paired oversized parameters with undersized data, leaving model capacity underutilized and yielding poor returns per unit of compute. The common correction has been to scale data volume alongside parameter count.
2. Training duration: With the same data and parameters, the number of training steps and the curriculum schedule directly affect convergence quality. Learning rate scheduling, data mixing ratios, and rejection sampling strategies in the post-training phase can significantly alter final capability.
3. Inference cost: This is the most overlooked factor. Training is a one-time expense; inference is a recurring one. A configuration that looks efficient during training can become a poor deal overall if it leads to high online inference costs. Optimizing only for training cost while ignoring long-term inference cost is a classic local optimum trap.
4. Dense vs. MoE: Sparse expert architectures use fewer active parameters while retaining a larger total capacity, offering a trade-off between inference cost and model capacity. Total parameters set the capacity ceiling; active parameters determine per-forward computation. Both need to match the target deployment scenario.

Within this framework, the significance of the “same-architecture post-training experiment” becomes clear: it fixed architecture-level variables (total parameters, active parameters, layer count) and varied only the post-training and RL effort, isolating the marginal contribution of post-training to model capability. This is a highly engineering-driven approach—locking down uncontrollable variables and varying only one at a time to observe the curve.

The Boundary of Post-Training Gains: Why “Thinking Longer” Is Not “Thinking Better”

A common misconception is that post-training and RL simply make a model better at answering questions. In reality, RL modifies the model’s reasoning behavior policy—how many intermediate steps it generates before answering, how it self-verifies, and when it decides to stop thinking.

This introduces a key trade-off: chain-of-thought length versus inference cost.

For smaller models, using longer reasoning chains to compensate for limited capacity is a strategy of “trading time for depth.” The mechanism is not mysterious: Transformer layers are fixed, and a single forward pass can only compute so much. But each generated reasoning step writes intermediate hidden states back into the context, allowing subsequent forward passes to build on them. In effect, the model produces multiple drafts, using sequential generation steps to approximate deeper computation.

This strategy works especially well on verifiable tasks, such as code generation. The model can iteratively reason, self-check, backtrack after detecting errors, and rewrite. Longer reasoning chains can yield measurable accuracy gains. This is why some mid-sized models can match much larger ones on benchmarks—by simply “thinking longer.”

But “thinking longer” has clear costs and limits:

• Cost implications: Output token count directly drives inference bills. Longer reasoning chains mean higher latency and higher per-request cost, especially under high concurrency.
• Diminishing returns: Not all tasks benefit from long reasoning chains. For fact recall, format conversion, or simple classification, extended deliberation can introduce self-doubt and drift, reducing stability.
• Marginal failure: When model capacity is insufficient to cover the required knowledge and reasoning patterns, longer chains simply loop over incorrect assumptions without converging to a correct answer.

Notably, in this same-architecture post-training experiment, the deployed model did not exhibit abnormally long reasoning chains. This suggests that the capability gains from post-training may stem more from improved reasoning strategies and instruction-following stability than from simply longer thinking times. If this interpretation holds, the engineering implication is direct: capability improvements need not come at the cost of linearly rising inference costs, which is especially important for cost-sensitive online services.

Model Selection via the Pareto Frontier

The Pareto frontier is a practical tool for model selection. Plot candidate models on a two-dimensional plane with capability score on one axis and unit inference cost on the other. A point is on the frontier if no other model has both higher capability and lower cost. The goal is not to find the “strongest” model, but to find the one that lies on the frontier within your specific constraints.

A practical workflow looks like this:

# Pseudocode: Pareto-based model filtering
candidates = load_benchmark_results()          # score, input/output price, latency
constraints = {"max_unit_cost": 0.8, "p95_latency_ms": 1200}
def dominates(a, b):
    # a dominates b: capability >=, cost <=, at least one strictly better
    return (a.score >= b.score and a.cost <= b.cost) and \
           (a.score > b.score or a.cost < b.cost)
frontier = [m for m in candidates
            if not any(dominates(o, m) for o in candidates if o is not m)]
feasible = [m for m in frontier if meets(m, constraints)]
best = max(feasible, key=lambda m: m.score / m.cost)  # or weighted by business priorities

Three points are critical here:

1. Capability must be measured on task distributions that match your business. Public benchmark indices are aggregate scores and only useful for initial screening. Real selection requires A/B testing on your own task samples.
2. Cost must be calculated per completed business task, not per token. A model with a low per-token price may require far more tokens to complete a task, inflating the actual per-task cost.
3. Latency must be treated as a hard constraint. In many interactive scenarios, p95 latency matters more than the average. A model that is fast on average but slow in the tail can severely degrade user experience.

Engineering Recommendations: Treat Post-Training as a Continuous Pipeline

If post-training yields meaningful marginal gains, development teams should treat it as a continuously running pipeline rather than a one-off project. Practical steps include:

• Build a regression evaluation set: Sample from real business logs to construct an evaluation set covering typical tasks and long-tail edge cases. Run the same evaluation suite after every post-training iteration to ensure capability gains do not come at the expense of certain scenarios. Without stable evaluation, iterative improvement is not credible.
• Run controlled ablations: Change only a few variables per iteration (e.g., data mixing, RL reward design, curriculum length) and record both capability and cost changes. This avoids the “changed everything, cannot attribute the result” problem.
• Monitor the distribution of chain-of-thought length: Track output token quantiles as a monitoring metric. If median length rises significantly without proportional capability gains, the improvement likely comes from “thinking longer” rather than “thinking better,” and the reward design should be revisited.
• Deploy by tier: Route simple tasks through shorter reasoning paths and complex tasks through longer ones. This reduces average cost more effectively than using the longest reasoning chain uniformly.
• Track active parameter ratio in MoE models: The ratio of total to active parameters directly affects per-forward cost. When selecting models, estimate total cost of ownership based on expected QPS, not just parameter count.

Summary

This experiment—same base model, same architecture, same parameter count, with only post-training and RL extended—offers more than a single benchmark score. It provides a reproducible methodological example: post-training still has meaningful marginal returns even when the base model is fixed. But the source of those returns matters. Whether they come from better reasoning strategies or simply longer thinking determines whether the gains are sustainable and economically viable.

For developers, three practical conclusions stand out. First, model capability is a multi-variable function; when selecting models, evaluate capability, inference cost, and latency together, and filter using Pareto methods. Second, long reasoning chains are a “time-for-depth” tool that works well on verifiable tasks, but need to be actively constrained in cost-sensitive and simple-task scenarios. Third, treating post-training as an engineering pipeline, driven by regression evaluations, yields more stable capability growth than chasing a single benchmark ranking.