Post-Training for LLMs: Enhancing Coding Capabilities and Optimizing Costs

1. Why Post-Training Outweighs Parameter Scaling

Once a foundation model crosses the 100-billion parameter mark, the marginal gains from simply scaling up parameters diminish significantly. What engineering teams really care about is whether the model can reliably execute multi-step instructions, operate terminals and browsers in real development environments, maintain consistency throughout long-horizon tasks, and control invocation costs while maintaining high capabilities. The answers to these questions do not come from larger parameter counts, but from a more systematic post-training pipeline—which includes supervised fine-tuning, preference alignment, reinforcement learning, tool-use training, and multi-turn interaction optimization. This article explores common industry technical pathways across five dimensions: capability evolution, training methodologies, toolchain synergy, cost control, and engineering selection, without being tied to any specific product or cloud vendor.

2. Foundation Model Upgrades: From General to Specialized

2.1 Foundation Consolidation and Exposing Shortcomings

A common industry practice is to first build a foundation model with broad semantic and coding capabilities through large-scale pre-training. However, once the model enters real-world development scenarios, its shortcomings are quickly exposed: it cannot continuously maintain the working directory context, lacks retry strategies when commands fail, is insensitive to command differences across operating systems, and easily breaks when encountering permission or dependency issues. These are fundamentally “behavioral policy” issues that cannot be resolved by scaling parameters.

2.2 Three Shifts in Post-Training

Engineering teams typically focus on three directions:

1. Shifting from single-turn generation to multi-turn, stateful environment interaction.
2. Shifting from single-domain fine-tuning to joint reinforcement learning, preventing specialized improvements from causing general capability degradation.
3. Shifting from pursuing the “most powerful” model to pursuing the “highest cost-effectiveness”—minimizing the unit task cost while meeting business thresholds.

3. Joint Reinforcement Learning: Enabling Stable Model Execution in Environments

3.1 Joint Training Framework and Prerequisites

Applying reinforcement learning (RL) solely to coding or terminal operations often leads to overfitting. A more robust approach is to jointly optimize multiple task domains within the same training framework: code generation and refactoring, command-line operation and repair, browser operation and frontend vision, long-horizon task planning, and error analysis and retrying. Common implementations rely on distributed training frameworks based on Proximal Policy Optimization (PPO) or similar policy gradient methods. Model parameter scales typically range from tens of billions to hundreds of billions. Sampling strategies employ domain-specific proportional sampling (e.g., code domain 40%, terminal domain 25%, frontend domain 20%, planning domain 15%), with typical training runs lasting from several thousand to tens of thousands of steps. Different tasks share underlying reasoning capabilities, but their behaviors are constrained by differentiated reward signals.

3.2 Reward Signal Design and Weighting Reference

The reward function is typically decomposed into a multi-objective format:

Total Reward = Task Completion Reward (Weight: 0.4~0.5)
             + Execution Efficiency Reward (Weight: 0.2~0.3)
             + Behavioral Compliance Reward (Weight: 0.1~0.2)
             + Environment Consistency Reward (Weight: 0.1~0.15)

Here, task completion measures the correctness of the final result; execution efficiency encourages fewer steps; behavioral compliance restricts dangerous or invalid operations; and environment consistency ensures behavioral stability in real-world scenarios. Tuning experience indicates that during the early stages, slightly increasing the efficiency weight can accelerate convergence, while in later stages, it should be dialed back to ensure compliance and consistency.

3.3 Training Monitoring and Stability

Training cycles are typically measured in weeks and require continuous monitoring to ensure comprehensive capabilities do not degrade, specialized capabilities are rising, behavioral drift does not occur in outputs, and there is no inter-domain interference. If drift is detected, adjustments must be made to sampling ratios, reward weights, or data temperature. Post-training is therefore a continuous engineering effort rather than a one-time action.

4. Toolchain Synergy: Improving Terminal and Frontend Capabilities

4.1 From Writing Code to Getting Work Done

Real-world development is not just about writing code; it includes reading repositories, understanding dependencies, running tests, operating terminals and browsers, and maintaining multi-turn context. The model needs to understand the input/output boundaries of various tools and invoke them at the appropriate time.

4.2 Terminal Operation Capabilities and Safety Reminders

A model with strong terminal capabilities should be able to read directory states, adjust commands based on error messages, distinguish between development and production environments, and provide alternative solutions when permissions are restricted. Training usually relies on massive amounts of environment interaction data—for example, progressively locating and providing a fix path in an “install dependency failed” scenario. Important Note: Actual deployment must be executed in sandboxed or container-isolated environments. Models must be strictly prohibited from directly executing high-risk commands like rm -rf. Command whitelists and human-in-the-loop confirmation mechanisms should be implemented.

4.3 Frontend Vision: Distinguishing 2D and 3D

Frontend vision tasks can be divided into two categories: 2D page layout and 3D scene understanding. 2D tasks focus on page structure, visual hierarchy, and interaction logic, making them suitable for Vision Transformer (ViT)-based visual encoders. 3D tasks require an understanding of spatial relationships and materials, usually necessitating multimodal encoders or point cloud/voxel inputs. Post-training must introduce a large volume of visual annotation data and optimize it jointly with code generation.

5. Reducing Inference Costs: Post-Training as a Key Lever

5.1 Inference Cost Composition and Engineering Optimization

Inference costs are primarily concentrated in VRAM usage, compute time, batching efficiency, and long-context overhead. Once parameters are fixed, optimization methods include more efficient attention implementations, sensible caching, operator fusion, and dynamic batching scheduling.

5.2 The Impact of Post-Training on Task Costs

Post-training indirectly reduces total costs by improving the first-attempt success rate. In internal testing for a certain type of terminal repair task, a model that underwent joint RL post-training reduced the number of interaction rounds required to complete tasks by approximately 60% compared to the baseline, resulting in a corresponding significant drop in end-to-end task costs. Therefore, evaluation should focus on the “total cost of completing a real task” rather than the price per API call.

5.3 The Engineering Value of High Cost-Effectiveness

For enterprises, high cost-effectiveness translates to more requests, lower failure rates, less human intervention, and more stable system integration. In actual engineering, these values are often more important than leaderboard scores.

6. Evaluation Evolution and a Four-Step Enterprise Selection Methodology

6.1 From Static Leaderboards to Real-World Tasks

Traditional static Q&A evaluations struggle to reflect capabilities in continuous state maintenance, error recovery, and multi-tool switching. The industry is shifting toward real-world task evaluations, such as repairing broken codebases, generating and deploying web pages from scratch, and decomposing ambiguous requirements and executing them.

6.2 Enterprise Model Selection Recommendations

It is recommended to follow these steps:

Step	Action	Key Takeaways
1	Select task sets closest to your business	Cover at least two categories among code, terminal, and frontend
2	Build an internal evaluation environment	Include sandboxes, toolchains, and log collection
3	Compare candidate models under identical conditions	Record success rates, time consumed, and costs
4	Holistically evaluate cost-effectiveness and stability	Factor in human intervention and maintenance costs

7. Practical Insights: An Actionable Checklist

7.1 Define Task Boundaries: Classify automation levels by risk—full automation for low-risk, human-machine collaboration for medium-risk, and human confirmation retained with sandbox isolation for high-risk tasks.

7.2 Build Observable Agent Workflows: Log every step’s inputs and outputs, tool invocation results, errors and retries, and environment state changes. This is used both for troubleshooting and as optimization data for future iterations.

7.3 Replace Single-Performance Mindset with Cost Mindset: Comprehensively calculate inference fees, retry fees, human intervention costs, and integration maintenance costs. Choose the solution with the lowest total cost.

7.4 Continuously Accumulate Internal Evaluation Data: Build and regularly update evaluation sets based on your business to ensure model capabilities evolve in sync with requirements.

8. Conclusion

A successful post-training upgrade for an ultra-large model is the co-evolution of capabilities, toolchains, evaluation, and cost strategies. Not scaling parameters but strengthening post-training can bring cross-generational practical improvements. Joint reinforcement learning helps models work stably in real environments; toolchain synergy allows models to transition from “writing code” to “getting work done”; and post-training simultaneously drives capability enhancement and cost optimization. For developers and technical decision-makers, the core proposition is not leaderboard scores, but whether the model can reliably complete real-world tasks at a controllable cost within business scenarios.