From Right-or-Wrong Judging to Real Execution: The Infrastructure Behind LLM Post-Training

1. The Competitive Focus Is Shifting: From Bigger Models to More Capable Models

Over the past two years, the industry has measured large language model teams by three things: parameter scale, pretraining data volume, and public benchmark scores. That framework made sense when pretraining was the primary driver of progress. Leading open-source models kept crossing into new orders of magnitude, and only a handful of teams worldwide could afford pretraining at that scale.

Recent developments tell a different story. Without major changes to the base model, significant gains on complex tasks can come from doubling down on post-training—supervised fine-tuning combined with large-scale reinforcement learning. Public releases from multiple teams point to the same conclusion: the main competitive front is shifting from pretraining to post-training. Teams that go deeper on post-training can convert the same base-model potential into stronger real-world execution.

2. Two Technical Paths and How They Complement Each Other

From a training pipeline perspective, LLM capability comes from two distinct stages.

2.1 Pretraining: Defining the Foundation

Pretraining focuses on:

• Scaling parameter count and context length;
• Optimizing model architecture, such as more efficient sparse designs;
• Improving training data scale, cleaning quality, and mix ratios.

Pretraining shapes a model’s knowledge breadth and language ability, but it is expensive, slow, and—as high-quality data becomes harder to find—showing clear diminishing returns.

2.2 Post-Training: Defining the Ceiling

Post-training mainly consists of two approaches:

• Supervised Fine-Tuning (SFT): Using high-quality demonstration data to align instruction following, output format, and basic behavior;
• Reinforcement Learning (RL): Over the past one to two years, large-scale RL has become the key path for improving reasoning and complex task performance. The typical form is reinforcement learning with verifiable rewards—training first on tasks where answers can be objectively judged, then gradually expanding to multi-turn, multi-step, tool-using agentic tasks.

A notable industry trend is that programming is becoming the foundational skill for this kind of training. The reason is straightforward: code is executable, testable, and verifiable, making it an ideal training ground for RL. The plan-execute-observe-correct loop that models learn through coding can transfer to data analysis, scripted operations, workflow automation, and other professional tasks that can be executed and verified repeatedly. This is where most post-training investment is heading.

3. A Paradigm Shift: From Judging Answers to Real Execution

Training a model to solve math problems with RL is fundamentally different from training it to complete complex engineering tasks.

3.1 Math Problems: Rewards Are Instant and Verifiable

The model gives a final answer, and a checker compares it against the expected result. The interaction is single-turn, the environment is nearly stateless, and verification is cheap.

3.2 Complex Tasks: Rewards Hide in Multi-Step Execution

Take “fixing a failing test” as an example. The model must read code, locate the problem, modify files, run tests, observe errors, and revise again. Every step must happen in a real execution environment with a file system, dependencies, an interpreter or compiler, and a test framework. Feedback is no longer a single right-or-wrong bit; it includes compiler logs, test output, and runtime exceptions. A single task often requires dozens of execution rounds. If any step runs in an inconsistent environment, the entire training trajectory becomes invalid.

This means three fundamental changes to the training paradigm:

• From single-turn Q&A to multi-turn interaction inside an environment;
• From comparing answers to evaluating an entire execution trajectory;
• From treating the environment as irrelevant to treating it as part of the training infrastructure itself.

4. Three Infrastructure Layers for Agentic RL

Large-scale agentic RL may look like an algorithmic problem, but it is fundamentally an infrastructure problem. It requires at least three layers working together.

4.1 Large-Scale Sandboxed Execution Environments

• Isolation: Each task gets its own containerized sandbox with an independent file system, network policy, and resource quota, so tasks do not interfere with each other;
• Reproducibility: Environments must support snapshotting, resetting, and reproduction, with fixed dependency versions and fixed random seeds. If the same task produces different results across two runs, the reward signal becomes untrustworthy;
• Security: Sandboxes must prevent malicious code from escaping while also enforcing CPU, memory, disk, and network limits so a single runaway task cannot bring down the cluster.

4.2 High-Concurrency Scheduling

When thousands of tasks run concurrently and each task involves dozens of interaction steps, the number of sandbox instances needed during rollout is enormous. The scheduling system must address three things:

• Low overhead: Sandbox creation and teardown must be far cheaper than the interactions themselves. In practice, this relies on image preheating and warm-pool reuse;
• Asynchronous decoupling: Environment interaction is mostly CPU- and IO-bound, while parameter updates are GPU-bound. These operate at completely different rhythms and must be orchestrated asynchronously. Otherwise, expensive training GPUs sit idle waiting for environment execution;
• Fault tolerance and rollback: Sandbox crashes and network interruptions are normal. The scheduler must automatically retry and discard corrupted trajectories to keep the training data clean.

Here is a pseudocode example showing the lifecycle of a single trajectory:

# Pseudocode: one rollout trajectory in agentic reinforcement learning
for task in task_batch:                        # thousands of concurrent tasks
    sandbox = scheduler.acquire(image=task.env_image,
                                quota=task.resource_limits)
    state = sandbox.reset(task.init_snapshot)  # reset environment to initial state
    steps = []
    for t in range(task.max_turns):
        action = policy(state)                 # model decision: edit code / run command
        obs = sandbox.execute(action)          # real execution, returns logs and test results
        steps.append((state, action, obs))
        state = obs
        if verifier.is_terminal(obs, task):    # all tests pass / turn limit reached / environment crash
            break
    reward = verifier.score(steps, task)       # score based on real execution results
    scheduler.release(sandbox)                 # return to warm pool for the next task
    trainer.push(steps, reward)                # trajectory enters RL update

4.3 Quality Systems for Tasks and Feedback

• Task side: The task set must be diverse, difficulty-tiered, and close to real-world distribution. It also requires decontamination—any task appearing in pretraining data must be removed or rewritten. Otherwise the model is memorizing answers rather than solving problems;
• Feedback side: Evaluation must be stable, deterministic, and resistant to gaming. Flaky test cases need to be fixed. The same trajectory must produce the same score across multiple runs. Verification rules need to block reward hacking, such as a model learning to comment out failing tests instead of actually fixing the underlying issue.

5. Why Infrastructure Determines the Ceiling

The competitiveness of post-training can be roughly expressed as:

Effective capability gain ≈ number of high-quality tasks × trial-and-error rounds per task × reliability of the feedback signal

If any of these three factors approaches zero, the overall investment turns into wasted compute:

• Few and homogeneous tasks teach the model patterns, not capabilities;
• Unstable environments and noisy scoring corrupt reward signals and can even trigger reward hacking. Models are extremely sensitive to reward loopholes in RL, and an unreliable verifier can derail the entire training run;
• Inefficient scheduling and high sandbox overhead reduce the number of valid trajectories per unit of compute, which directly slows iteration speed.

Post-training competition is therefore fundamentally about building high-quality trial-and-error environments. Whoever can provide models with more high-quality, repeatable, reliably rewarded environments will move further along the capability curve. As algorithms converge across teams, what actually separates them is the unglamorous infrastructure: sandboxes, schedulers, task pipelines, and evaluation systems.

6. Three Recommendations for Developers and Enterprise Users

6.1 When Choosing Models, Look at the “Post-Training Content”

Beyond general conversational ability, pay close attention to complex-task performance: multi-step planning, tool use, long-horizon error correction, and failure recovery. These capabilities depend heavily on post-training investment and directly determine whether a model can handle real enterprise scenarios—not just chat.

6.2 For Private Post-Training, Start by Turning Your Workflows into Environments

Enterprises do not need to build all infrastructure from scratch. A pragmatic path looks like this:

• Use a container platform to package internal tools and business processes into executable task environments;
• Convert historical tickets, code review records, and runbooks into task sets with explicit verification checkpoints;
• Use object storage to persist trajectory data, and use logging and monitoring to keep environments observable and debuggable;
• Design feedback from simple to complex: start with deterministic rules, such as compilation passing, tests passing, or API responses matching a contract, then gradually introduce model-assisted review.

General-purpose cloud capabilities—containers, object storage, logging, and monitoring—can serve as the foundation for building such training environments while avoiding much of the operational overhead of a fully self-hosted stack.

6.3 Make Feedback Reliable Before Scaling Up

In practice, feedback reliability matters more than task volume. It is better to have fewer tasks with trustworthy rewards than many tasks with noisy ones. Noisy rewards get amplified during reinforcement learning. After investing significant compute, you may end up with a model that is good at pleasing the evaluator rather than actually solving problems.

7. Conclusion

The LLM story is shifting from “bigger” to “better at doing things.” Pretraining defines the knowledge foundation; post-training determines how much of that foundation can be converted into real execution capability. And execution capability comes from repeated trial and error in thousands of high-quality environments with reliable feedback.

Once you see this, you see the main thread of LLM competition over the next one to two years—and where enterprises should actually invest. Rather than fixating on leaderboard scores, evaluate the density of executable, verifiable tasks in your own domain. That is the key variable that determines how much of a model’s potential you can actually realize.