Post-Training at Scale: The RSI Feedback Loop and the 100T-Token Data Goal
The next phase of large language model development is shifting from one-time fine-tuning to a continuous capability production system. This article examines environment-based training, parallel post-training across multiple model branches, recursive self-improvement (RSI), scalable correctness, and the engineering controls required to turn massive data targets into reliable model improvements.
1. Why Post-Training Is Becoming a Capability Production System
The center of competition in large language models is shifting. Pretraining still determines a model’s knowledge breadth and sets the upper bound for its base capabilities. However, whether a model can complete tasks reliably, plan and recover during long interactions, and enter production at a manageable cost increasingly depends on its post-training system.
Recent public signals surrounding next-generation models—including parallel post-training across multiple models, code sandboxes, quantitative finance environments, 100T-scale data targets, and the research goal of “100% RSI”—all point to the same underlying question:
How can post-training evolve from a one-time fine-tuning operation into a sustainable capability production system?
One important distinction is necessary. “100% RSI” is better understood as a research objective or long-term direction, not as an established engineering fact. RSI generally refers to recursive self-improvement: a system uses the results of its own execution to identify problems, generate improvements, validate them, and enter the next iteration.
The real challenge is not making a model propose an improvement. It is making that improvement verifiable, reproducible, scalable, and resistant to introducing new regressions.
From an engineering perspective, the observable direction can be summarized in three areas:
- Task environments: Training moves beyond static corpora into interactive settings such as code sandboxes, financial simulations, and tool-use environments.
- Parallel post-training and capability aggregation: Multiple internal models or expert branches are post-trained independently, after which their trajectories, failure cases, preference signals, and capabilities are fed back into the next-generation base model.
- Self-improvement loops: The system gradually expands from optimizing runtime infrastructure and modifying agent frameworks to generating parts of its own training data.
Together, these directions determine the ceiling of a post-training system and whether a 100T-scale data target creates real value. Data volume is not the destination. The critical issue is whether the system can produce high-quality, verifiable, and traceable training signals.
2. Environment-Based Training: Moving from Reading Data to Performing Tasks
Traditional supervised fine-tuning is effective at teaching models to answer in a human-like way. However, static question-and-answer data has difficulty covering the real constraints involved in programming, financial analysis, and complex tool use.
The core idea of environment-based training is to turn training examples into executable, interactive processes that generate feedback and measurable outcomes.
2.1 Code Sandboxes: Making Error Recovery a Training Objective
A code sandbox is not simply a code execution tool. It is a controlled task environment that typically needs to provide:
- Isolation: Restrict access to the file system, network, processes, and computational resources so that task execution cannot affect the host environment.
- Reproducibility: Fix dependencies, random seeds, runtime versions, and input data so that the same task can be evaluated repeatedly.
- Multi-level feedback: Compile errors, unit tests, static analysis, performance constraints, and style requirements can all provide reward signals.
- Support for long-horizon tasks: Allow the model to plan over multiple turns, call tools, locate errors, and apply fixes rather than merely generate a single block of code.
Coding tasks have relatively clear verification signals, but they also present significant risks. A model may hard-code test cases, bypass checks, or produce a repair that is difficult to maintain. Therefore, the reward function cannot only measure whether the code passes. It should also include process constraints such as the scope of changes, readability, complexity, regression-test results, and newly introduced security risks.
A general environment interaction process can be represented as follows:
run(task_spec, policy):state = env.reset(task_spec)trajectory = []while not state.done:action = policy.act(state)state, reward, done, info = env.step(action)trajectory.append((state, action, reward, info))return trajectory, env.metrics()
The important point is not the exact interface. The training system must preserve the full state, action, reward, and failure information. Only with complete execution trajectories can the system perform failure attribution, preference modeling, and later data filtering.
2.2 Quantitative Finance Environments: Training Under Strong Constraints
The value of a quantitative finance environment is not merely that it adds another vertical domain. It introduces complex tasks with strict constraints, strong feedback, and significant risk.
Such an environment may simulate data cleaning, feature construction, strategy generation, backtesting, risk constraints, and attribution analysis. Compared with ordinary coding tasks, it places greater emphasis on:
- Time-series consistency: Training, validation, and backtesting data must be strictly separated to prevent future-information leakage.
- Constraint satisfaction: Position limits, drawdown, turnover, costs, and risk exposure must be part of the evaluation system.
- Interpretable attribution: Evaluation should examine not only return metrics but also why a strategy failed and where its risks originated.
- Reproducible experiments: The same strategy should be evaluated consistently across different time windows and cost assumptions.
These environments can train planning, tool use, error recovery, and multi-step reasoning. They also highlight an important limitation: the closer an environment is to a real business setting, the less appropriate it is to use a single metric as the reward. Otherwise, the model may learn to optimize the metric rather than solve the underlying problem.
3. Multi-Model Post-Training and Aggregation into the Next Base Model
The idea of running post-training on multiple internal models and then aggregating the results into a next-generation base model represents one of the most important structural changes in this direction. Post-training becomes a parallel exploration process rather than a single-model, single-track iteration.
A typical architecture might look like this:
base_model├── coding_branch -> code sandbox rollouts├── finance_branch -> quantitative environment rollouts├── agent_branch -> tool-use and long-horizon tasks└── reasoning_branch -> verifiable reasoning taskscollect:trajectories, failures, preferences, reward signals, cost metricsfilter and normalize:deduplicate, verify, classify failure modes, remove contaminated samplesaggregate:train reward models / preference models / data selectorsdistill useful behaviors back to next base modelevaluate:regression suite + scale ladder + safety gates
The key point is that aggregation does not mean simply merging models or feeding every trajectory back into the base model without filtering. Effective aggregation requires at least three layers of governance:
- Data stratification: Successful trajectories, failed trajectories, repair trajectories, preference comparisons, and environment metadata should be managed separately.
- Quality gates: Rules-based verification, unit tests, cross-evaluation, and human sampling should be used to remove low-quality examples.
- Capability regression testing: The next-generation base model must pass both legacy capability tests and new environment evaluations. This helps prevent a specialized improvement from causing general capability degradation.
Parallel post-training also creates a practical challenge: reward scales differ across environments. A coding task may use pass rate, a financial task may use risk-adjusted returns, and an agent task may use completion rate combined with cost. Directly mixing these signals can allow one task category to dominate training.
Reward normalization, task-sampling ratios, and curriculum learning are therefore required. Evaluation should report results by domain rather than relying only on a single aggregate score.
4. The RSI Loop: From System Optimization to Training-Data Generation
The value of RSI is that it gradually shifts improvement design from human-driven processes toward system-discovered improvements. From an engineering perspective, it can be divided into three levels.
4.1 Self-Optimization of Runtime Systems
The first level allows the model to participate in designing, debugging, and optimizing the infrastructure that supports its own operation. Examples include analyzing logs, locating performance bottlenecks, proposing configuration changes, generating monitoring rules, and producing repair scripts.
Here, the object being improved is the runtime system. Verification signals are relatively clear: latency, throughput, cost, error rate, and resource utilization can be compared directly.
4.2 Self-Modification of Agent Frameworks
The second level allows an agent to review execution records, analyze failures and wasted work, and modify its own runtime framework, tool-use policies, or task-decomposition strategies.
Engineering practice has shown that such loops can improve task performance and reduce per-task cost. However, they require an external evaluator. Without one, the model may change something that merely appears inefficient while damaging behavior that was previously stable.
4.3 Self-Generation of Training Data
The third level explores whether coding agents can generate parts of their own training data. This is the step closest to the ideal form of RSI and also the highest-risk step.
Self-generated data can introduce three major problems:
- Distribution drift: The model may generate tasks that resemble what it already understands, causing the training distribution to become increasingly narrow.
- Reward hacking: The model may learn to satisfy the evaluator instead of solving the real problem.
- Error accumulation: If unverified generated data enters the training set, existing biases and errors may be amplified.
For this reason, self-generated data must be paired with strong validation mechanisms, including executable tests, formal constraints, cross-checking by multiple models, human sampling, and contamination detection.
A more conservative approach is to restrict self-generated data to verifiable tasks, establish a small closed loop first, and expand it gradually.
“100% RSI” should also be decomposed into measurable indicators rather than treated as a single percentage. Possible metrics include:
- The proportion of automatically discovered improvements that pass validation.
- The magnitude of performance gains after improvement.
- The regression failure rate.
- The cost per successful improvement.
- The ability to transfer improvements across different scales.
RSI becomes meaningful as an engineering system only when these indicators improve together.
5. Scalable Correctness: Do Small-Scale Results Hold at Larger Scales?
A system that continuously iterates must answer a fundamental question: how does it know that it has genuinely improved?
This is the problem of scalable correctness. The central principle is that teams should not only validate conclusions from small experiments. They should also systematically test whether those conclusions remain valid as model size, data volume, or training steps increase.
A practical approach is to use a scale ladder:
for scale in [small, medium, large]:run controlled experimentrecord metric_curve[scale]estimate trend and uncertaintycheck invariants across scalesif predicted_large_metric within toleranceand invariants holdand failure_modes remain bounded:allow scale-upelse:return to hypothesis and redesign
Several considerations are important:
- Use a fixed experimental protocol: Data-filtering rules, training hyperparameters, evaluation sets, and stopping conditions should remain as consistent as possible. Otherwise, changes in scale become mixed with changes in strategy.
- Compare multi-scale curves: Do not look only at individual results. Examine slopes, turning points, and confidence intervals as metrics change with scale.
- Measure extrapolation quality: Use small-scale results to predict large-scale performance and record prediction error. If the error is too large, the system should not be scaled directly.
- Check invariants: Certain safety, formatting, and tool-use constraints should remain stable across scales.
- Classify failure modes: Hallucinations, degradation, reward hacking, and long-tail errors that emerge at larger scales should be tracked separately.
Scalable correctness is not a one-time evaluation. It is a release gate. Teams must treat success in a small experiment as a hypothesis rather than a conclusion, and treat large-scale validation as a necessary cost rather than an optional step.
6. Engineering Checklist
The following checklist can help turn these ideas into an executable system:
- Environment platform: Standardize task descriptions, isolated execution, resource quotas, state recording, and reproducible configurations.
- Data pipeline: Support trajectory collection, failure attribution, deduplication, filtering, contamination detection, version tracking, and access control.
- Evaluation system: Combine unit tests, regression tests, long-horizon task evaluation, cost evaluation, and safety evaluation.
- Model and data registry: Record base-model versions, post-training branches, training-data sources, evaluation results, and release status.
- Observability: Collect logs, metrics, call traces, and failure samples so that agents can review their behavior and support automated analysis.
- Cost accounting: Include inference costs, environment execution costs, and data-generation costs in reward and release decisions.
- Security isolation: Apply graded authorization to code execution, network access, file-system access, and sensitive data.
- Release gates: Require successful small-scale experiments, scale-ladder validation, regression testing, and red-team evaluation before expanding training or deploying a system.
7. Conclusion
The competition in next-generation post-training will not be determined solely by larger parameter counts or larger datasets. A 100T-scale data target primarily tests an organization’s combined ability to produce high-quality data, build useful environments, establish rigorous verification, and operate a self-improvement loop.
Code sandboxes and financial simulation environments provide interactive training grounds. Multi-model post-training and capability aggregation provide a path for exploration at scale. RSI offers a long-term direction for continuous evolution. Scalable correctness provides a methodology for determining whether an improvement is real.
For engineering teams, the most practical approach is to begin with highly verifiable scenarios. First establish reproducible environments and strict evaluation. Then introduce parallel post-training branches, followed by controlled self-improvement loops. Every stage should retain external validators and rollback mechanisms.
Only in this way can RSI move beyond a concept, and only in this way can a 100T-scale data target be converted into sustainable model capability.
