Self-Improvement for Next-Generation Models: From Post-Training to a 100T-Scale Data Loop
This article examines the shift in large-model development from scaling a single base model to building model systems that can improve continuously. It reviews the traditional pretraining, post-training, and deployment pipeline and its limitations, then explains recursive self-improvement (RSI) and why “100% RSI” should be treated as an R&D goal rather than a simple metric. It also discusses why coding agents are a practical first testbed, the additional challenges of finance and other specialized task environments, and the data-quality controls needed when agents generate their own training data. Finally, it outlines three layers of self-improvement, the problem of scaling correctness, and a layered reference architecture for enterprise adoption, concluding that the real breakthrough lies in the quality of the closed loop.
1. Model Competition Is Shifting from “Scale Expansion” to a “Training Loop”
Recently, some public pages and R&D records have shown related clues about a new generation of general-purpose models. Although the specific model names, parameter scales, and final release dates still need to be confirmed by official information, these clues point to a relatively clear technical trend: the focus of large-model R&D is gradually shifting from “training a larger base model” to “building a model system that can improve continuously.”
Traditional large-model training can usually be abstracted into three stages:
- Pretraining: Learning general knowledge and foundational capabilities from large-scale text, code, image, or multimodal data.
- Post-training: Making the model better aligned with task requirements through supervised fine-tuning, preference optimization, reinforcement learning, and similar methods.
- Application deployment: Using the model for question answering, coding, analysis, retrieval, or agent tasks.
The main problem with this pipeline is that, after model training is complete, the large amount of interaction experience generated during application is often not recovered efficiently. Errors, invalid operations, tool-call failures, and planning defects encountered by the model while executing tasks usually have to be manually reorganized and converted into new training data.
More advanced technical routes attempt to form the following loop:
Model executes a task → System records the trajectory → Problems are automatically identified → Improvement samples are generated → The model is retrained or the runtime framework is adjusted → The model executes the task again
This type of mechanism is often called recursive self-improvement, or RSI. The “self” here does not mean that the model has independent consciousness. Rather, it means that the model can participate in generating training data, diagnosing execution errors, optimizing task flows, and even helping modify the system that hosts its own operation.
2. “100% RSI” Is More of an R&D Goal Than a Simple Metric
“100% RSI” cannot be simply understood as every model output automatically becoming better, nor does it mean that human involvement is completely unnecessary. A more accurate interpretation is: trying to make every part of the agent R&D process feedback-enabled, evaluable, and optimizable.
A complete self-improvement system involves at least four types of objects:
1. Model capabilities themselves
The model needs to be able to complete tasks, understand the environment, call tools, write code, and explain results. If the foundational capabilities are insufficient, the training data generated later may contain many errors, which can instead amplify system defects.
2. Task execution framework
An agent is not a language model running alone. It is composed of a planner, tool invoker, context manager, memory module, retry logic, and result validator. When improving an agent, it is not enough to adjust model parameters; the system must also check whether the runtime framework causes unnecessary loops, repeated calls, or context bloat.
3. Data production mechanism
Trajectories generated by agents while executing tasks can be converted into supervised data, preference data, error samples, and evaluation samples. However, this data must be filtered and validated; otherwise, “automatically generated data” can easily become self-replication of low-quality data.
4. Evaluation and feedback system
Without reliable evaluation, it is impossible to determine whether the model has actually improved. Faster execution does not necessarily mean higher quality, and longer answers do not necessarily mean more accurate reasoning. Therefore, the system must track accuracy, task completion rate, number of tool calls, cost, latency, stability, and safety at the same time.
It follows that the core of RSI is not to let the model loop indefinitely, but to ensure that each round of the loop has a clear basis for improvement.
3. Why Coding Agents Are Suitable as a First-Stage Testbed
From an engineering perspective, coding tasks are often more suitable than open-ended question answering for self-improvement research, for three main reasons.
First, results are easier to verify
Whether code compiles, passes test cases, and passes static checks usually has relatively clear criteria. Compared with subjective tasks such as “whether an article is insightful,” software engineering tasks make it easier to build automated evaluation systems.
Second, execution trajectories are more complete
A coding task often includes requirements analysis, file retrieval, code modification, test execution, error localization, and further fixes. These steps naturally form a relatively complete task trajectory, making it easier to analyze where the model failed.
Third, feedback can directly enter the next round of training
For example, after an agent modifies code for the first time and a test fails, the system can record the error log, modification content, and failure reason. If the second modification succeeds, the process can be organized into an “error—analysis—fix” training sample.
A general coding-agent loop can be represented as:
Read task↓Analyze code repository↓Create modification plan↓Call editing and execution tools↓Run tests and static checks↓Collect error information↓Generate fix plan↓Verify again↓Store successful and failed trajectories
In this process, training data should not retain only the final correct answer. It should also retain valuable intermediate processes, such as:
- Which files were incorrectly identified as target files;
- Which tool call caused an invalid modification;
- Which error messages are most helpful for locating the problem;
- Why a fix seemed reasonable but failed the tests;
- Whether the model introduced regressions after multiple rounds of modification.
This information can help the model learn “how to avoid failure,” rather than merely memorizing the final code.
4. New Challenges from Finance and Specialized Task Environments
The public clues also mention specialized task environments such as quantitative analysis and financial programming. The value of these scenarios is that they often have clear constraints, structured data, and computable results, making them suitable for reinforcement learning and agent evaluation.
However, specialized task environments also impose higher requirements.
First, results cannot be judged only by surface format. Even if a program runs, it may still have problems such as data leakage, backtesting bias, or incorrect handling of boundary conditions. The system needs multiple layers of validation for data sources, time order, computation flow, and output results.
Second, specialized tasks often emphasize risk control. An agent must distinguish between “theoretically feasible” and “actually executable.” It cannot directly draw a reliable conclusion just because a strategy performed well on historical data.
Third, the data distribution of specialized tasks may change. Good performance in a training environment does not mean the model will remain effective when facing new market conditions, anomalous data, or different constraints. Therefore, task environments need to introduce perturbations, hidden test sets, and cross-scenario validation.
For high-risk domains such as healthcare, finance, and law, a more prudent technical route should be:
- First conduct simulation training in a closed environment;
- Use rules, programs, and expert review for multiple checks;
- Position model output as auxiliary advice rather than automated decisions;
- Set human confirmation and permission boundaries for critical operations;
- Continuously record model behavior for auditing and traceability.
5. The Key to Agents Generating Their Own Training Data Is Data Quality
Having models generate training data is an important way to expand data scale, but it is also the stage most prone to risk. Without external validation, the model may repeatedly reproduce existing errors, forming “error self-distillation.”
A reliable data generation pipeline should include at least the following steps:
1. Task sampling
Select samples from real tasks, synthetic tasks, and adversarial tasks to avoid training data covering only scenarios familiar to the model.
2. Multi-path solving
Have multiple independent execution processes complete the same task and compare differences in their approaches. For coding tasks, different paths can be required to generate implementations, which are then filtered through tests and static analysis.
3. Automated validation
Use compilers, test frameworks, rule engines, numerical validators, or structured scorers to make programmatic judgments about results.
4. Quality tiering
Divide data into high-confidence success samples, repairable failure samples, ambiguous samples, and clearly erroneous samples. Data of different tiers should not participate in training with the same weight.
5. Deduplication and contamination detection
Large-scale automatically generated data is prone to duplication, templating, and training-set contamination. Cleaning requires methods such as text fingerprints, semantic similarity, and execution-trajectory comparison.
6. Human spot checks
Even if automated validation passes, samples should be regularly extracted for human review, especially data involving safety, privacy, and high-risk decisions.
Therefore, the difficulty of very large-scale data targets is not only “how much data is collected,” but also “how much data is truly effective.” If data quality control is insufficient, the larger the data scale, the faster noise may accumulate.
6. From Optimizing the Runtime Framework to Optimizing the Model Itself
Agent self-improvement can be divided into three layers.
Layer 1: Runtime efficiency optimization
The system analyzes tool-call counts, repeated reasoning, context length, and task duration, and automatically reduces ineffective steps. This layer usually does not require directly modifying model parameters, so engineering risk is relatively low.
Layer 2: Agent framework optimization
Based on historical trajectories, the system adjusts task decomposition, retry strategies, memory structure, and error-recovery logic. For example, when a certain type of task often fails during execution, it can add pre-checks or change the order of tool calls.
Layer 3: Model training optimization
Validated execution trajectories are converted into supervised data, preference data, or reinforcement learning signals to improve the model’s planning, reasoning, coding, and tool-use capabilities.
These three layers should proceed step by step. If the model is allowed to directly modify the training process from the beginning, errors may be amplified, and evaluation criteria may even be “gamed.” A more prudent approach is to first let the system optimize observable, rollback-capable runtime parameters, and then gradually expand the scope of automation.
7. Scaling Correctness: Can Small Experiments Guide Large Training Runs?
As training scale continues to grow, R&D teams face a key question: do conclusions observed in small-scale experiments still hold under larger models, more data, and longer training cycles?
This is the problem that scaling correctness aims to solve.
For example, a new data mix may improve accuracy on a small model, but after training scale is expanded, returns may diminish. A reinforcement learning strategy may be effective on short tasks, but on long-horizon tasks it may cause the model to over-optimize for local rewards. A context management method may perform stably with a small number of tool calls, but when tasks become more complex, it may cause context contamination.
To improve predictive capability, a multi-scale experiment matrix can be established:
- Different parameter scales;
- Different training data volumes;
- Different numbers of training steps;
- Different task difficulties;
- Different inference budgets;
- Different tool-call depths.
Then observe the change curves of key metrics, including task success rate, error rate, cost per task, average number of execution steps, and long-horizon stability. Only when an improvement shows a consistent trend across multiple scales is it more likely to apply to large-scale training.
At the same time, “metric gaming” should be guarded against. A model may improve a surface metric by increasing output length, repeatedly calling tools, or avoiding difficult samples. Therefore, the evaluation system must include capability metrics, efficiency metrics, and constraint metrics at the same time.
8. A Reference Architecture for Enterprise Adoption
If an enterprise wants to build a similar agent self-improvement system, it can adopt a layered architecture:
Task layer: business requirements, coding tasks, analysis tasks, process tasks↓Execution layer: planning, tool calls, memory, retries, permission control↓Observability layer: logs, trajectories, errors, cost, latency, results↓Evaluation layer: automated tests, rule validation, human spot checks, adversarial evaluation↓Data layer: success samples, failure samples, preference samples, hard-case samples↓Training layer: supervised fine-tuning, preference optimization, reinforcement learning, prompt and framework adjustment↓Release layer: canary validation, rollback mechanisms, version comparison, continuous monitoring
For infrastructure, object storage can be used to store trajectories and data, message queues can carry asynchronous tasks, container platforms can isolate execution environments, and logging services and monitoring/alerting systems can track model behavior. These are generic technology categories and do not depend on a specific platform.
During implementation, the following capabilities should also be prioritized:
- Traceability: Every model output can be linked to its input, tool calls, data version, and evaluation result.
- Rollback capability: Any automated optimization must be able to restore a stable version.
- Permission isolation: Code and operations generated by the model cannot directly access production systems.
- Cost accounting: Track inference counts, execution time, and resource consumption by task.
- Security auditing: Record sensitive data, external calls, and high-risk behavior.
- Human takeover: Hand tasks over to humans promptly when the model cannot make a judgment or fails repeatedly.
9. Conclusion: The Real Breakthrough Is Loop Quality
The evolution of next-generation large models is no longer only a competition over parameter count, training data, or single-evaluation scores. It is also a combined competition over training systems, execution environments, data engineering, and evaluation methods.
A high-proportion self-improvement goal must be built on three premises:
- The model can generate valuable execution trajectories in real tasks;
- The system can accurately distinguish success, failure, and accidental success;
- Newly generated data and strategies can undergo independent validation.
Without a reliable evaluation mechanism, so-called self-improvement may be merely self-repetition. Without safety boundaries, automated loops may expand the impact of errors. Without scaling validation, conclusions from small experiments may not support large-scale training.
Therefore, what is more worth watching in the future is not whether a model claims to have “fully self-improvement,” but whether it can form a stable, explainable, rollback-capable, and measurable technical loop. Whoever can truly connect task execution, data generation, error repair, and large-scale evaluation is more likely to build a sustained advantage in the next stage of competition among agents and general-purpose models.
