Recursive Self-Improvement (RSI) for Large Models: Self-Evolution and the Correctness Loop

    Recursive Self-Improvement (RSI) is shifting the focus of large-model competition from scaling up the base pretraining toward building an engineering system that lets models improve themselves continuously, reliably, and cheaply. This article defines RSI, breaks down its four technical paths — scalable post-training environments, a self-improvement loop that starts from coding agents, models feeding back into inference infrastructure, and scaled correctness — and explains the methodology that runs through all of them: how to prove that an improvement is real, and how to make small-scale findings hold up at scale. It also covers the engineering architecture, an implementation checklist, and the risks and governance guardrails needed to keep the loop under human control.

    1. Introduction: When Models Begin to Improve Themselves

    Recent reports about a leading large-model team’s next-generation flagship model surfaced several development threads: a trillion-parameter scale, a data target on the order of ten trillion tokens, and — more notably — training records for a batch of internal models. Those models were post-trained for specific scenarios such as quantitative finance and code sandboxes, and their experience was then consolidated back into the next-generation base model. At the same time, the team listed “100% RSI” as a research goal and mapped out a path that starts with a coding-agent loop and later expands to medical agents.

    These threads are less a single-product tracker than a technology barometer. The focus of large-model competition is shifting from “making the base pretraining bigger” toward “building an engineering system that lets models improve themselves continuously, reliably, and at low cost” — that is, Recursive Self-Improvement (RSI).

    This article first defines RSI, then breaks down its four technical paths, and finally discusses the methodology that runs through all of them: how to prove that an improvement is real, and how to make small-scale validation continue to hold at large scale.

    1.1 An Intuitive Definition of RSI

    The classic RSI idea is that an intelligent system can understand and modify its own structure and algorithms, so that the improved system has a stronger ability to improve itself again. In today’s large-model systems, this concept needs a more pragmatic definition:

    RSI refers to a model and its companion agent system taking on parts of the work in data collection, training, evaluation, deployment, and even infrastructure optimization — under goals and constraints set by humans — so that a portion of each iteration is completed by the previous generation of the system itself.

    This is a matter of degree, not an on/off switch. Breaking the R&D pipeline apart:

    Stage Traditional approach Form with model/agent participation
    Data production Manual annotation, hand-written seed samples The model generates candidates, then rules, models, or humans jointly validate them
    Training Engineers design recipes and tune hyperparameters The model helps analyze curves, search hyperparameters, and diagnose training anomalies
    Evaluation Fixed test sets plus manual spot checks Automated evaluation pipelines plus “scaled correctness” extrapolation
    Infrastructure A dedicated team optimizes by hand Agents take part in design, debugging, and tuning; model workloads feed back into the system
    Runtime framework Engineers maintain agent scaffolds and toolchains Agents analyze execution records, then automatically propose and validate changes

    The “100% RSI” goal can be read along this table: let as many stages as possible be completed by the previous generation of the system, while concentrating human effort on setting goals, defining boundaries and constraints, and making key decisions.

    1.2 Four Technical Paths

    Path Core question Main approach Maturity
    Path 1: Scalable post-training environments Can experience be deposited into capability? Task environments, reinforcement learning, verifiable rewards Relatively mature engineering
    Path 2: A loop starting from coding agents Can agents use experience to improve themselves? Experience mining, framework self-rewriting, evaluation-driven iteration Actively being explored
    Path 3: Models feeding back into inference infrastructure Can the system hosting the model evolve along with the model? Agents taking part in design, debugging, and performance tuning Early adoption
    Path 4: Scaled correctness How to judge whether an improvement is real and reliable? Multi-scale experiments, curve extrapolation, conservative decisions Methodology level

    2. Path 1: Building Post-Training Environments at Scale

    2.1 From Static Data to Interactive Environments

    Traditional post-training is dominated by supervised fine-tuning, where the model learns to imitate answers from “input–output” sample pairs. In policy-optimization reinforcement learning (such as Proximal Policy Optimization, PPO), the model instead interacts with an environment over multiple rounds and then adjusts its policy based on the rewards it receives. Training quality then depends on three things:

    • Whether the environment is realistic: Can it reproduce the constraints and feedback of the target task?
    • Whether the reward is verifiable: Can the outcome be judged automatically by a program?
    • Whether difficulty is adjustable: Can the task distribution evolve as capability improves?

    The practical starting point for this path is often the model’s edge on coding tasks. Code can be compiled, run, and verified by unit tests, so it naturally provides a cheap, objective, automatable reward signal. Coding environments therefore usually become the breakthrough point for scaling post-training.

    2.2 Key Points for Building Domain Task Environments

    The quantitative-finance training environment and the coding sandbox mentioned in the reports above represent two typical kinds of task environment.

    A coding sandbox can be seen as a reinforcement-learning version of a “code agent + code evaluation” system, with attention to:

    1. Isolation: Compilation and execution happen in restricted containers, with limits on network, filesystem, and CPU and memory resources, so that untrusted code cannot affect the host.
    2. Multi-turn feedback: Allow the model to go through the full loop of “run fails → read the error → fix → rerun,” so that error-recovery ability enters the training signal.
    3. Verifiable rewards: Rely mainly on programmatically judgeable metrics such as test-case pass rate and code-review rules, reducing the noise from subjective judgment.

    Finance-type environments focus on decision-making and risk, with attention to:

    1. Temporal causality: Strictly separate the information available at the decision point from future data, to prevent leakage of future information.
    2. Costs and constraints: Fold transaction fees, slippage, position limits, and drawdown constraints into the environment; otherwise the model easily learns strategies detached from reality.
    3. Multi-dimensional rewards: Returns, risk, and stability need to be combined with weights; a single return metric easily induces aggressive behavior.

    The common thread across both kinds of environment is this: the reward must correspond to a reliable “verifier.” This is also the starting point for scaled correctness later in this article.

    2.3 Implementation Skeleton of the RL Training Loop

    The pseudocode below shows a minimal viable post-training loop, with the emphasis on decoupling reward computation from parameter updates:

    1. # Verifiable-reward post-training loop (illustrative pseudocode)
    2. def train_step(policy, env_factory, verifier, kl_coef=0.05):
    3. batch = []
    4. for _ in range(NUM_ENVS):
    5. env = env_factory() # Create a fresh isolated sandbox each time
    6. task = env.task
    7. trajectory = []
    8. obs = env.observe()
    9. # Multi-turn interaction: the model produces actions, the environment
    10. # returns observations and immediate feedback
    11. for step in range(MAX_STEPS):
    12. action = policy.act(obs)
    13. obs = env.step(action) # Run code / submit a decision
    14. trajectory.append((obs, action))
    15. if env.done():
    16. break
    17. # Verifiable reward: an independent verifier scores the final result
    18. reward = verifier(task, trajectory, env.final_state())
    19. batch.append((task, trajectory, reward))
    20. # Group normalization + KL regularization to keep the policy
    21. # from drifting too far from the reference model
    22. advantages = normalize_group(batch)
    23. loss = policy_gradient_loss(advantages) + kl_coef * kl(policy, reference)
    24. policy.update(loss)

    Three engineering points are worth noting:

    • Separate the verifier from the policy: The verifier should ideally be an independent, deterministic program, so the model does not act as both player and referee.
    • Parallel sampling: The concurrency of environment instances often directly determines the number of effective gradient samples per unit time, and the environment side is often more likely to become the bottleneck than the model side.
    • KL constraint: Taking small steps near the reference model both preserves general capability and lowers the risk of catastrophic forgetting.

    2.4 Multiple Experience Lines Converging on the Base Model

    A single task environment can quickly make a model stronger at one class of tasks, but it can also skew capability. Consolidating post-training results from finance, code, reasoning, and other environments into a unified base model is essentially a problem of multi-task distillation and capability alignment. Three approaches are common in practice:

    1. Capability-mix experiments: Run small-scale controlled comparisons across different mixing ratios of source data to find a range that balances general and specialized capability.
    2. Curriculum ordering: Train reasoning and general capability first, then layer on domain specialties, reducing mutual interference.
    3. Regression evaluation: After each consolidation, run a regression on a fixed evaluation set to prevent “improve one thing, degrade several others.”

    3. Path 2: A Self-Improvement Loop Starting from Coding Agents

    3.1 Why Start with Coding Agents

    Coding agents have three scarce properties that make them the most natural testbed for RSI:

    • Verifiable: Code runs and can be tested, so success or failure is close to objective.
    • High leverage: Code is both a deliverable and the “raw material” for improving the model itself.
    • Observable: The execution process can be fully recorded as trajectories, making post-hoc analysis easy.

    The idea mentioned in the reports — “start with the coding loop, then expand to areas such as medical” — leverages exactly these three points. For high-constraint scenarios such as healthcare, letting the model improve itself directly carries very high risk; a more realistic approach is to first borrow the mature experience-mining methods of coding agents, then layer on domain-expert review.

    3.2 From Experience to Improvement: A Four-Step Loop

    The self-improvement loop can be broken into four steps:

    1. Record: Fully preserve the task inputs, tool calls, intermediate artifacts, failure reasons, and resource consumption.
    2. Analyze: Separate failure types (misunderstanding, tool misuse, insufficient context) from waste types (repeated retrieval, redundant reasoning, ineffective retries).
    3. Rewrite: Turn the analysis conclusions into changes to the runtime framework — for example, adjusting the toolset, adding validation steps, compressing context, or optimizing the retry strategy.
    4. Validate: Run a regression on a fixed task set; only changes where “metrics do not drop and cost stays controlled” are kept.
    1. # Self-modification flow for an agent harness (illustrative pseudocode)
    2. def self_improve(harness, suite, rounds=5):
    3. best = evaluate(harness, suite)
    4. for _ in range(rounds):
    5. traces = run_with_trace(harness, suite.sample())
    6. issues = analyze(traces) # Failure classification + cost analysis
    7. patch = propose_patch(harness, issues) # The model generates candidate changes
    8. if not policy_allows(patch): # Permission and compliance check
    9. continue
    10. candidate = apply(patch)
    11. scores = evaluate(candidate, suite)
    12. if accept(best, scores): # Accept only if metrics don't drop and cost stays controlled
    13. best, harness = scores, candidate
    14. else:
    15. rollback() # Roll back if it fails
    16. return harness

    3.3 Key Design: Keeping “Self-Modification” Controllable

    The difficulty of this path lies not in “changing” but in “changing in a controlled way.” At least four constraints need to be built in:

    • Changes must be rollback-able: Bring framework configuration under version control, so every change maps to a traceable version.
    • Evaluation comes first: Without a stable regression task set, no self-modification can be judged as effective or not.
    • Cost is part of the objective function: Improvement cannot come at the price of unboundedly rising inference cost. Existing research has shown that after agents automatically optimize the runtime framework, task performance improves while per-task execution cost goes down.
    • Clear capability boundaries: Actions involving high-risk operations (such as modifying production configuration or deleting data) must be explicitly prohibited.

    4. Path 3: Letting Models Feed Back into Inference Infrastructure

    4.1 Models and Systems Shape Each Other

    The stronger the model’s capability, the greater the resource consumption of a single inference; and the more efficient the inference system, the more training and evaluation iterations the same compute can support. So “the model, in turn, improving the system that hosts its own runtime” is a high-leverage path. Typical work includes:

    • Scheduling and batching: Continuous batching, request prioritization, and timeout control affect throughput and tail latency.
    • Memory management: Reuse of the key-value cache (KV Cache), paged management, and offloading strategies determine effective concurrency in long-context scenarios.
    • Decoding acceleration: Speculative decoding, parallel sampling, and cache-hit optimization directly affect time-to-first-token and per-token cost.
    • Precision and parallelism: Combinations of quantization, tensor parallelism, and pipeline parallelism determine effective output per unit of compute.

    4.2 Forms of Agent Participation in Infrastructure Work

    Having agents take part in the work above usually takes three forms:

    1. Diagnostic: Read metrics and logs, locate performance bottlenecks, and produce optimization hypotheses.
    2. Experimental: Batch-try configuration combinations in an isolated environment and screen candidates with benchmarks.
    3. Remedial: Modify configuration or code within a controlled scope, and manage risk through canary rollouts and rollback mechanisms.

    It must be emphasized that this kind of work needs clear metrics (such as throughput, latency percentiles, and per-token cost) and human review gates; otherwise it easily degenerates into ineffective iteration that “looks optimized but is actually unstable.”

    5. Path 4: Scaled Correctness — The Foundation of RSI’s Trustworthiness

    5.1 The Problem: Why Small-Scale Conclusions Fail

    The reliability of the self-improvement loop depends on whether evaluation conclusions are trustworthy. A common trap is this: the optimal configuration found in a small-scale experiment no longer holds after scaling up, and may even reverse. The reasons include:

    • Nonlinear emergence: Some capabilities appear only after parameter count or data volume crosses a threshold, and cannot be observed on small models.
    • Overfitting to tuning: Repeatedly tuning on a small evaluation set is effectively fitting noise.
    • Distribution drift: The training data and the real invocation distribution are inconsistent, and the difference is not obvious at small scale.
    • Changing cost structure: The scheme that is optimal at small scale may be dominated by communication or scheduling overhead at cluster scale.

    5.2 The Method: Multi-Scale Experiments and Extrapolation

    The core claim of “scaled correctness” is: do not equate “effective on a small model” with “effective on a large model.” Instead, run controlled experiments at at least two or three scales, observe the trend of the metric as scale changes, and make an interval estimate of large-scale performance rather than a point estimate.

    An illustration:

    1. # Scaled-correctness check (illustrative pseudocode)
    2. def scaled_correctness(change, sizes=(0.5, 1.0, 2.0)):
    3. results = []
    4. for size in sizes:
    5. base = run_baseline(size, config=CURRENT)
    6. new = run_variant(size, config=change)
    7. results.append({
    8. "size": size,
    9. "base_score": base.score,
    10. "new_score": new.score,
    11. "lift": new.score - base.score,
    12. "cost": new.cost_per_token,
    13. "ci": confidence_interval(new, trials=3) # Repeat across multiple random seeds
    14. })
    15. trend = fit_trend(results) # Fit the trend of the metric as scale changes
    16. return {
    17. "stable": trend.slope_ci_lower > 0, # Lower bound of the confidence interval of the lift trend
    18. "extrapolated": trend.predict(next_scale()), # Interval prediction for the next scale
    19. "raw": results
    20. }

    A few companion practice principles:

    • Repetition and intervals: Repeat each configuration with multiple random seeds, and report confidence intervals rather than a single value.
    • Stratified evaluation: Report statistics stratified by task difficulty, language, and domain, so an overall average does not hide local regressions.
    • Contamination detection: Check for overlap between the evaluation set and the training set, and introduce perturbation-based evaluation sets when needed.
    • Conservative decisions: When extrapolation results carry large uncertainty, postpone a full rollout and first validate with a small-traffic canary.

    5.3 Relationship with RSI

    Scaled correctness is the “foundation of trustworthiness” for RSI for a direct reason: every step of self-improvement needs a judgment criterion, and if that criterion only holds at laboratory scale, the whole loop is amplifying noise. A pragmatic engineering principle:

    Any improvement that cannot pass the scaled-correctness check does not enter the data or configuration of the next round of self-training.

    6. Engineering Implementation: From Experiment to Production

    6.1 Architectural Layers

    A deployable self-improvement system can be roughly divided into five layers:

    Layer Responsibility Common general-purpose cloud capabilities
    Training layer Pretraining, post-training, distillation Container platforms, elastic compute scheduling, distributed training frameworks
    Environment layer Sandboxes, task environments, evaluation executors Isolated containers, image registries, object storage (datasets and checkpoints)
    Data layer Data collection, cleaning, deduplication, mixing Object storage, message queues, data pipelines
    Evaluation layer Regression evaluation, scaled-correctness checks Batch jobs, monitoring and alerting, logging services
    Governance layer Permissions, auditing, cost control Identity and access management, audit logs, resource quotas

    6.2 Implementation Checklist

    1. Are the goals quantifiable: Does every self-improvement stage have a clear success metric?
    2. Are the rewards verifiable: Do they rely on signals a program can judge, rather than subjective evaluation?
    3. Is evaluation independent: Are the verifier and the policy, and the evaluation set and the training set, kept separate?
    4. Are changes rollback-able: Can framework, configuration, and model versions all be traced to a specific change?
    5. Is there a cost ceiling: Are there budget constraints on per-iteration and per-task execution?
    6. Is scale validated: Do the conclusions hold at at least two scales, with intervals given?
    7. Is risk controlled: Are high-risk actions explicitly blocked, and are there audit records?

    7. Risks and Governance

    The risks brought by RSI concentrate in three categories:

    • Reward hacking: The model learns to exploit flaws in the verifier rather than truly completing the task, showing up as rising evaluation scores but declining real-world effectiveness. The response is to introduce human spot checks and adversarial evaluation sets.
    • Distribution collapse: Repeatedly training on data generated by the model itself may reduce output diversity. The response is to keep a certain proportion of real data and monitor diversity metrics of the generated distribution.
    • Recursively amplified errors: Once the evaluation standard itself is biased, self-improvement keeps reinforcing that bias. The response is version control of evaluation standards plus periodic human review.

    On governance, three bottom lines are recommended: human confirmation for high-risk operations, two-person review for key changes, and a traceable, auditable record for every self-improvement action.

    8. Conclusion

    Abstracting away from the recent discussion about the new generation of large models, one clear judgment emerges: large-model self-improvement is moving from “concept” to “engineering.” Its engineering requires four pieces in place at the same time:

    • Scalable post-training environments: deposit experience into trainable signals.
    • A loop starting from coding agents: let agents learn to improve themselves on verifiable tasks.
    • Models feeding back into inference infrastructure: let the efficiency of the hosting system rise in step with model capability.
    • Scaled correctness: provide a trustworthy judgment criterion for all improvements.

    For individual developers, a good starting point is “verifiable coding tasks plus automated regression evaluation” to build a minimal loop. For enterprises, the focus is on depositing the four capabilities of environment, data, evaluation, and governance into reusable platform capabilities. On Baidu AI Cloud, such a platform is typically assembled from general-purpose cloud products such as container platforms, object storage, logging services, and monitoring and alerting. It is worth emphasizing that the boundary of RSI is always set by humans: goals, constraints, and final decision authority still belong to people.