100% RSI and 100T Tokens in LLM R&D: Recursive Self-Improvement, Data Engineering, and Practical Contracts

    This article examines two unofficial rumors around next-generation large language model development: “100% RSI” and a “100T token” training-data target. It clarifies what recursive self-improvement means in practice, the levels of automation and prerequisites involved, what 100 trillion training tokens imply for data engineering, and how developers and teams can prepare through verifiable workflows, evaluation assets, and clear intent contracts.

    1. Background: Two Keywords in Recent R&D Rumors

    Recent industry discussions around next-generation large language model development have surfaced two recurring keywords:

    • “100% RSI”: the completion level of Recursive Self-Improvement, or RSI, is described as 100%;
    • “100T”: the training-data target is described as reaching a scale of roughly 100 trillion tokens.

    These claims come from unofficial channels, are unverified, and do not point to any specific vendor. This article does not assess whether they are true. Instead, it treats them as two technical propositions and answers three questions:

    1. What does 100% RSI mean technically, and what prerequisites does it require?
    2. Why might a 100T token data target become an R&D direction, and what does it require from data engineering?
    3. What do these trends mean for front-line developers, and how should individuals and teams prepare?

    2. RSI: From Human-in-the-Loop to System-in-the-Loop

    2.1 Clarifying the concept

    RSI refers to models increasingly participating in the process of “producing the next-generation model.” This includes generating, filtering, and balancing training data; searching for training strategies and hyperparameters; building evaluation systems; and optimizing training and inference infrastructure. Its essence is to hand as much of the “develop-evaluate-improve” loop as possible to the model itself, shortening iteration cycles and increasing capability gains per unit of compute.

    “100% RSI” does not mean “no human involvement.” A more accurate description is “system in the loop”: humans define the objective function and acceptance criteria, the system autonomously plans experiments, executes them, evaluates results, and iterates, while humans sample-check and correct course at key nodes.

    RSI is often confused with related concepts. The differences are as follows:

    Concept What is automated Closed-loop?
    Knowledge distillation Transferring knowledge from a large model to a smaller model No; usually a one-time process
    AutoML / hyperparameter search Model structure and hyperparameters No; the optimization space is usually predefined by humans
    Automated data labeling Data labels No; usually a single-stage task
    Synthetic data augmentation Training samples Partially; often depends on external quality control
    Recursive self-improvement The R&D process itself Yes; models participate in defining the next round of improvement

    The key distinction is that the first few approaches are designed by humans, and machines execute within an established framework. RSI allows machines to participate in improving the process itself.

    2.2 Three progressive levels

    Level Human-machine division of labor Typical forms Main risks
    L1 Assistance Humans define the problem and acceptance criteria; the model generates candidates Automatically generated data samples, candidate code, hyperparameter suggestions Misuse; adoption without validation
    L2 Collaboration The model proposes improvement hypotheses; humans define acceptance; automated pipelines evaluate Automated experiment management, automated evaluation, failed-sample feedback Gaming the evaluation metrics
    L3 Autonomy The model sets subgoals, plans experiments, analyzes results, and adjusts training End-to-end self-improvement loop Goal drift, error accumulation

    As systems move from L1 to L3, the human role shifts from “executor” to “objective setter and supervisor.” Even with 100% RSI, objective functions, acceptance criteria, and high-risk decisions should remain under human control. Otherwise, the system is merely optimizing a possibly wrong objective more efficiently.

    2.3 Two prerequisites for RSI

    Prerequisite 1: Feedback must be automatically verifiable.

    The self-improvement loop is “generate-validate-feed back.” The quality of the feedback signal determines the direction of the loop:

    • Objective feedback, such as unit-test pass rates or numerical metrics, tends to be high-quality and low-cost;
    • Subjective feedback, such as human preference labels, depends on labeling consistency and is more expensive and slower;
    • Model self-evaluation is the most problematic, because the evaluator and the evaluated may share the same blind spots.

    Therefore, high-completion RSI is more likely to land first in verifiable domains such as code, mathematics, and logical reasoning.

    Prerequisite 2: Improvement must converge.

    Model outputs have distributions. If models are recursively trained on model-generated data, distributional bias can accumulate across generations. Existing research calls this phenomenon model collapse. Its main symptoms include:

    • shrinking diversity;
    • loss of long-tail knowledge;
    • increasing homogeneity of outputs.

    If every round moves toward the “most likely” output, rare but correct information can be systematically filtered out over multiple rounds. Possible mitigations include:

    • using verifiable rewards as the core signal;
    • retaining a certain proportion of real data as anchors;
    • monitoring output entropy and diversity after each round.

    3. 100T Tokens: Why It May Become a Data Target

    3.1 Understanding the scale of training data

    Data source Typical scale Notes
    High-quality public text corpora Trillions to tens of trillions of tokens Statistical definitions vary widely, and the scale is close to the upper limit of existing supply
    Code and code-related text Several trillion tokens Publicly available portions are growing quickly; license terms must be verified by category
    Raw web crawls Far larger than the above totals After cleaning, deduplication, and quality filtering, the usable proportion is usually very low
    Model-generated synthetic data Theoretically unlimited Constrained by seed-data quality, diversity, and verifiability

    These figures are rough order-of-magnitude references. Different statistical definitions can vary significantly.

    If “100T” refers to effective training tokens, its scale is roughly one order of magnitude higher than the several trillion to low tens of trillions of tokens used by many mainstream models today. Public high-quality corpora alone would struggle to support this scale. Such a target is more likely to depend on a combination of:

    • multilingual and multi-domain corpora;
    • rigorously filtered synthetic data;
    • enterprise private data, used within authorized scope.

    3.2 Using scaling laws as a reference

    Scaling-law research, such as Chinchilla-style studies, generally concludes that under a fixed compute budget, parameter count and training-token count should grow roughly proportionally. This is the idea of a “compute-optimal” ratio. A commonly cited rule of thumb is approximately 20 training tokens per parameter.

    A rough estimate:

    • If the parameter count is 200 billion, the compute-optimal token count is about 4 trillion; 100T is about 25 times that.
    • If the parameter count is 500 billion, the compute-optimal token count is about 10 trillion; 100T is about 10 times that.

    In either case, 100T clearly falls into the “over-training” range. If performance is similar, halving the parameter count usually means:

    • memory usage, inference latency, and per-call cost are roughly halved;
    • the model is easier to quantize, distill, and deploy at the edge.

    The following simplified code demonstrates the difference between “compute-optimal” and “over-training.” This example is illustrative. Real R&D work should conduct systematic scaling experiments based on the specific architecture, data mixture, and compute budget.

    1. def optimal_tokens(params: float, ratio: float = 20.0) -> float:
    2. """Estimate token count using the compute-optimal rule of thumb (about 20 tokens per parameter)."""
    3. return params * ratio
    4. params = 200e9 # 200 billion parameters
    5. d_opt = optimal_tokens(params)
    6. d_target = 100e12 # Assume the R&D target is 100T tokens
    7. print(f"Compute-optimal range is about {d_opt / 1e12:.0f}T tokens")
    8. print(f"The target data volume is {d_target / d_opt:.0f} times the optimal value, indicating over-training")
    9. print(f"If performance is similar, halving parameters under the same compute budget can roughly halve inference cost")

    3.3 Three engineering questions behind 100T

    Proposition 1: Effective tokens matter more than raw data volume.

    High-quality tokens are much more expensive to obtain than ordinary web text. Effective data determines the upper bound of model capability, while ordinary data mainly affects throughput scale.

    Proposition 2: Data mixture is strategy.

    The ratio of different data sources directly affects model performance on code, mathematics, multilingual, and domain-specific tasks. Adjusting the mixture is essentially a series of research questions that require large-scale ablation experiments. This is exactly where automated experiment-management platforms can help.

    Proposition 3: Data lineage and compliance must come first.

    The source of data, license scope, and sensitive-information status should be traceable. In enterprise environments, whether data can be used legally and whether a solution can pass security and compliance review often determine feasibility earlier than model-capability metrics.

    4. Developer Perspective: Performance Surplus and Wishful Programming

    Rumors pointing to stronger self-improvement and larger data scales may lead some people to assume that models are becoming so capable that humans only need to “state requirements.” Front-line engineering experience points to the opposite.

    4.1 Performance surplus: capability beyond the task

    For many development tasks, frontier models exhibit obvious performance surplus:

    • writing a ridge regression model;
    • performing one round of data cleaning;
    • completing format conversion.

    These tasks are limited in difficulty, and most models can handle them. The difference is that large-parameter models tolerate ambiguity in instructions better. Developers can omit a few explanations and still get usable results. Smaller models that have undergone high-quality post-training usually need clearer specifications, but with a little more communication effort they can achieve similar outcomes.

    Capability surplus is not inherently bad. It means higher fault tolerance and lower communication cost. But fault tolerance is not the same as skipping acceptance checks. The surplus should be used to handle complexity, not to cover gaps in requirement definition.

    4.2 The pitfall of wishful programming

    The idea that “say one sentence and the system completes all the work” is better suited to non-technical marketing than to daily engineering. There are three main reasons:

    1. Ambiguity is risk. Natural language is inherently ambiguous, and hidden constraints in requirements cannot be reliably guessed by a model.
    2. It is not reproducible. The same prompt may produce different results under different model versions or contexts, making version management and regression testing difficult.
    3. Accountability is unclear. When outputs are not constrained by verifiable conditions, problems often appear only after deployment.

    A simple self-check is useful: if you replaced the model with a newly hired colleague, could your requirement description allow that colleague to complete the task independently and pass acceptance? If not, the problem is probably with the requirements, not the model.

    4.3 A typical counter-example: ridge regression

    Consider a concrete case: you give the model three years of data and ask it to build a ridge regression model. The result uses only the most recent three months, and the model severely underfits.

    This problem can be analyzed at four levels:

    • Model level: the model misunderstood the reference to “three years” or simplified the time window;
    • Data level: the date range and frequency were not validated; missing values, adjustments, suspensions, holidays, and other details were not handled;
    • Method level: the closed-form solution of ridge regression is sensitive to feature scale. Without standardization, the regularization term may favor features with different units. If the regularization coefficient is too large, weights may be over-compressed;
    • Acceptance level: if the code runs and outputs coefficients, it may be treated as “done.” Underfitting does not raise an error in code; it can only be found through business metrics.

    Data-intensive fields such as finance deserve special attention for two reasons:

    • problem definitions are relatively clear, data density is high, and business constraints are explicit;
    • many steps can be objectively verified, such as numerical computation, reconciliation, and statistical testing.

    These tasks are well suited for human-machine collaborative iteration. But financial data also has specific traps:

    • whether future information leaks into the model;
    • how adjustments and trading suspensions are handled;
    • whether cross-validation is split chronologically;
    • whether results are explainable.

    Any mishandling can turn “plausible-looking” output into wrong conclusions.

    4.4 Turn wishes into contracts

    The solution is not to ask the model to be “smarter,” but to convert vague intent into executable, verifiable contracts:

    1. Explicitly declare the data range: start and end dates, field definitions, and sampling frequency;
    2. Explicitly declare the split method: time-series data must be split chronologically; random shuffling is prohibited;
    3. Explicitly declare preprocessing: standardization should be fitted only on the training set and then applied to the validation set;
    4. Explicitly declare acceptance criteria: for example, the number of training samples, validation-set R², and the search range for the regularization coefficient;
    5. Codify these criteria with assertions and tests.

    The following is a minimal illustrative implementation. Before running it, df, feature_cols, and target_col must be prepared.

    1. import numpy as np
    2. def ridge_fit(X, y, alpha):
    3. """Closed-form ridge regression: w = (X^T X + αI)^-1 X^T y; the intercept is not regularized."""
    4. X1 = np.column_stack([np.ones(len(X)), X])
    5. penalty = alpha * np.eye(X.shape[1] + 1)
    6. penalty[0, 0] = 0.0 # Do not penalize the intercept
    7. return np.linalg.solve(X1.T @ X1 + penalty, X1.T @ y)
    8. def ridge_predict(X, w):
    9. return np.column_stack([np.ones(len(X)), X]) @ w
    10. def r2_score(y_true, y_pred):
    11. ss_res = np.sum((y_true - y_pred) ** 2)
    12. ss_tot = np.sum((y_true - np.mean(y_true)) ** 2)
    13. return 1 - ss_res / ss_tot
    14. # 1) Assert the data range: clarify what “three years” means
    15. assert df["date"].min() >= START_DATE and df["date"].max() <= END_DATE
    16. # 2) Split chronologically; never shuffle randomly
    17. df = df.sort_values("date").reset_index(drop=True)
    18. n_train = int(len(df) * 0.8)
    19. train, valid = df.iloc[:n_train], df.iloc[n_train:]
    20. # 3) Fit standardization parameters only on the training set to avoid information leakage
    21. mu = train[feature_cols].mean()
    22. sigma = train[feature_cols].std(ddof=0).replace(0, 1.0)
    23. X_tr = ((train[feature_cols] - mu) / sigma).to_numpy()
    24. X_va = ((valid[feature_cols] - mu) / sigma).to_numpy()
    25. y_tr, y_va = train[target_col].to_numpy(), valid[target_col].to_numpy()
    26. # 4) Automatically search for the regularization coefficient
    27. best = max(
    28. ((r2_score(y_va, ridge_predict(X_va, ridge_fit(X_tr, y_tr, a))), a)
    29. for a in np.logspace(-3, 3, 13)),
    30. key=lambda x: x[0],
    31. )
    32. # 5) Acceptance assertion: fail immediately if the result is unacceptable, rather than delivering an underfit model
    33. assert len(train) >= 100, "Too few training samples; the data range may have been truncated"
    34. assert best[0] >= 0.5, f"Validation R² is too low ({best[0]:.3f}); suspected underfitting"
    35. print(f"best alpha={best[1]:.4g}, valid R²={best[0]:.3f}")

    This code reflects four engineering principles:

    • every implicit assumption becomes an assertion;
    • model selection is based on objective metrics;
    • acceptance criteria can be executed automatically;
    • unverified outputs cannot enter the next stage.

    The same approach can be applied to other tasks:

    • Data cleaning: add checks for null rates, outlier rates, and primary-key uniqueness;
    • Report development: use fixed samples for regression testing;
    • API development: define structural constraints for inputs and outputs, and add unit tests.

    4.5 Can smaller models replace larger models?

    For tasks with performance surplus, smaller models with targeted post-training, clear input specifications, and automated acceptance can often reach usable quality while significantly reducing per-call cost. Whether replacement is feasible can be tested through a small controlled experiment:

    1. Sample 50 to 200 cases from real tasks to build a benchmark set;
    2. Record the first-pass rate and manual rework rate for both large and small models;
    3. Record call cost, latency, and data-compliance requirements;
    4. Decide on a routing strategy after considering all factors.

    A common layered strategy:

    Task type Suggested approach
    High-frequency, pattern-fixed tasks Smaller models or rule-based solutions
    Tasks requiring cross-document reasoning Large-parameter models
    Tasks involving funds, compliance, or public release Human review required regardless of model size

    5. Preparing for Higher RSI: What Teams Should Do

    Even as RSI completion improves, the following work will not lose value. Instead, it will become a decisive factor in how fast a system can iterate:

    1. Build a library of verifiable tasks. Turn repetitive prompt work into standard processes with tests and assertions. This is the main source of automated feedback signals.
    2. Invest in evaluation assets. Build evaluation sets that closely reflect real business scenarios, run regular regressions, and continuously add failed samples.
    3. Write intent as contracts. Requirement documents should include input and output definitions, constraints, and acceptance criteria, reducing vague expressions.
    4. Define human-machine boundaries. Keep humans in high-risk decisions and final acceptance, and retain complete operational and data audit trails.
    5. Use general cloud capabilities.

      • Object storage can version data assets;
      • Message queues can decouple data pipelines;
      • Container platforms can run training and evaluation tasks;
      • Logging and monitoring/alerting services can track task status and quality metrics.

      On cloud platforms such as Baidu AI Cloud, these general capability categories are often available and can be combined as needed.

    6. Control synthetic-data risk. Keep real data as anchors, continuously monitor output diversity, and prevent quality degradation from accumulating unnoticed.

    6. Conclusion

    The rumors around “100% RSI” and a “100T data target” remain unverified, but the direction they point to is traceable:

    • R&D iteration is shifting from being human-driven to being system-driven;
    • data scale and data quality are becoming key constraints on the upper bound of capability.

    For developers, the more practical preparation is not waiting for an “omnipotent model,” but turning daily work into a verifiable, reproducible, and continuously improvable loop.

    In one sentence: the stronger the model capability, the more valuable the ability to “state requirements clearly and verify results accurately” becomes.