Internal Model Tiering and Recursive Self-Improvement: Engineering Practices for AI R&D Productivity

1. Background: Multiple Model Tiers, Not One “Strongest Model”

A frontier AI research organization has recently been publicly discussed because it reportedly does not maintain only a single external-facing model. Instead, it runs multiple models with different capability levels. Its internal exploratory versions are significantly more capable than the stable versions released externally.

A co-founder of the organization responded publicly that internal toolchains should naturally use more aggressive models than external products, and that this is standard practice.

This claim is technically coherent because internal research and external services have very different tolerance for failure.

• External-facing models are constrained by stability, cost, compliance, explainability, and inference latency. Any capability jump must pass through safety evaluation, regression testing, and staged rollout. These cycles are usually measured in months.
• Internal research models can prioritize capability over stability. They may tolerate occasional failures, high per-token cost, and unfinished interfaces. Their value is to give researchers a longer exploration window during the research phase.

This often produces an effect described as being “several months ahead.” However, it is important to clarify that this time difference is not simply a single-model advantage. It is the result of compounding effects: internal teams get stronger assistance earlier, write more experimental code, complete more ablation runs, and discover wrong directions sooner. Attributing the entire benefit to “one model being a few months stronger” can mislead teams about where to invest.

2. Designing Internal Model Tiers: The Capability–Cost–Risk Triangle

If model tiering is done casually, it quickly degenerates into three separately maintained inference services. A practical tiering design should be built around three constraints at the same time: capability, cost, and risk.

Tier	Typical Use	Capability Orientation	Cost Orientation	Risk Tolerance
Exploration tier	Generate experiment code, construct evaluation sets, brainstorm hypotheses	Capability first	High	High; failures allowed
Production tier	External APIs, batch inference, offline labeling	Balanced	Controlled	Low
Lightweight tier	Log classification, routing, cache-hit decisions, structured extraction	Good enough	Very low	Medium

There are three key design principles:

1. Use a unified abstraction layer.
   All tiers should expose capabilities through the same calling protocol. Route them with logical aliases such as tier-explore, tier-prod, and tier-lite. Business code should not bind directly to specific model identifiers. When an internal model is upgraded, only the routing configuration needs to change, not the callers.

2. Make fallback paths explicit.
   Exploratory models often produce unstable outputs. Callers must be able to recognize that a given call is not trustworthy and fall back to a more conservative path. Uncertain results should not be written directly into experimental conclusions.

3. Make cost attributable.
   Calls to each tier must be distinguishable in billing and monitoring. Otherwise, if internal usage becomes uncontrolled, the first thing to be cut is likely to be research exploration itself.

3. What RSI Actually Accelerates: Breaking Down the R&D Loop

Recursive Self-Improvement, or RSI, is often mystified in public discussions. Stripped of embellishment, it means: allowing AI systems to participate in part of their own R&D process, so that improvement efficiency itself increases as capability increases.

In engineering terms, it only works within a closed loop:

Hypothesis generation → Experiment design and code implementation → Batch execution and sampling → Automatic evaluation and attribution → Conclusions become input for the next round
    ↑                                                                                              │
    └──────────────────────────────────────────────────────────────────────────────────────────────┘

In practice, the middle three stages are usually accelerated:

• Experiment implementation. Boilerplate work from requirements to runnable code is greatly compressed. This is the most certain source of benefit.
• Experiment execution. Automation of orchestration, parameter sweeps, and log aggregation reduces time spent waiting for resources.
• Evaluation and attribution. Automatically generating comparisons, extracting conclusions in batches, and marking anomalous batches help researchers see signals faster.

The hardest stages to accelerate are the first and last ones: deciding which problems are worth working on, and abstracting experimental conclusions into transferable knowledge. These still depend on human domain judgment. This defines the ceiling of any RSI effort: it shortens the time needed to validate an idea, not the time needed to come up with the right idea.

4. “Shortening Experiment Time by 80%”: Why Measurement Definitions Matter More Than the Number

Public discussions have mentioned large reductions in experiment time, but without verifiable details. In engineering evaluation, such numbers must be questioned by definition; otherwise, they cannot guide decisions. At minimum, the following should be distinguished:

• Wall-clock time reduction. Time from submission to result becomes shorter. This may be caused only by higher concurrency or shorter queues, and may have nothing to do with AI being “smarter.”
• Effective experiment throughput improvement. The number of trustworthy experiments completed per unit of time increases. This is the meaningful metric.
• Change in cost per experiment. The total token spend and compute spend. Saving time while multiplying cost is not necessarily a net gain.
• Conclusion effectiveness rate. The proportion of experiments that produce information usable for decisions. If AI generates many low-value experiments, throughput may look good, but conclusion effectiveness may drop.

Therefore, any claim of “N% productivity improvement” should be accompanied by a controlled experiment design: the same batch of tasks, the same dataset, and the same scorer version, with only one variable changed—whether an assistance step is introduced. Report mean values and variance, not a single best-case result.

5. Reproducible Experiment Orchestration: Make Configuration the Single Source of Truth

To make productivity gains measurable, the first step is to ensure that every run has identical and traceable inputs. The core artifact is the experiment manifest:

from dataclasses import dataclass, asdict
import hashlib, json, time, uuid
@dataclass
class RunSpec:
    tier_alias: str        # Logical alias, e.g. "tier-explore"
    dataset_digest: str    # Dataset content hash
    prompt_version: str    # Prompt and template version
    decode_params: dict    # Sampling parameters
    judge_version: str     # Automatic scorer version
    code_digest: str       # Code-under-test hash
    def manifest_id(self) -> str:
        payload = json.dumps(asdict(self), sort_keys=True, ensure_ascii=False)
        return hashlib.sha256(payload.encode("utf-8")).hexdigest()[:16]
def run_experiment(spec: RunSpec):
    run_id = f"{spec.manifest_id()}-{uuid.uuid4().hex[:6]}"
    started = time.time()
    result = execute(spec)                     # Actual execution body, decoupled from orchestration
    record = {
        "run_id": run_id,
        "started_at": started,
        "elapsed": time.time() - started,
        "metrics": score(result, spec.judge_version),
        "spec": asdict(spec),
    }
    emit_structured_log(record)                # Write to the unified logging service
    return record

Combined with tiered routing, the configuration can be written in a declarative form that is easier to review and roll back:

routing:
  default: tier-prod
  rules:
    - match: {task: "code_generation", stage: "explore"}
      use: tier-explore
      on_failure: tier-prod          # Explicit fallback; no silent failure
      max_tokens: 8192
    - match: {task: "log_classify"}
      use: tier-lite
budgets:
  tier-explore: {daily_token_cap: 40_000_000, alert_threshold: 0.8}

Three engineering disciplines matter more than model selection:

• Input hashing. Both datasets and code should have digests.
• Scorer versioning. Changing the scorer must be treated as a new experiment.
• Failure attribution. Timeouts, rate limiting, and parsing failures should be statistically distinguished from insufficient model capability.

6. Required Guardrails: RSI Is Not an Exemption Lane

The most overlooked risk of recursive self-improvement is not an abstract fear of “capability loss,” but several concrete engineering failure modes that have repeatedly appeared in industry:

1. Evaluation contamination.
   If an assistance model helps generate evaluation data and also participates in scoring, metrics will be systematically inflated. Countermeasure: physically separate training and evaluation data, produce evaluation sets through an independent process and freeze them, and ensure the scorer and generator are not the same source.

2. Metric overfitting, or reward hacking.
   When the optimization target can be perceived by the machine, the system optimizes the score rather than the capability. A typical symptom is output that fully conforms to the scorer’s expectations but is hollow in content. Countermeasure: keep a human sampling review queue and periodically relabel a batch of golden samples.

3. Knowledge bases being flooded by low-quality content.
   If automatically accumulated conclusions are not validated, the next round of input becomes noise, and the benefit cancels itself out. Countermeasure: set a human admission threshold before conclusions enter the knowledge base, and annotate confidence levels and source batches.

4. Permission and data boundaries.
   When automated processes access R&D code repositories, follow least-privilege principles: read-only scope, independent credentials, and full audit logging. In particular, avoid writing credentials into automatically generated code or configuration.

A recommended admission model is tiered release: low-risk changes such as documentation, test completion, and log analysis can be automatically merged; changes involving core logic, data pipelines, or security-related configuration require mandatory human review, with retained diffs and reasons.

7. Cloud Capability Mapping

This closed loop does not require a special platform. It can be assembled from generic cloud capabilities:

• Heterogeneous compute and container orchestration platforms. Host experiment tasks, queue them by priority, and run exploratory tasks in lower-priority queues to avoid competing with production inference.
• Object storage. Store dataset snapshots, experiment artifacts, and model checkpoints. Name them with content hashes to achieve immutable versions.
• Message queues and batch processing. Decouple task generation from task execution, smooth out peaks and valleys, and allow failed tasks to be replayed.
• Logging services and monitoring/alerting. Centrally collect run records, token consumption, and failure rates, and support full-link retrieval by run_id.
• Key management and access control. Manage credentials for different model tiers independently, authorize by role, and retain operation traces.

For teams that want to build their own AI R&D pipeline, generic cloud categories such as container platforms, object storage, message queues, logging, and monitoring/alerting can serve as component-selection directions. On Baidu AI Cloud, these generic capabilities may also be considered as part of this architecture. Specific specifications and billing should be based on the official documentation.

8. Four-Week Rollout Plan

• Week 1: Baseline. Freeze an evaluation set. Run the existing process three times and calculate mean and variance. Record wall-clock time, cost per experiment, and conclusion effectiveness. Without a baseline, there is no productivity conclusion.
• Week 2: Abstraction layer. Introduce logical alias routing and fallback strategies. Consolidate scattered model calls into a single client.
• Week 3: Closed loop. Launch the experiment manifest and structured logging. Connect an orchestration queue so the path from hypothesis to result is end-to-end automated.
• Week 4: Guardrails and review. Set human review thresholds, evaluation isolation, and budget alerts. Compare against the Week 1 baseline and output a reproducible report.

Conclusion

The coexistence of multiple internal model tiers and recursive self-improvement are fundamentally R&D infrastructure optimization problems, not mysteries of model capability. Only three transferable lessons matter:

1. Use logical aliases rather than specific model names for tiering.
2. Measure productivity gains with reproducible controlled experiments.
3. Pair every automated step with human review and clear permission boundaries.

If these three points are implemented, teams can capture the most certain part of AI R&D productivity gains even without using any frontier internal model.