What Top Research Agents Lack Is Not Parameters but a First-Principles Closed Loop

1. The real gap is not just “more parameters”

Whenever a lab releases a new reasoning model or research agent, a familiar claim quickly follows: the difference is mainly due to insufficient parameter count or insufficient compute. However, when we examine actual performance in hard scientific domains such as physics and mathematics, the problem is more specific.

A model can produce the right terminology. It can build a polished-looking framework. It can even generate formally self-consistent derivations. But once the task enters algorithm decomposition, boundary-condition analysis, dimensional-consistency checks, conservation-law constraints, and numerical verification, errors often slip in silently.

In one case, a theoretical physics research group at a university handed a quantum gravity derivation—originally written by hand over nearly one year and totaling nearly 500 pages—to one of the most capable general-purpose reasoning models available at the time. The model reproduced the core computation in about 10 hours, produced more than 180 pages of derivation notes and complete LaTeX source code, and identified several computational errors in the original derivation.

In another case, a theoretical physicist from the same group used a research agent to complete a research paper in quantum field theory. The paper included a new factorization theorem. From initial formulation to a finished manuscript, the process took only two weeks, with the researcher mainly directing the direction through prompts. However, the researcher also clearly noted that the model was still not rigorous enough and that expert evaluation remained irreplaceable.

In a third case, a team used a research agent to orchestrate about 1,000 parallel instances. In roughly 30 hours, it completed a full formalization of a well-known theorem in number theory inside an interactive theorem prover, and also participated in auxiliary computations related to open problems in black hole physics.

Taken together, these examples suggest that the real variable is not parameter scale but “closed-loop capability”:

• It can decompose a vague objective into executable and verifiable subtasks.
• It can extend a reasoning chain sufficiently far while keeping every step traceable.
• It can call external tools—symbolic computation, numerical solvers, theorem provers—to perform self-verification.

In short: parameters determine whether a model can state the correct answer; a closed loop determines whether it can prove that the answer is correct.

2. What is a “first-principles closed loop”?

“First principles” means reasoning from basic assumptions and relying only on generally accepted laws, rather than borrowing ready-made conclusions. In physics research, first-principles elements typically include the principle of least action, symmetry and Noether’s theorem, dimensional analysis, conservation laws, statistical assumptions, and uncertainty relations.

A “closed loop” turns an open-ended derivation into an “execute-check-revise” cycle. If any stage lacks a check-and-revise回路, the process is merely open-loop generation.

A complete closed loop contains four components:

1. Explicit assumptions.
   Each assumption should record its valid range, such as “non-relativistic,” “weak coupling,” or “higher-order terms neglected,” so that boundary-condition checks can verify them item by item.

2. Long-chain structured reasoning.
   Every derivation step should have a clear basis: an axiom, a previously verified lemma, or a computational operation. The chain should be represented as a machine-readable structure, not as free-form natural-language narration.

3. Independent verification layer.
   The checker should be decoupled from the derivation process and should not directly trust the model’s self-report. Verification can include dimensional consistency, conservation laws, symmetry and covariance, limiting cases, comparison with known analytical solutions, and numerical convergence order.

4. Tool fallback and failure rollback.
   Symbolic computation handles analytical transformations, numerical methods provide cross-validation, and theorem provers handle formalization. Once an inconsistency is detected, the system should roll back to the last trusted state and select a new path.

The standard of judgment is not whether the answer “looks right,” but whether:

• every conclusion can be traced back to its assumptions,
• every assumption has passed validation, and
• any failure can be localized to a specific step.

3. Failure mechanisms: why long-chain physics reasoning is especially hard for large models

1. Error accumulation under probabilistic sampling

Language models sample each step from a probability distribution. Even if the per-step correctness rate reaches 99%, the probability of getting 100 consecutive steps right is only about 0.99^100 ≈ 0.37. Physics derivations often involve dozens or even hundreds of steps. Pure generative progression almost inevitably “moves forward with errors.”

The problem is worse because errors can be disguised very well: the form may look correct, the dimensions may coincidentally match, and the magnitude may seem reasonable, yet the result may fail quietly at some approximation order or under some boundary condition.

2. Training-data distribution and scarcity of visible reasoning traces

High-value physics problems have few answers, and complete derivation processes are even scarcer. Online corpora often show conclusions, but rarely show complete error-correction trajectories. As a result, models often learn the “shape of a conclusion” rather than the process of reaching it.

3. Lack of ground-truth signals

Open problems do not have standard answers, so they cannot be trained or filtered using simple right-or-wrong labels. In this situation, the only available signal comes from internal consistency:

• Do different paths reach the same result?
• Do limiting cases reduce to known conclusions?
• Do numerical solutions agree with analytical solutions?

This requires a structure of “self-doubt” in the model, rather than confidently writing straight through to the end.

4. Combinatorial explosion and context limitations

When a research agent orchestrates hundreds or thousands of instances, the real difficulty is not raw compute but state management. The assumption set, symbol table, verified lemmas, and unverified conclusions must be maintained consistently. Otherwise, branches may cite each other’s unverified conclusions, amplifying errors in the form of false “consensus.”

4. Measurable evaluation dimensions: turning “not rigorous enough” into metrics

To close the gap, the gap must first be measurable. An evaluation system can be built around the following dimensions:

Evaluation Dimension	Specific Checks	Pass Criteria
Dimensional consistency	Whether both sides of each equation have matching dimensions; whether the dimensions of the final result match physical expectations	All equations pass
Conservation laws	Whether conserved quantities such as energy, momentum, charge, and probability current are violated during derivation	Residuals remain within a specified tolerance
Symmetry and covariance	Whether the given symmetry group of the problem is satisfied; whether the form remains invariant under coordinate transformations	Results are consistent before and after transformation
Limiting cases	Whether the result reduces to known classical results in limits such as weak field, low velocity, weak coupling, or large N	Order-by-order agreement with analytical limits
Numerical verification	Relative error of the analytical result against high-precision numerical solutions; convergence order of numerical solutions	Error decreases with step size at the theoretical order
Boundary conditions	Whether conditions at infinity, singularities, and initial/final values are omitted or misused	Each condition is listed and checked
Formal completeness	Whether the proof contains no sorry placeholders and whether axiom dependencies are acceptable	The theorem prover passes with zero warnings

Convergence-order checking can be written as a compact independent verification function:

def check_convergence(solve, h_values, expected_order, tol=0.15):
    """
    Check the convergence order of a numerical solution using three different
    step sizes and compare it with the expected order.
    solve(h) returns the numerical result for a given step size.
    """
    e = [abs(solve(h) - exact_reference) for h in h_values]  # Compare with the reference solution
    orders = [
        math.log(e[i] / e[i + 1]) / math.log(h_values[i] / h_values[i + 1])
        for i in range(len(e) - 1)
    ]
    return all(abs(p - expected_order) <= tol for p in orders)

The value of this kind of check is that it is completely independent of the model’s “narrative ability.” No matter how fluent the derivation sounds, if the convergence order does not match, the conclusion does not hold.

5. Engineering implementation: a derive-check-rollback pipeline

The above ideas can be turned into an executable orchestration skeleton, as shown in the following illustrative code:

class Assumption:
    text: str
    valid_range: str   # Valid range, e.g., "non-relativistic: v/c << 1"
def close_loop(problem, max_rounds=12, verifier=None):
    state = Workspace(assumptions=[], lemmas=[], failed=[])
    while not state.solved and state.round < max_rounds:
        todo = planner.select(state)            # 1) Plan: select an executable subtask
        out = executor.run(todo, state)         # 2) Derive: symbolic derivation / numerical solving / proof search
        report = verifier.check(out, state)     # 3) Verify: dimensions, conservation, limits, convergence
        if report.ok:
            state.commit(out)                   # 4) Commit: store assumptions, lemmas, intermediate results
        else:
            state.rollback(out, report.failed)  # 5) Roll back: undo only the last step and record counterexamples
    return state.diagnostics()

There are four key design points:

• Externalized assumptions.
  Assumptions should be written in structured fields rather than buried inside prose. During boundary-condition checks, each assumption can be compared item by item, making it immediately clear which assumption is violated in a given region.

• Decoupled generator and verifier.
  The verifier should preferably be implemented with deterministic algorithms, or at least with different prompting strategies or different models, to avoid the model grading its own exam.

• Minimal rollback.
  On failure, only the last committed state is rolled back, and the triggering cause is recorded in a failure list. This helps the next round of task selection avoid the same type of error.

• Readable state machine.
  The assumption set, symbol table, verified lemmas, and unverified conclusions should be persisted in structured storage. At any moment, the system should be able to answer: “Which trusted foundation are we currently standing on?”

On top of this, generic cloud platforms such as Baidu AI Cloud can provide general engineering building blocks, including container platforms, object storage, message queues, logging services, and monitoring alerts. In such a pipeline, parallel instances can be scheduled by the container platform using elastic compute; intermediate artifacts and derivation snapshots can be archived in object storage; subtasks can be distributed through message queues; and operational metrics such as failure rate, average derivation depth, and rollback count can be collected by logging and monitoring systems. These are general engineering capabilities and do not involve any specific product or proprietary interface.

6. Common failure modes and countermeasures

• Terminology hallucination.
  The model stacks terms such as “renormalization group” or “holographic duality” without performing actual computation.
  Countermeasure: require every concept to be attached to a computable object—an equation, a piece of code, or a numerical result.

• Silent approximation.
  The model drops higher-order terms without explanation.
  Countermeasure: require all approximations to be explicitly declared, together with the conditions under which the approximation breaks down.

• Confident mistakes.
  An incorrect conclusion is accompanied by a complete-looking derivation.
  Countermeasure: assign verification to independent components and prohibit the model from making the final judgment on its own conclusion.

• Circular verification.
  The same model and the same prompt template are used to check the derivation that the model just generated.
  Countermeasure: use cross-validation or deterministic algorithms for independent review.

• Cross-branch over-citation.
  Parallel branches cite each other’s unverified conclusions.
  Countermeasure: distinguish “verified lemmas” from “unverified conjectures,” and prohibit unverified content from entering downstream derivations.

7. Practical recommendations: close the loop in four steps

1. Build checkers before chasing model capability.
   Deterministic components such as dimensional checks, conservation-law checks, and limiting-case regression sets have the highest return on investment and are largely model-independent.

2. Construct domain regression test sets.
   Collect problems with known analytical solutions, old derivations containing known errors, and boundary-condition analysis problems for classic open questions. These can serve as fixed benchmarks for continuous evaluation.

3. Turn “state machine + minimal rollback” into infrastructure.
   Make every derivation traceable, reproducible, and comparable, rather than a one-off question-and-answer session.

4. Use human review as the final safeguard.
   In scientific research, professional judgment is irreplaceable. The role of the agent is to accelerate derivation and expose errors, not to replace conclusions.

Conclusion

The gap in physics AI is not parameter count. It is whether the system can turn “assumptions-derivation-verification-rollback” into a truly closed loop. Whoever engineers this loop first will enable models to move from “talking about physics” to “computing physics and verifying physics.”