Breakthroughs in Large Model Reasoning: Formalizing Fermat's Last Theorem and Scientific Agent Practices

1. From Answering Questions to Completing Research Tasks

Traditional language models primarily complete Q&A, summarization, and code completion by generating text. They can explain mathematical concepts, provide proof ideas, or generate plausible code snippets, but this does not equate to actually completing verifiable scientific research.

The core shift in research-oriented agents is that the system no longer just outputs a single result but continuously executes tasks around a specific goal:

1. Understanding Tasks: Analyze the constraints, inputs, outputs, and acceptance criteria of a problem.
2. Decomposing Goals: Break down complex tasks into several independently verifiable sub-problems.
3. Calling Tools: Utilize formal proof environments, compilers, testing frameworks, retrieval systems, or simulators.
4. Checking Results: Modify the approach based on error messages and verification feedback.
5. Continuous Iteration: Iterate until a final result satisfying the constraints is achieved, or explicitly report the reasons why it cannot be completed.

This workflow is closer to a research assistant capable of using tools, rather than a mere text generator. For mathematical proofs, program implementations, and experimental designs, “whether the result can be machine-verified” is far more important than “whether the expression is fluent.”

2. Technical Challenges in Complex Mathematical Formalization

Fermat’s Last Theorem already has a mature mathematical proof, but transforming it into a formalized proof remains a massive engineering project. The goal of formalization is not to restate the proof process, but to encode all definitions, lemmas, derivations, and dependencies into a strict structure that can be step-by-step verified by a proof checker.

This process typically involves the following steps:

1. Establishing Formal Objects

The system must clearly define mathematical objects such as natural numbers, integers, rational numbers, prime numbers, elliptic curves, and modular forms, while handling type conversions between different objects. Phrases like “obviously” or “easily derived” in natural language must be decomposed into explicit rules in a formalized environment.

2. Filling in Intermediate Lemmas

Manual papers often omit basic derivations, but formal systems do not accept omitted steps by default. The model must identify which conclusions already exist, which require new lemmas, and establish correct dependency relationships.

3. Handling Proof Search

Complex theorems usually contain numerous branches. The agent needs to choose among multiple proof directions, such as proof by contradiction, mathematical induction, algebraic transformations, or calling existing theorem libraries. A single generation is rarely stable enough to complete this, so a closed loop of “generate-check-correct” is typically required.

4. Resolving Environment and Version Issues

Formal proofs heavily rely on library versions, syntax rules, and compilation environments. A mathematically correct idea might fail the check due to name changes, type mismatches, or missing dependencies. Therefore, the engineering system must preserve the complete proof environment and build logs to ensure the results are reproducible.

It is worth noting that claims of “completing formalization in tens of hours” should clarify the statistical criteria: whether it started from scratch or reused existing math libraries; whether it was completed independently by a single model or through parallel searches by multiple agents; whether it included human-designed routes; and whether the final result was comprehensively checked by an independent verifier. Only when these conditions are transparent can the results be comparable.

3. Capability Boundaries of Research-Oriented Models

Improvements in frontier models for intelligent coding, mathematical reasoning, scientific literature analysis, computer operation, and security testing usually stem from the combination of multiple capabilities, rather than a single parameter scale metric.

Long Context and Structured Memory

Complex tasks often last for hours or even days. The model needs to retain task goals, attempted paths, failure reasons, tool outputs, and to-do items. Relying solely on the context window is unreliable; engineering solutions also require external memory, summary compression, and state management.

Planning and Reflection

High-quality agents do not blindly repeat generation but can adjust strategies based on failure types. For example, compilation failures require checking syntax and types; mathematical counterexamples might indicate insufficient assumptions; abnormal experimental results require re-examining data and measurement methods.

Multi-Agent Collaboration

A system can assign tasks to different roles: a planner formulates the route, an executor writes code, a verifier checks results, and an auditor looks for vulnerabilities. Multi-agent systems are not inherently superior to single models; communication costs, error propagation, and role conflicts all need to be controlled.

Tool Usage Capability

A truly valuable system must be able to call deterministic tools. Mathematical tasks require proof checkers, software tasks require compilation and testing, research tasks require data processing and simulation environments, and security tasks require authorized testing platforms. The model proposes hypotheses and action plans, while the tools provide objective feedback.

4. How to Scientifically Evaluate “Comprehensive Leadership”

When promoting model capabilities, a single score can easily be misleading. Enterprises or research institutions should establish a multi-dimensional evaluation system.

1. Result Correctness

Beyond looking at the final answer, it is also necessary to check whether it passes compilation, unit tests, proof checking, or independent review. For scientific tasks, data sources, experimental conditions, and statistical methods should also be verified.

2. Task Completion Rate

Complex tasks should be classified as “fully completed,” “partially completed,” “unable to complete,” or “produced erroneous results,” rather than merely tracking whether the model outputted text.

3. Reliability and Stability

Whether running the same task multiple times yields similar results, and whether the model suddenly deviates due to context changes, are important metrics. For enterprise scenarios, stability is usually more important than occasionally achieving the highest score.

4. Cost and Latency

It is necessary to track input/output consumption, tool invocation counts, parallel resources, and manual review time simultaneously. A theoretically powerful system that requires extensive manual error correction may not be suitable for production environments.

5. Security

When involving networks, code execution, bioinformatics, or real-world devices, least privilege, sandbox isolation, approval workflows, and operation audits must be applied. The stronger the model’s capability, the less permissible it is to allow unbounded access to production systems or sensitive data.

5. General Architecture for Enterprises

A deployable scientific agent platform can be divided into five layers:

1. Interaction Layer: Receives natural language tasks and displays execution progress, evidence chains, and final reports.
2. Orchestration Layer: Responsible for task decomposition, role assignment, retry strategies, and concurrency control.
3. Model Layer: Selects models with different reasoning capabilities and response speeds based on task difficulty.
4. Tool Layer: Connects code execution environments, formal verifiers, retrieval systems, data analysis components, and experimental simulators.
5. Governance Layer: Provides access control, approval for sensitive operations, log retention, cost monitoring, and result auditing.

A simplified task loop can be represented as:

Receive Goal
  ↓
Generate Task Plan
  ↓
Execute via Tool Calls
  ↓
Read Verification Feedback
  ↓
Modify Plan or Output Result
  ↓
Independent Review and Archiving

Here, independent review cannot simply be a self-confirmation by the same generation process. A more robust approach is to use different prompts, different verifiers, or have human experts spot-check key conclusions.

6. Beware of the “Intelligence Explosion” Narrative in Deployment

The idea that “models can independently improve their architecture, optimize underlying code, and rapidly enhance recursively” is a highly sensitive judgment that requires strict evidentiary support. Models can generate optimization suggestions and modify experimental code, but this does not mean they can independently complete the full loop from proposing hypotheses, designing experiments, scheduling resources, and verifying results to autonomous deployment.

In real-world systems, models are typically subject to the following constraints:

• Unable to automatically obtain unlimited computing resources;
• Must face compilation, testing, and experiment failures;
• Optimization goals may be inconsistent with actual performance;
• Modifying underlying systems carries compatibility and security risks;
• Critical operations require permission approval and human confirmation.

Therefore, the focus of engineering governance should not be simply debating whether the model is “smart enough,” but clearly defining what resources it can access, what actions it can perform, whether each step is reversible, and how anomalous behavior can be detected and blocked.

7. Practical Recommendations for Developers

For daily development, models should be prioritized to handle verifiable tasks such as code explanation, test generation, API documentation, refactoring suggestions, and fault localization. For complex scientific problems, a phased approach is recommended:

1. First, require the model to provide problem definitions and hypotheses;
2. Then, have it formulate an executable plan;
3. Establish independent verification for each sub-conclusion;
4. Save all intermediate artifacts and tool logs;
5. Finally, have experts review the conclusions and scope of application.

In particular, avoid treating model-generated proofs, experimental conclusions, or security analyses as facts. Models can significantly reduce exploration costs, but they cannot replace formal verification, peer review, and production environment testing.

Conclusion

The true breakthrough of frontier agents lies not in generating a stunning piece of text at once, but in their ability to work continuously toward complex goals over the long term, transforming results into verifiable, reproducible, and auditable outcomes through tools. Mathematical formalization, automated programming, and scientific research are merely representative scenarios of this trend.

Future competition will shift from “whose model chats better” to “who can build a more reliable task loop”: possessing both strong reasoning capabilities and clear permission boundaries, comprehensive verification mechanisms, and sustainable engineering governance. For enterprises, the most reasonable path is not to blindly chase single-dimension leaderboards, but to gradually implement measurable, auditable, and risk-controllable intelligent tasks based on their own business needs.