From Exploring Phenomena to Proving Structures: How AI Learns to Think Like a Scientist
Recent advancements in AI are pushing the boundaries from simple data fitting to genuine theory building. This article explores how AI agents are transitioning from recording isolated phenomena—akin to astronomical logs—to constructing unified, provable mathematical frameworks. By examining interactive game evaluations, unsolvable level experiments, and Algebraic World Models (AWM), we uncover the limitations of empirical rule induction and the breakthrough of structural modeling. The article also highlights engineering implications for building intelligent systems capable of rigorous proof and verifiable reasoning.
A Thought Experiment: Can AI Rediscover Gravity?
Consider an interesting question: If an AI were trained only on all human knowledge available before Newton proposed the law of universal gravitation, could it independently discover gravity? Taking it a step further: facing a completely unfamiliar world, could an AI rely solely on observation and experimentation to distill a basic theoretical framework, and then use that framework to understand and predict the world?
This is not science fiction, but a real proposition in current frontier research. Its essence examines whether AI possesses the capability to leap from “data fitting” to “theory building.” Historically, Tycho Brahe spent his life amassing vast and precise astronomical observation data but failed to propose planetary motion laws; Kepler induced empirical rules from this data; and Newton used a unified mathematical framework to explain everything from falling apples to orbiting planets. Most AI today remains in the “Tycho phase”—adept at recording and fitting phenomena, but struggling to extract the underlying structures governing those phenomena.
Recent research is driving this transformation. One approach involves “Algebraic World Models” (AWM), attempting to let agents autonomously construct the algebraic structures of the world through interaction. Another approach uses open problems in theoretical computer science as a “test paper” to evaluate the real research capabilities of frontier large models—finding that models have already surpassed the best published human results in certain specific mathematical constructions. Both paths point to the same conclusion: AI is moving from “exploring phenomena” to “proving structures,” and from “answering test questions” to “doing research.”
Evaluating Rule Induction in LLMs via Interactive Games
To understand this shift, consider a straightforward experimental scenario: interactive game evaluation. In these mini-games, players are not told the rules or the objectives—what can be moved, how mechanisms trigger, and how to win must all be explored by the player. Each game is a mini-world waiting to be understood, serving as a natural touchstone for an agent’s “world modeling” capabilities.
Currently, the best-performing agent architectures, combined with adaptation layers, can achieve near-perfect scores in these evaluations. The key mechanism is that the agent organizes game mechanics into symbolic rules during gameplay. These rules have a high information density, covering dimensions such as game state, multi-step plans, control methods, time, and position, yielding more precise and complete descriptions than those produced by other models.
However, the problem is: these rules resemble Tycho’s experimental logbook rather than Newton’s law of gravity. They record experimental phenomena item by item but lack a unified structure describing the essential laws governing the game world. The rules are isolated from one another, lack hierarchy or axiomatic organization, and cannot support strict reasoning.
Such “phenomenon-level understanding” performs acceptably in-distribution, but reveals its fragility once the environment changes slightly.
The Unsolvable Level Experiment: Limits of Empirical Extrapolation
Researchers conducted a clever stress test: adjusting the content of two game levels and setting the initial step budgets to 18 and 35 steps, making the levels mathematically unsolvable. For instance, one level requires at least 19 steps to win, making 18 steps logically impossible.
They then asked the strongest agent to play and explicitly state its conclusion if it found a level unsolvable. The results were dramatic: facing the unsolvable level, the agent tried for 8 rounds, became “somewhat frustrated,” and concluded that 18 steps were insufficient to win. The direction of the conclusion was correct, but it further claimed “winning requires at least 22 steps”—a number that was wrong. If a human player trusted this judgment, they would incorrectly abandon a solvable level when given a budget of 19, 20, or 21 steps.
Moreover, this was not an isolated case. Across 12 runs by two baseline models, only 5 correctly identified the impossibility, and 4 of those provided the wrong minimum step count; the single correct calculation was admittedly unverified by the model for path optimality.
This experiment precisely captures the fundamental flaw of the current paradigm: the model can empirically “feel” that something is impossible, but it cannot “prove” it. Intuition errs, but proof does not. The unsolvability of 18 steps is a proposition provable via state space analysis—such as through invariance arguments or reachability analysis—demonstrating that the target state is unreachable under any 18-step action sequence. However, agents that only induct empirical rules lack both the reasoning tools and the awareness to construct such arguments.
From Rule Lists to Algebraic Structures: The Right Way to Open World Models
How do we cross this chasm? The “Algebraic World Model” research suggests not settling for collecting scattered symbolic rules. Instead, it requires agents to organize observed phenomena into an algebraic whole—operations form sets, composite relationships between operations form group or semigroup structures, and state transitions follow laws that can be strictly deduced.
This corresponds exactly to the historical leap from Tycho to Newton in the history of science:
- Tycho-style AI: Faithfully records “after executing action a at step n, the object moves from position p1 to p2”—a mass of isolated facts.
- Kepler-style AI: Induces “this type of mechanism always triggers when condition c is met”—empirical rules.
- Newton-style AI: Discovers “all operations form a transformation structure acting on the state space, and level solvability is equivalent to whether the target state lies within the orbit of the initial state”—a unified theoretical framework supporting strict proof.
Once equipped with this structural perspective, the aforementioned unsolvable level problem is easily resolved: solvability no longer relies on “intuition” after trial and error, but can be deduced directly from algebraic properties. Similarly, facing unseen level variants, an agent with a world model can generalize based on theory rather than starting from scratch.
Another Touchstone: LLMs Doing Real Math Research
Echoing world model research, another study tested 9 frontier large models using open problems in theoretical computer science. These problems are characterized by having no readily available answers to retrieve and cannot be solved by pattern matching; they require constructing new mathematical objects or providing new proofs.
The results showed that LLMs have surpassed human-published best results in certain specific mathematical constructions. The significance of this fact lies not in “AI is stronger than humans,” but in proving that when a task requires construction and proof rather than retrieval and imitation, properly guided models can indeed produce verifiable new knowledge beyond their training distribution.
This corroborates the world model research: whether facing game worlds or mathematical worlds, the key capability leap is the same—shifting from generating “plausible-sounding answers” to producing “strictly verifiable structures.”
Engineering Implications: Building “Proof-Capable” Intelligent Systems
For developers, these studies offer several actionable insights:
- Distinguish between “rule induction” and “structure modeling.” When designing agents that require environmental understanding, do not stop at having them output natural language rule lists. You should further require that rules be organized, reasoned, and falsifiable.
- Introduce verifiable feedback loops. Math problems are excellent training grounds precisely because answers can be machine-verified. Similarly, engineering systems should design “executable, testable” intermediate representations wherever possible, letting model outputs undergo formal checks rather than manual visual inspection.
- Use counterexamples and stress tests to expose pseudo-abilities. The unsolvable level experiment reminds us that high in-distribution scores can mask reasoning flaws. When evaluating intelligent systems, proactively construct boundary samples and infeasible instances to test whether the system can provide a strict “impossible” judgment and its basis.
- Leverage cloud infrastructure for large-scale experiments. Such research involves massive parallel interactions, long-sequence trajectory storage, and repeated evaluations. Engineering-wise, you can build experiment pipelines using general container platforms, object storage, and logging services to ensure the reproducibility and scalability of experiments.
Conclusion
From rule induction in interactive games to structural breakthroughs in open math problems, a clear main theme is emerging: the evaluation standard for AI is shifting from “how many questions answered correctly” to “whether it can conduct real research.” Making AI think like Newton is not about having it memorize more phenomena, but teaching it to abstract axioms from phenomena and deduce the world from axioms. That day has not fully arrived, but the paradigm shift from “exploration” to “proof” has already begun.
