Verifiable AI-Assisted Research: Capability Boundaries and a Practical Workflow
AI models can generate fluent research-style reasoning, but fluency does not guarantee logical validity. This article explains how to assess AI-assisted research outputs through deterministic verification, using a graph-girth checker as a concrete example. It also reviews publicly reported evaluation signals, identifies where models can contribute effectively, discusses where human judgment remains essential, and offers practical recommendations for implementing a verifiable workflow.
Why Research Outputs Need to Be Verifiable
Results in mathematics and algorithm research often take the form of conjectures, constructions, or proofs. Their value depends on whether others can review them. A plausible claim that lacks a checkable object is difficult to carry into follow-up research. This issue becomes more prominent when large language models participate in research: models are good at generating fluent derivations, but fluent language does not mean the logic is valid.
Therefore, whether a model output has research value can be judged against three conditions:
- The output is a concrete object, such as a graph structure, a parameter combination, a numerical bound, or a formal proof.
- A deterministic checking program exists that can decide within a finite time whether the object is acceptable.
- The checking result and human conclusions corroborate each other and do not rely on the output producer’s self-description.
Only outputs that satisfy these three conditions can enter peer review and iteration.
A Reviewable Construction Example
Consider the condition “the graph has girth at least 5.” Such a graph cannot contain a cycle of length 3 (a triangle) or a cycle of length 4. Given the number of vertices and the edge set, the following program can perform all checks:
from itertools import combinationsdef check_girth_at_least_5(n, edges):adj = [set() for _ in range(n)]for u, v in edges:if u == v:return False, "Self-loop found"if v in adj[u]:return False, "Duplicate edge found"adj[u].add(v)adj[v].add(u)# Triangle: the two endpoints of each edge must not have a common neighborfor u, v in edges:if adj[u] & adj[v]:return False, f"Triangle found involving edge ({u},{v})"# Four-cycle: any two distinct vertices must not have more than one common neighborfor u, v in combinations(range(n), 2):if len(adj[u] & adj[v]) >= 2:return False, f"Four-cycle found involving vertices {u} and {v}"return True, f"Passed: vertices={n}, edges={len(edges)}"
The checking logic is straightforward. If two vertices have two common neighbors, a four-cycle exists between them; if two adjacent vertices have a common neighbor, a triangle exists. The checking result is deterministic: the same input always produces the same judgment. In this way, “the model produced a graph that satisfies the conditions” becomes a proposition that can be automatically verified.
Note that this checker can only verify whether the graph itself satisfies the conditions. It cannot determine whether the graph is the optimal solution to the problem. Judging whether it exceeds a known reference value requires a separate comparison of edge counts, and judging whether it is a new result requires domain background.
Signals from a Public Evaluation
A publicly reported evaluation from a research team provides a set of reference data. In a test involving 20 theorems, the best-performing agent completed 14 without being provided intermediate lemmas, for a success rate of 70.0%. Another agent succeeded 23 times in 60 attempts, covering 9 different theorems, for a success rate of 38.3%. The remaining six agents all had success rates no higher than 11.7%.
When calculating success rates, coverage is as important as the success rate itself. The following function reports both:
def success_stats(results):total = len(results)solved = [r for r in results if r["passed"]]covered = {r["task_id"] for r in solved}return {"success_rate": round(len(solved) / total, 3),"covered_tasks": len(covered),}
The same evaluation also shows that a single ranking cannot represent all capabilities. On a reliability task related to probabilistic provability, one agent completed all three attempts, while an agent with a higher overall score did not solve that task. Therefore, models should be compared by task type rather than by overall leaderboard position alone.
Two constructive results in the evaluation are more specific. First, an agent constructed a graph with 124 vertices and 630 edges that contains no triangles or four-cycles, exceeding the reference value of 629 edges. The checker above can directly verify this result. Second, an agent tightened a coding upper bound from 84 to 82. Conclusions of this kind require a complete proof and usually also need review by experts or formal tools.
Capability Landscape: Which Parts Can Be Delegated
Based on the data above, models perform better when the following conditions are met at the same time:
- Clear problem boundaries: the input, output format, and decision criteria are all explicit.
- Verifiable feedback: an automatic checking program exists that can return a pass or fail result.
- Sufficient budget: multiple attempts are allowed, and all intermediate results are retained.
- Decomposable tasks: a large problem can be split into local subproblems, such as local derivations, parameter searches, or program optimization.
In these scenarios, models can take on work such as constructing candidate objects, quickly screening large numbers of options, and completing local proof steps. Researchers can then focus their time on harder judgments.
Where Humans Are Still Needed
- Choosing research directions: models have difficulty judging which problem is worth pursuing and assessing the upstream and downstream value of a problem.
- Judging the value of results: whether a bound is meaningful depends on its impact on existing theory, which requires domain background.
- Tackling major open problems: in current public results, breakthroughs remain concentrated on decomposable and verifiable subproblems. There are no cases of autonomously solving major open problems.
- Identifying incorrect assumptions: a checking program can only check the conditions it is told to check. If the decision criteria are written incorrectly, an incorrect conclusion will pass as if it were correct.
Practical Implementation Recommendations
If a team wants to introduce model-assisted research, it can refer to the following practices:
- Write tasks as checkable specifications. Specify the input format, output structure, and decision criteria, and avoid vague requests such as “give me a good result.”
- Separate generation from checking. Equip each type of output with an independent checking program. The checking program should be reviewed by humans and should not be written or modified by the generating model.
- Evaluate by task, not by overall leaderboard. Record every attempt, calculate the success rate and the number of covered tasks, and observe differences across tasks.
- Repeatedly verify unstable results. If a success cannot be reproduced, it should be treated as an incidental result rather than a stable capability.
- Keep human review checkpoints. Any claim involving a “new conclusion” or exceeding a known bound must be verified by experts before publication.
With this workflow, model outputs can be checked item by item, and the research team can clearly know which results are reliable and which still need further scrutiny.
