AI-Assisted Research: Progress and Boundaries of Verifiable Tasks

    Evaluating AI capabilities in scientific research is challenging due to the difficulty of objectively grading research outputs. This article explores how integrating AI research outputs into verifiable systems provides a reliable framework for assessment. We discuss key evaluation design principles, how to interpret empirical results, and real-world examples from constructive problems such as extremal graph theory and coding theory. Finally, we outline the current boundaries between AI capabilities and human expertise, offering practical advice for R&D teams looking to implement AI-assisted research workflows.

    1. Why Put Research Output into Verifiable Systems?

    Evaluating AI’s research capabilities is difficult because research outputs are hard to grade. A plausible-sounding argument might contain hidden logical flaws, and a seemingly novel conclusion might merely be a restatement of existing results. Therefore, the first step in evaluation design is to ensure that every output can be independently verified.

    Common research outputs can be divided into three categories:

    • Objects: A graph, code, or sequence that satisfies specific constraints. The verification method is to check each constraint individually.
    • Parameters: A set of numerical values that allow a construction or algorithm to meet certain metrics. The verification method is to substitute the values and compare the results.
    • Proofs: A complete chain of reasoning. Verification methods include formal verification or step-by-step review by experts.

    Among these three types of outputs, the first two are the easiest to automate. As long as the verification program can provide a clear pass or fail conclusion, the credibility of the evaluation is significantly enhanced.

    2. Key Points in Evaluation Design

    A reliable evaluation system typically needs to control four variables simultaneously.

    1. Clear problem boundaries. If a problem comes with numerous intermediate lemmas or hints, the model’s score primarily reflects the quality of the hints rather than its autonomous reasoning ability. Therefore, a more persuasive setup is to provide only the original problem without intermediate conclusions, requiring the system to complete the proof from scratch.

    2. Sufficient attempts. A single success might be luck. Multiple independent attempts with a tracked success rate are necessary to distinguish stable capability from random chance. The denominator of the success rate must also be clearly stated. For instance, “completing 14 out of 20 problems” and “succeeding 23 times out of 60 attempts” represent two different statistical metrics.

    3. Coverage over repetition. If a system repeatedly solves the same easy problem, its success rate might be high, but its capability range remains narrow. Counting the number of distinct successfully solved problems better reflects the system’s generalizability.

    4. Baselines for comparison. To judge the magnitude of progress, results must be compared within the same framework against other systems, human experts, or historical methods.

    3. How to Interpret Empirical Results

    In one evaluation, the top-performing system completed 14 out of 20 theorem-proving problems, achieving a 70% success rate. Another system succeeded 23 times out of 60 attempts, covering 9 different theorems, resulting in a 38.3% success rate. The success rates of the remaining six systems did not exceed 11.7%.

    These numbers need to be understood in layers.

    First, a ranking does not indicate a comprehensive advantage. In a reliability-related task, the second system completed all three attempts, while the top-ranked system failed to solve it. Different systems have different strengths, and a single ranking can obscure these differences.

    Second, the success rate must be considered alongside problem difficulty and coverage. At a similar success rate of around 40%, the system covering more distinct problems usually has greater reference value.

    Third, all conclusions should be based on reproducible data. Evaluation reports should disclose the problems, criteria, and number of attempts to facilitate independent verification by other teams.

    4. Constructive Problems: Verifiable New Results

    More intuitive achievements in evaluations come from constructive problems. One example is the extremal graph problem: given a fixed number of vertices, find a graph with as many edges as possible that satisfies specific local constraints.

    The constraint here is a “girth of at least 5,” meaning the graph contains neither 3-cycles (triangles) nor 4-cycles. In one task, a system constructed a graph with 124 vertices and 630 edges satisfying this constraint, surpassing the reference value of 629 edges given in the task.

    The value of such results lies in their direct verifiability. The code below demonstrates the verification logic, using an adjacency set to represent the graph, checking for triangles and 4-cycles, and counting the edges.

    1. def has_triangle(adj):
    2. """A triangle exists if and only if an edge has common neighbors at both ends."""
    3. for u in adj:
    4. for v in adj[u]:
    5. if adj[u] & adj[v]:
    6. return True
    7. return False
    8. def has_c4(adj):
    9. """A four-cycle exists if and only if two different vertices have at least two common neighbors."""
    10. vertices = list(adj)
    11. for i, u in enumerate(vertices):
    12. for v in vertices[i + 1:]:
    13. if len(adj[u] & adj[v]) >= 2:
    14. return True
    15. return False
    16. def verify_girth_at_least_5(adj):
    17. """Assume adj is the symmetric adjacency set of an undirected graph, returns (whether constraints are met, number of edges)."""
    18. edges = sum(len(nbrs) for nbrs in adj.values()) // 2
    19. ok = not has_triangle(adj) and not has_c4(adj)
    20. return ok, edges
    21. # Example: A pentagon has a girth of 5 and 5 edges
    22. pentagon = {
    23. 0: {1, 4},
    24. 1: {0, 2},
    25. 2: {1, 3},
    26. 3: {2, 4},
    27. 4: {3, 0},
    28. }
    29. print(verify_girth_at_least_5(pentagon)) # (True, 5)

    The verification logic in this code is simple, but it embodies a core principle: constructive results must be independently confirmed by a program, not just by narrative description.

    Another example is an upper bound problem in coding theory. A system proved that the number of feasible codewords under specific parameters does not exceed 82, tightening the previously published upper bound of 84. Improving an upper bound similarly requires a complete mathematical proof and acceptance by expert review.

    5. What Models Can Do vs. What Humans Must Do

    Synthesizing the cases above, the current boundaries of capability can be summarized as follows.

    Capabilities Already Achieved

    • Completing partial proofs under conditions where the problem description is clear, the computational budget is sufficient, and feedback can be automatically verified.
    • Optimizing the performance of existing programs and confirming improvements through benchmark testing.
    • Finding objects better than reference values in constructive problems, or improving known bounds.

    Stages Still Relying on Humans

    • Topic selection: Determining which problems are worth studying requires a holistic grasp of the structure and significance of the field.
    • Value judgment: Whether a result is important cannot be judged solely by whether the numbers are better; it also depends on whether it reveals new mechanisms and can be generalized.
    • Major open problems: These problems usually require proposing entirely new conceptual frameworks, and current systems still have limited performance in this regard.

    Therefore, a more realistic division of labor is to let the system handle a large amount of verifiable exploratory work, while researchers are responsible for posing questions, designing verification criteria, and interpreting results.

    6. Practical Advice for Research and Engineering Teams

    If you plan to introduce a similar AI-assisted research workflow within your team, consider the following practices.

    1. Build the verifier before the generator. Before letting the system produce results, write the verification program first to ensure every result can be automatically judged.
    2. Keep complete attempt records. Record the input, output, judgment result, and time consumed for each attempt. This facilitates calculating success rates and analyzing failure causes.
    3. Set up independent test sets. Evaluation problems should not appear in the system’s training or tuning data; otherwise, scores will be artificially inflated.
    4. Layered review of results. Results that can be automatically verified are entered directly into the database, while proofs that cannot be automatically verified are handed over to experts for step-by-step review.
    5. Clarify responsibility. Published conclusions should be verified by named researchers, with system outputs serving only as material pending review.

    By adopting these practices, the role of AI in scientific research can be leveraged more stably, avoiding the presentation of unverified conclusions as established findings.