Verifiable Evaluation: How to Measure AI Agents' Research Capabilities

    Large language model agents have evolved from answering questions to completing tasks, but measuring whether they can actually conduct research still lacks a unified standard. This article presents a verifiable evaluation method designed around the research process. Instead of comparing a final answer against a reference answer, it assesses whether outputs—constructions, parameters, and proofs—can be independently verified by deterministic programs. Using Theoretical Computer Science as a testing ground, the article explains the core design logic, four task categories, strict and relaxed evaluation tracks, the sandbox-and-verifier framework, a multi-dimensional metrics system, and how to read results as capability profiles rather than a single ranking. It also covers engineering implementation, contamination resistance, and common pitfalls. Readers will learn how to build a reproducible, comparable, and traceable evaluation pipeline that diagnoses and compares agents' research abilities.

    1. Background: Why Measuring Research Ability Is Hard

    LLM-driven agents have moved from “answering questions” to “completing tasks.” They can run code in a sandbox, retrieve information, call symbolic computation tools, and carry out dozens of consecutive steps. Yet we still lack a unified standard for measuring whether they can actually do “research.”

    Most traditional benchmarks rely on static question banks and suffer from three limitations:

    • Fixed answer format: Multiple-choice questions, standard Q&A, or code unit tests mainly assess knowledge already acquired and routine programming skills, and struggle to reflect open-ended exploratory ability.
    • Susceptible to being “memorized” from training data: Once public problems and solutions enter the corpus, scores become inflated and no longer measure true ability.
    • Scoring depends on answer matching: Open-ended outputs have no single correct answer, so scoring with string similarity, keywords, or model-based review often produces misjudgments where an answer “looks right but doesn’t hold up.”

    Theoretical Computer Science (TCS) research offers a nearly ideal testing ground:

    1. Clear problem structure: Many results in fields such as Extremal Combinatorics, Coding Theory, and Proof Complexity ultimately land on three concrete types of output: a construction, a set of parameters, or a proof.
    2. Correctness is in principle decidable: Whether something is correct can be strictly determined in principle, without relying on subjective opinion.
    3. Difficulty is naturally layered: Problems range from standard textbook exercises to open questions that require months of dedicated research.
    4. Outputs drive real progress: Many results can directly add new records to mathematical databases and the literature.

    Against this backdrop, a team of frontier researchers and their collaborators proposed a verifiable evaluation method aimed at the research process (referred to below as “this method”), bringing different stages and different types of outputs into a single, reviewable system. This article does not focus on the rankings of any particular evaluation run; instead, it dissects the design logic and the transferable engineering practices behind it.

    2. Core Idea: Verifiability First, So Every Attempt Leaves Reviewable Evidence

    The most important shift in this method is this: the object of evaluation is no longer “whether the final answer looks similar,” but “whether the output can be independently verified.”

    Traditional Q&A-style evaluation can be summarized as “question → answer → compare against a reference answer,” which naturally encourages memorized responses. Verifiable evaluation rewrites this as:

    1. Task (task package + inputs + reference material)
    2. ↓
    3. Agent execution (multiple attempts allowed, restricted tools, explicit budget)
    4. ↓
    5. Output (construction / parameters / proof / proof sketch)
    6. ↓
    7. Independent verifier (a deterministic program fully isolated from the agent)
    8. ↓
    9. Structured record (pass/fail, output size, verification time, failure reason)

    This redesign brings three changes:

    • Verification does not depend on a gold answer: The benchmark only needs to provide a “threshold” or “reference value,” rather than maintaining a reference solution. The reference value draws a starting line that can be provably surpassed; it is not a template for answering.
    • Surpassing matters more than reproducing: If an output meets or exceeds the reference value and can be independently verified, it constitutes a new contribution. This means evaluation results and research output can flow in both directions.
    • Provable evidence is preferred: A “plausible-looking new answer” cannot enter the evaluation. Only two kinds of objects can: constructions or parameters that a program can check, and derivations that a proof checker can verify. Evaluation thus shifts from “looking at the wording” to “looking at the evidence.”

    It is worth noting that “verifiable” does not mean “easy.” When constructing extremal graphs, once the number of vertices and edges grows even slightly, brute-force enumeration becomes infeasible and the search space explodes combinatorially; formal proofs also require advancing step by step through an enormous set of logical rules. Verifiability solves “how to judge,” while difficulty addresses “whether it can be produced,” and the two are complementary.

    3. Task Design: What Kinds of Tasks Fit Verifiable Evaluation

    3.1 Four Task Categories by Output Type

    Task type Example problem Output Verification Scoring
    Construction In an extremal graph problem with girth at least 5, construct a graph with n vertices that contains no triangles or quadrilaterals Adjacency list or edge list of the graph A deterministic program checks the structural constraints and counts the edges Whether it satisfies the constraints and exceeds the reference edge count; can be scored by “percentage exceeded”
    Bounding Find the maximum number of codewords A(n,d) for a binary code of length n where any two codewords differ in at least d positions, and prove a tighter upper bound Upper-bound value + proof The derivation is verified by a formal proof checker Whether it gives a verified upper bound smaller than the reference value
    Proof-system Decide the alphabet reliability problem in probabilistically checkable proof (PCP) systems Proof or counterexample + reasoning chain Formal checking, or review by an independent reviewer Whether it is correct and the derivation is complete
    Theorem reproduction Independently re-prove a classical theorem under given constraints Complete proof or proof sketch Formal checking, or a reviewer checks against a checklist Number of different theorems covered, attempt success rate

    Taking construction tasks as an example, a reference value of 629 edges means: the evaluation does not require the agent to invent a new method from scratch, but to find and verify an instance better than the current record within a limited budget. Such tasks test three abilities at once:

    • Modeling ability: translating a natural-language problem into precise constraints;
    • Search and construction ability: exploring efficiently in a space of combinatorial explosion;
    • Self-checking ability: verifying legality before submission, rather than “finishing and handing in.”

    3.2 Breaking the Research Process into Scorable Stages

    Research is not a one-shot “from problem to answer”; it is a staged pipeline:

    Literature and background → problem formalization → construction or proof → verification and correction → conclusion drafting

    Matching verifiable tasks to each stage both pinpoints capability gaps and reduces cases where “the answer is right but the process was cobbled together.” The specific matching is:

    • Literature review: ask the agent to identify key results and their constraints from bibliographic information, or use citation-relation questions to test literature tracing.
    • Problem formalization: given a natural-language description, require precise, unambiguous formal definitions and constraints.
    • Construction and proof: directly correspond to construction, bounding, and theorem-reproduction tasks.
    • Verification and correction: correspond to two task types—“find a flaw in your own proof” or “find the erroneous step in a given proof.”
    • Conclusion drafting: research tasks often require output that is human-readable and reviewed by someone with the relevant background, so its evaluation should differ from purely machine-checked tasks. It is listed here to illustrate the boundary of layered evaluation.

    3.3 Six Principles of Task Design

    1. Single responsibility: each task assesses one clear research step, avoiding multiple mixed abilities that make attribution difficult.
    2. Verifiable reference values: reference values must carry a source—for example, a published result or an instance independently constructed by the benchmark maintainers with a deterministic program—and be annotated with the year of publication.
    3. Strict and relaxed dual tracks: strict conditions provide no intermediate lemmas or reference material, testing the ability to “start from scratch”; relaxed conditions allow the given reference material, testing the ability to advance on top of existing results. Scores under the two conditions cannot be compared directly.
    4. Reference values go only into the verifier: reference values are used solely to determine whether the threshold is met and are not disclosed to the tested agent, so it does not spend effort “replicating the reference construction.”
    5. Contamination resistance and rotation: the core task set is not public; the public set periodically generates new problems from parameter variants—for example, by changing the number of vertices, the code length, or the minimum distance—so as to test the method rather than memory.
    6. Controllable cost: problem sizes should let the verifier finish in seconds to minutes; otherwise the per-task cost limits the number of attempts.

    4. Evaluation Framework: Task Specification, Execution Environment, and Independent Verification

    4.1 Task Specification: Machine-Readable Task Packages

    Task definitions should use structured, versioned task packages rather than natural-language descriptions scattered across documents. The illustrative code below gives a minimal task-package structure:

    1. {
    2. "task_id": "extremal-graph-girth5-v1",
    3. "version": "1.0",
    4. "category": "construction",
    5. "statement": "Construct a simple undirected graph with n vertices that contains no triangles or quadrilaterals and has strictly more than reference_edges edges.",
    6. "input": { "n": 124 },
    7. "output_schema": {
    8. "format": "edge_list",
    9. "vertices": "integers 1..n",
    10. "edges": "[[u, v], ...], no self-loops, no duplicate edges"
    11. },
    12. "budget": { "max_steps": 50, "max_tool_calls": 20, "wall_clock_seconds": 600 },
    13. "allowed_tools": ["symbolic_compute", "graph_lib", "solver"],
    14. "reference": {
    15. "edges": 629,
    16. "source": "published record, 2024",
    17. "hidden_from_solver": true
    18. },
    19. "scoring": {
    20. "pass_if": "valid && edges > reference.edges",
    21. "partial_credit": "min(1, (edges - reference.edges) / reference.edges)"
    22. },
    23. "verifier": "verifiers.graph.verify_girth5"
    24. }

    The task package offers several benefits:

    • Reproducibility: inputs, budget, tool allowlist, and verifier are all explicitly declared, so anyone who gets the same task package obtains identical execution conditions.
    • Composability: verifiers are registered independently, making cross-checking easy and making it easy to add new conditions such as “no tools” or “time-limited.”
    • Governability: version numbers and source information support result tracing and variant generation.

    4.2 Execution Environment: Isolation, Budget, and Reproducibility

    Sandbox isolation

    • The agent runs in a restricted container: the task package is mounted read-only, the network is off by default (enabled per task only when retrieval is genuinely needed), and only allowlisted tools are exposed.
    • The evaluation framework itself runs in a controlled network, isolated from the agent’s network.
    • The container platform provides run instances, object storage holds outputs and run records, a message queue handles task distribution, and monitoring and logging services observe task backlog, failure rates, and resource usage.
    • The verifier and the tested outputs execute in an isolated environment to prevent malicious constructions from profiting by tampering with the verification process.

    Budget constraints: every task must set three kinds of budgets:

    • number of steps (the agent’s decision count);
    • number of tool calls;
    • wall-clock time and token consumption.

    Without budget constraints, comparisons between different systems are meaningless: the one that takes longer is not necessarily stronger.

    Independent verifier: the verifier must be a deterministic program fully isolated from the agent and must not rely on model judgment. Taking the “no triangles, no quadrilaterals” constraint as an example, the checking logic can be implemented like this:

    1. from collections import deque
    2. def has_short_cycle(n, edges, max_len=4):
    3. """Check whether the graph contains a simple cycle of length <= max_len."""
    4. adj = [[] for _ in range(n + 1)]
    5. seen_edges = set()
    6. for u, v in edges:
    7. if not (1 <= u <= n and 1 <= v <= n) or u == v:
    8. raise ValueError("invalid vertex id or self-loop")
    9. if (min(u, v), max(u, v)) in seen_edges:
    10. raise ValueError("duplicate edge")
    11. seen_edges.add((min(u, v), max(u, v)))
    12. adj[u].append(v)
    13. adj[v].append(u)
    14. for s in range(1, n + 1):
    15. dist = {s: 0}
    16. parent = {s: -1}
    17. queue = deque([s])
    18. while queue:
    19. u = queue.popleft()
    20. if dist[u] >= max_len - 1:
    21. continue
    22. for v in adj[u]:
    23. if v not in dist:
    24. dist[v] = dist[u] + 1
    25. parent[v] = u
    26. queue.append(v)
    27. elif v != parent[u] and dist[u] + dist[v] + 1 <= max_len:
    28. return True
    29. return False
    30. def verify_girth5(n, edges, reference_edges=629):
    31. """Return (passed, edge_count). Triangle-free and quadrilateral-free is equivalent to girth >= 5."""
    32. if has_short_cycle(n, edges, max_len=4):
    33. return False, len(edges)
    34. return len(edges) > reference_edges, len(edges)

    A few implementation notes:

    1. Validate inputs first: reject out-of-range vertices, self-loops, and duplicate edges up front so the verifier is not derailed by formatting issues.
    2. Controllable detour-search complexity: the search-depth cap is 3, giving a complexity of roughly O(n·Δ³) (Δ is the maximum degree). For a scale of 124 vertices, this completes in milliseconds.
    3. Binary decision plus continuous scoring: a binary decision on “pass/fail” combined with a continuous score on “how far the reference is exceeded” can distinguish “barely passing” from “a large breakthrough.”
    4. Cross-checking: the same instance is checked again for reachability using an adjacency matrix. Only when two independent implementations agree is a pass confirmed.

    5. Metrics: From “One-Shot Success Rate” to a “Capability Profile”

    A single aggregate score cannot reflect an agent’s true ability, so this method uses a multi-dimensional metrics matrix.

    5.1 Global Metrics

    • Total score: a weighted aggregation of scores across tasks. Weights should be set by task difficulty and representativeness and disclosed in the report.
    • Task completion rate: the proportion of tasks completed in a single run. For example, on 20 theorem-reproduction tasks, an agent scored 71.9 points with a one-shot success rate of 70.0% (14/20), and all 14 were completed without intermediate lemmas. This shows that behind the high score lies strict independent derivation rather than help from hints.

    5.2 Process Metrics: Attempts and “Coverage”

    Open-ended research usually has no unique solution, and a single failure may just reflect an unfavorable exploration path. This method therefore lets the agent make multiple attempts on the same task (e.g., 60), then tallies two kinds of information:

    • Attempt success rate: 23 successes out of 60 attempts, about 38.3%. It reflects the reliability of a single attempt.
    • Number of theorems covered: the 23 successes covered 9 different theorems. It reflects the capability boundary, i.e., “how many different problems it can solve.”

    Viewed together with single-run metrics, this distinguishes three capability profiles:

    • Stable and comprehensive: a high one-shot success rate, and broad coverage of different problems across multiple attempts;
    • Occasional hits: repeated attempts on the same familiar problem with a not-low success rate, but very narrow coverage;
    • Exploratory progress: a lower one-shot success rate, yet broad coverage, indicating that capability comes from wide exploration.

    In particular, these numbers cannot be interpreted in isolation:

    • The success rate over 60 attempts is usually higher than the one-shot success rate; the two cannot be compared directly;
    • Coverage saturates as the number of attempts grows, so the number of attempts should be reported alongside it when comparing.

    5.3 Summary Example

    System Global total One-shot success (strict) Multi-attempt success Theorems covered Capability profile
    A 71.9 70.0% (14/20) — — Comprehensive leader, stable under strict conditions
    B 57.4 — 38.3% (23/60) 9 Good exploration breadth, weaker one-shot stability
    C–H ≤ 11.7 — — — Clear gap from research-level tasks

    The numbers in the A and B columns come directly from evaluation results reported by this method. To facilitate horizontal comparison, two additional statistics are recommended:

    • 95% confidence interval for the success rate: for example, with n=60 and 23 successes, a Wilson interval estimate is roughly [27%, 51%], indicating that the single estimate has a wide fluctuation range;
    • Scores stratified by difficulty.

    5.4 Efficiency and Stability Metrics

    • Unit cost: average tokens, tool calls, and time consumed per verifiable output obtained;
    • Verification time: the verifier’s own runtime overhead, which determines whether evaluation can scale;
    • Run-to-run variance: repeat runs under fixed conditions to observe the magnitude of fluctuation.

    Illustrative summary code:

    1. import math
    2. from collections import defaultdict
    3. def wilson_interval(k, n, z=1.96):
    4. """Wilson score interval, more robust than the normal approximation for small samples."""
    5. p = k / n
    6. denom = 1 + z * z / n
    7. center = (p + z * z / (2 * n)) / denom
    8. half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / denom
    9. return round(center - half, 3), round(center + half, 3)
    10. def summarize(records):
    11. """
    12. records: each entry looks like
    13. {"system": "B", "task_id": "t1", "attempt": 0, "passed": True, "tokens": 1200}
    14. """
    15. attempts = defaultdict(list)
    16. successes = defaultdict(int)
    17. tokens = defaultdict(int)
    18. for r in records:
    19. key = (r["system"], r["task_id"])
    20. attempts[key].append(r)
    21. successes[key] += int(r["passed"])
    22. tokens[key] += r["tokens"]
    23. total_attempts = defaultdict(int)
    24. total_success = defaultdict(int)
    25. covered = defaultdict(set)
    26. total_tokens = defaultdict(int)
    27. for (system, _), rs in attempts.items():
    28. total_attempts[system] += len(rs)
    29. total_success[system] += sum(int(r["passed"]) for r in rs)
    30. total_tokens[system] += sum(r["tokens"] for r in rs)
    31. if any(r["passed"] for r in rs):
    32. covered[system].add(_)
    33. report = {}
    34. for system in total_attempts:
    35. n = total_attempts[system]
    36. k = total_success[system]
    37. lo, hi = wilson_interval(k, n) if n else (0.0, 0.0)
    38. report[system] = {
    39. "attempt_success_rate": round(k / n, 3) if n else 0.0,
    40. "ci95": (lo, hi),
    41. "tasks_covered": len(covered[system]),
    42. "avg_tokens_per_success": round(total_tokens[system] / k) if k else None,
    43. }
    44. return report

    6. Interpreting Results: A Capability Profile Beats a Single Ranking

    6.1 “Comprehensive Lead” Is Not “Comprehensive Dominance”

    It must be stressed that a score gap across systems is not the same as capability dominance. In the same benchmark, the opposite result also occurred:

    • Upset on a hard specialist problem: on the alphabet reliability problem for PCP systems, a lower-ranked agent completed all three attempts, while the overall leader failed to solve it. This shows that an aggregate score can be “averaged out” by problem structure, and some specialized capabilities may show up entirely on individual hard problems.
    • The actual picture of each system fending for itself: in summary:
      • the overall leader completed 14 of 20 theorems;
      • the second-ranked system had a one-shot success rate of about 38.3%, covering 9 different theorems over 60 attempts;
      • the other six agents all scored no more than 11.7 points.

    From an engineering perspective, there is a clear gap between the top tier and the rest, while within the top tier each system has its own strengths.

    6.2 Why Outputs Must Be “Independently Verifiable”

    Whether it is a graph with 124 vertices, 630 edges, and no triangles or quadrilaterals, or tightening the upper bound of A(22,10) from 84 to 82, their persuasiveness comes not from “the model says it holds” but from two facts:

    • the former can be verified edge by edge against the structural constraints by a deterministic program, with the edge count tallied;
    • the latter is a derivation that a proof checker can verify and that can be organized into a paper submitted for peer review.

    This is the deeper value of verifiable evaluation: it makes the agent’s output directly carry the “verifiability” needed to enter the research process, strictly separating “model claims” from “system proofs.”

    6.3 Fairness Correction for Condition Differences

    Scores under strict and relaxed conditions must be interpreted separately. Completing a task “without intermediate lemmas” means the agent must carry out the key derivation steps itself; when reference material is provided, some difficulties have already been resolved in advance. So when comparing different systems, three questions must be answered:

    1. Are the execution conditions the same for both?
    2. Are the number of attempts and the budget the same?
    3. Is the verification standard the same?

    Only when all three match does a score difference have explanatory power.

    7. Engineering Implementation: Building a Reproducible Evaluation Pipeline

    7.1 Overall Architecture

    1. Evaluation control plane
    2. ├── Task repository (versioned task packages + reference values, visible only to the verifier)
    3. ├── Scheduler (task queue, concurrency control, budget allocation)
    4. ├── Execution layer (isolated sandbox + tool allowlist + system-under-test integration)
    5. ├── Verification layer (deterministic verifiers + cross-checking + failure archiving)
    6. └── Reporting layer (metric calculation, confidence intervals, cost statistics, historical comparison)

    Responsibilities of each layer:

    • Scheduler: allocates execution resources per task and controls the number of attempts;
    • Execution layer: guarantees identical run conditions every time;
    • Verification layer: strictly decoupled from the execution layer;
    • Reporting layer: turns raw records into comparable, traceable metrics.

    7.2 Result Record Structure

    Each execution record should contain at least the following fields:

    • task ID and version;
    • run ID and random seed;
    • input conditions (whether reference material is included, budget);
    • the output;
    • verifier return code and verification time;
    • tokens and elapsed time;
    • failure category (parse failure, verification failure, budget exhausted, timeout).

    Structured failure categories are the starting point for optimization: only by knowing “which step the system got stuck on” can you make targeted improvements to prompts, tools, or search strategies.

    7.3 Failure Attribution: From “Got It Wrong” to “Which Step Went Wrong”

    1. def classify(result, budget):
    2. if result["parse_error"]:
    3. return "output_format"
    4. if result["budget_exhausted"]:
    5. return "budget_exhausted"
    6. if result["timeout"]:
    7. return "timeout"
    8. if not result["valid"]:
    9. return "invalid_output"
    10. if not result["passed"]:
    11. return "below_reference"
    12. return "passed"
    13. # Tally the failure distribution per system to locate whether issues are in format compliance, resource planning, or core reasoning

    Different failure types point to different improvement directions:

    • Too many output-format failures: the ability to follow structured output is insufficient;
    • Too many budget-exhausted cases: resource planning and task-decomposition strategies need optimization;
    • Too many below-reference cases: there is a gap in core construction or derivation ability.

    7.4 Contamination Resistance and Fairness

    • Keep the core set confidential: the core tasks and reference values are not public; the public set contains only low-difficulty or already-published results.
    • Variant generation: derive new problems parametrically, preserving the difficulty distribution while disrupting memorization.
    • Consistent conditions: all systems use the same toolset, prompt templates, and budget. If you need to compare “with tools” vs. “without tools,” they must be reported as separate experiment groups.
    • Independent verification: verifiers are maintained by a team not involved in developing the system under test, with periodic cross-checking.
    • Blind testing: the verifier receives only the output, not information about “which system this is,” eliminating scoring bias at the source.

    7.5 Engineering Simplification Tips

    Even in an environment with only general cloud capabilities, you can build the whole pipeline:

    • Container platform: hosts isolated sandboxes, spun up on demand and destroyed after use;
    • Object storage: stores task packages, outputs, and historical results, naturally supporting version tracing;
    • Message queue: smooths peaks and fills valleys, supporting large-scale concurrent attempts;
    • Logging and monitoring/alerting: observe failure rates and resource bottlenecks in real time;
    • Unified model-invocation service: centrally connects to the model under test and uniformly controls the token budget;
    • Automated test pipeline: run evaluations on a schedule and correlate metric changes with system version changes.

    8. Common Pitfalls and How to Avoid Them

    1. Verifier loopholes and “score gaming”: the tested system may optimize against the verifier specifically—for example, satisfying only some constraints, exploiting formatting ambiguity, or repeatedly submitting near-identical solutions within budget.
      • Countermeasure: make the verifier “default-deny,” so illegal formats fail outright; periodically use mutation testing to check the verifier’s strictness—for example, inject fake constructions with missing edges or triangles to confirm the verifier correctly rejects them.
    2. False negatives: correct constructions are misjudged because of formatting issues.
      • Countermeasure: provide strict but clear input schemas, and tally formatting errors separately so they are not mixed with capability failures.
    3. Data contamination: tasks or solutions get into training corpora.
      • Countermeasure: keep the core set confidential, rotate variants, and continuously monitor whether scores on historical problems rise abnormally.
    4. Insufficient sample size: drawing conclusions from one or two attempts, which can reverse which system is better.
      • Countermeasure: report confidence intervals, and require multiple repeated experiments for key conclusions.
    5. Runaway verification cost: for proof tasks, checking time may far exceed generation time.
      • Countermeasure: set verification timeouts and priority queues; prioritize expanding machine-checkable tasks, and organize human-readable text tasks for separate review.
    6. Overfitting to the benchmark: tuning only for evaluation tasks at the expense of general ability.
      • Countermeasure: treat evaluation as a capability-diagnostic tool rather than the sole optimization target, while keeping a general-ability regression set.

    9. Conclusion: Making “Verifiability” the Cornerstone of Research-Ability Evaluation

    The value of this method can be summarized in four points:

    • Verifiable outputs: not relying on answer matching; constructions, parameters, and proofs can all be independently verified;
    • Decomposable process: research is broken into stages—literature, formalization, construction, verification, and drafting—each mapped to evaluable tasks;
    • Profileable capability: dimensions such as one-shot success rate, multi-attempt success rate, theorems covered, and cost together characterize the capability boundary, avoiding “one aggregate score decides everything”;
    • Reusable evaluation: structured task packages, isolated verifiers, and standardized records make evaluation results reproducible, comparable, and traceable.

    For developers and teams, three main threads suffice for implementation:

    1. Make verifiability the first design principle: prefer tasks whose outputs can be judged by deterministic programs;
    2. Make the independent verifier the root of trust: never let the system under test “grade its own paper”;
    3. Use multi-dimensional metrics as the basis for decisions: choose systems based on cost, stability, and coverage rather than just the top ranking.

    When agents begin producing genuinely usable results in mathematical, theoretical, and algorithmic research, a rigorous, open, and reproducible evaluation system is both a ruler for measuring ability and the cornerstone for maturing “agents participating in research.” Organizations can use the general cloud capabilities of Baidu AI Cloud to build their own research-ability evaluation pipeline following the methods in this article, turning evaluation from a one-off activity into an engineering practice of continuous iteration.