Evaluating Execution Reliability of LLMs in Multi-Turn Retrieval Tasks
Evaluating the execution reliability of Large Language Models (LLMs) in complex tasks is challenging, as fluent outputs often mask underlying execution failures. This article explores structured evaluation practices for multi-turn retrieval tasks and other common LLM task types. It details methods for breaking down evaluation dimensions, auditing execution traces, cross-checking model narratives against logs, and verifying algorithmic convergence. Additionally, it provides engineering recommendations for implementing layered testing and maintaining consistent evaluation standards to reduce execution errors before deployment.
1. Why Structured Evaluation is Necessary
LLMs typically generate fluent and well-formatted responses, creating the illusion that they have correctly executed the task. However, in multi-step tasks, common model issues are not found in the language expression but in the execution process:
- Unexecuted instruction constraints: For example, requiring a fixed number of operational rounds, but the actual count does not match.
- Result reuse and fabrication: Subsequent steps reuse earlier intermediate results while claiming to have newly retrieved them.
- Loss of key constraints: Including inapplicable entries in domain configuration answers, or omitting required core items.
Relying on subjective reading makes these issues difficult to detect. Evaluation must be broken down into observable and verifiable dimensions, checking each item individually.
2. Breaking Down Evaluation Dimensions
Before evaluation, clarify the task type to select the appropriate verification method. Common tasks can be divided into four categories.
2.1 Visual and Interaction Generation
The output of these tasks includes pages, animations, and graphical interfaces. The evaluation focus is on functional usability rather than visual aesthetics. This can be judged from the following points:
- Whether the interface loads normally without blank spaces, occlusions, or errors.
- Whether core interactions can be completed, such as producing expected state changes after clicking, dragging, or inputting.
- Whether the presentation format requested in the prompt is actually implemented. If the thought process claims to present a certain style, the final product is the standard; descriptions in the thought process cannot serve as acceptance criteria.
Using screenshots combined with state checks is a common practice. For scenarios requiring precise geometric relationships, you can assert the positions and collision parameters of key objects in the code.
2.2 Algorithm Implementation Tasks
Algorithmic tasks should be verifiable. Taking Q-learning in a grid environment as an example, evaluation can include:
- Whether the agent’s action layer aligns with the terrain layer, i.e., whether action positions are consistent with obstacle and endpoint coordinates.
- Whether the trained policy can navigate from the start to the endpoint.
- Whether the Q-value updates stabilize.
2.3 Multi-Turn Retrieval Tasks
This is the focus of this article. The core issue of retrieval tasks is “whether the execution process is real.” Therefore, it is necessary to record the query terms and returned results of each round and compare them with the model’s final narrative.
2.4 Domain Configuration Q&A
These tasks require answers that meet both completeness and accuracy. A typical error is introducing entries that do not belong to the current configuration or confusing similar concepts. Evaluation should establish two lists: must-include and must-exclude, and check them item by item.
3. Evaluation Methods for Multi-Turn Retrieval Tasks
3.1 Record Execution Traces for Each Round
To determine if the model truly executed the specified number of retrieval rounds, logs must be recorded at the tool-calling layer rather than relying on the model’s own statements. Each record should at least include the round number, query terms, and the result identifier.
Below is illustrative pseudocode demonstrating the basic logic of round auditing:
Function AuditRetrievalRounds(TraceList, ExpectedRounds):ProblemList = EmptyActualRounds = Length of TraceListIf ActualRounds != ExpectedRounds:ProblemList Append "Round mismatch: expected ExpectedRounds, actual ActualRounds"SeenResults = EmptySetFor each Round i Trace in TraceList:Duplicates = Items in Trace.ResultID that belong to SeenResultsIf i > 1 and Duplicates is not empty:ProblemList Append "Round i reused previous results: Duplicates"SeenResults Add Trace.ResultIDReturn ProblemList
This check identifies two types of problems: first, the actual number of calls does not match the requirement; second, the results of subsequent rounds completely overlap with previous ones, indicating that the retrieval did not actually produce new information.
3.2 Cross-Check Narrative Against Logs
Models often describe in their final answers, “I conducted several searches” or “based on a certain retrieval result.” Such statements should be compared with the logs. If the stated frequency does not match the logs, or the cited source is not in the current logs, it should be judged as an execution deviation.
In the evaluation report, it is recommended to record the following three items separately:
- The actual number of calls in the logs.
- The number of calls claimed by the model.
- Whether the sources cited in the final answer can all be found in the logs.
3.3 Automated Validation of Key Constraints
For domain configuration content, you can prepare a constraint list in advance and use simple rules to check the answer:
Function ValidateConstraints(ResponseText, MustInclude, MustExclude):MissingItems = Items in MustInclude not appearing in ResponseTextViolations = Items in MustExclude appearing in ResponseTextReturn {MissingItems, Violations}
String matching only covers explicit entries and cannot judge semantics. Therefore, it is suitable as a first-pass screening; samples hitting violation items should be manually reviewed. The constraint list itself needs to be maintained by domain experts and updated synchronously with configuration changes.
4. Verification Methods for Interaction and Algorithm Results
4.1 Fix Random Seeds and Run Multiple Times
The results of reinforcement learning experiments involve randomness. A single run of “converged” or “not converged” is insufficient to explain the problem. It is recommended to fix random seeds, repeat training under multiple seeds, and observe whether the convergence trend is stable.
4.2 Determine Convergence Using Numerical Metrics
To determine if Q-values have converged, you can compare the maximum change between adjacent iterations:
Function CheckConvergence(CurrentQ, PreviousQ, Tolerance):MaxChange = Max absolute difference element-wise between CurrentQ and PreviousQReturn MaxChange < Tolerance
When making a determination, the tolerance value, iteration count, and random seed must be stated simultaneously; otherwise, the conclusion cannot be reproduced.
4.3 Verify Policies with Behavioral Results
Numerical convergence does not guarantee a correct policy. It is also necessary to start from the origin, follow the learned policy to complete the entire path, and check whether the endpoint is reached and obstacles are avoided. This step can detect situations of “numerical convergence but behavioral error,” such as state encoding being inconsistent with the map.
5. The Role of Small Models in the Reinforcement Learning Process
In the Reinforcement Learning from Human Feedback (RLHF) process, bigger is not always better for model size. Small models are often used in the following stages:
- Data filtering: Preliminary filtering of large batches of candidate samples to remove obviously low-quality data, reducing annotation costs.
- Initial screening of reward signals: Used to quickly provide coarse-grained scores, followed by stronger models or manual review of key samples.
- Evaluation assistance: Quickly running a large number of use cases in regression testing to detect obvious degradation.
When using small models as evaluators or filters, be mindful of their judgment limits. They are suitable for handling rule-based issues like formatting, length, and obvious factual errors. For complex reasoning and professional knowledge, higher-level verification methods should be retained.
6. Engineering Implementation Recommendations
- Establish a fixed-version evaluation set covering the four task types mentioned above. Perform regression testing after each model or prompt change.
- Layer the checks: Format and structure checks first, followed by execution trace and constraint validation, and finally manual spot-checks.
- Categorize and record failed samples, such as round deviation, result reuse, constraint omission, interface unavailability, etc., to facilitate tracking improvement effects.
- Maintain caution regarding narrative conclusions. Any statement involving “completed” should be backed by logs, screenshots, or numerical results.
- The proportion of manual spot-checks should be determined by risk. For content involving core configurations or live business operations, the spot-check ratio should be higher.
Through these steps, teams can translate subjective impressions of the model into reproducible check results, reducing execution errors that only surface after launch.
