Designing and Practicing an Evaluation System for Real-World Agent Tasks

    As large language models and agent systems advance, determining whether a model is genuinely useful has become a shared challenge for developers, enterprises, and researchers. Traditional evaluation often relies on standard question sets, human scoring, or pairwise preference votes. These methods can reflect knowledge and expression quality, but they do not necessarily show how a model performs on complex, sustained, multi-step tasks. This article presents a practical framework for evaluating real-world agent tasks: shifting from answer quality to task outcomes, using real conversation signals, collecting and cleaning evaluation data, estimating net improvement through randomization, tracking terminal reliability, correction behavior, long-horizon stability, and controllability, avoiding common leaderboard pitfalls, and implementing an enterprise evaluation workflow. The goal is to help readers build an evidence system that supports model selection, version upgrades, and production deployment.

    With the rapid development of large language models and agent systems, accurately judging whether a model is truly useful is becoming a shared problem for developers, enterprises, and researchers. Traditional evaluation usually depends on standard question sets, human scoring, or users voting on which of two responses they prefer. These methods can reflect a model’s knowledge capacity and expression quality, but they do not necessarily indicate how the model will perform in complex, sustained, multi-step tasks.

    Real-world agent tasks often involve higher uncertainty. A model must not only understand user intent, but also decompose tasks, call external tools, read and modify files, execute terminal operations, adjust plans based on environment feedback, and recover after errors. A single task may last several minutes or longer, and the final result may not be directly judged from a single-turn answer. Therefore, building an evaluation system for real-world agent scenarios requires a shift from “whether the answer is pleasing” to “whether the task is actually improved.”

    1. From Answer Quality to Task Outcomes

    The core object of agent evaluation should not be only a single round of generated text, but the complete task process. Even if a model can generate fluent explanations, its practical value remains limited if it cannot correctly execute key steps or continues to follow a wrong plan after the environment changes.

    Evaluation for real-world tasks usually needs to focus on the following dimensions:

    1. Task completion: Whether the user’s goal is achieved, whether key artifacts are generated, and whether the final state meets requirements.
    2. Execution reliability: Whether the operations generated by the model are valid, and whether parameter errors, path errors, permission errors, or ineffective repeated operations occur frequently.
    3. Correction ability: Whether the model can adjust in time when the user adds information, corrects direction, or the environment returns an exception.
    4. Process efficiency: Whether the number of interaction rounds, operations, and ineffective attempts required to complete the task remains within a reasonable range.
    5. Controllability: Whether the model can follow user-specified constraints, formats, steps, and boundaries.
    6. Risk level: Whether the model performs destructive operations without confirmation, leaks sensitive information, or causes irreversible impact.

    Together, these metrics determine whether an agent can move from “being able to converse” to “being able to complete work.”

    2. Why Real Conversations Are More Valuable Than Simple Votes

    User preference voting is suitable for evaluating differences between two answers in language expression, information completeness, and subjective satisfaction, but it has clear limitations.

    First, users usually see only limited response snippets and have difficulty judging whether a long-cycle task has truly been completed. Second, voting results are easily influenced by writing style, length, and wording; a more detailed answer does not mean better execution. Third, many agent tasks have clearly delayed feedback, and users may discover only minutes later that a file was not generated, a configuration did not take effect, or a program cannot run.

    Real conversations contain richer behavioral signals. For example:

    • The user explicitly states that the task is complete or the result is usable;
    • The user approves, thanks, or gives positive feedback on a certain step;
    • The user complains that the result is incorrect or does not meet requirements;
    • The user issues a corrective instruction again, asking the model to undo, modify, or redo;
    • The environment returns an operation failure, syntax error, permission exception, or missing resource;
    • The user interrupts the task and switches to another approach;
    • The same task still does not reach the goal after multiple rounds of interaction.

    These signals are not simple sentiment labels, but indirect observations of task outcomes. By structurally analyzing conversations, it is possible to get closer to model value in real production environments.

    3. Collecting and Cleaning Evaluation Data

    Real conversation evaluation must first address data quality. Raw dialogues usually contain a large amount of casual chat, contextual noise, and incomplete tasks, and cannot be used directly for statistics.

    A relatively robust data processing workflow includes the following steps.

    1. Task Boundary Identification

    It is necessary to determine whether a continuous conversation belongs to the same task. This can be divided based on time intervals, changes in user intent, tool call chains, and whether the final goal is consistent. If the user has shifted from “modifying a program” to “writing documentation,” these should be treated as two different tasks to avoid mixing multiple goals in the calculation.

    2. Task Type Classification

    Tasks can be divided into categories such as work collaboration, software development, knowledge Q&A, content processing, data analysis, and system operations. Classification should not be too granular, otherwise sample sizes may become insufficient; it should not be too coarse either, otherwise difficulty differences between task types will be hidden by averages.

    3. Signal Extraction

    Extract events such as user confirmation, positive evaluation, complaints, corrective instructions, and environment errors from conversations, and record their location, severity, and contextual relationships. For example, a user requesting a minor format adjustment after the model has completed the core goal should not be treated as the same level of negative feedback as a user discovering that the result is completely wrong and asking for a redo.

    4. Anomaly Sample Exclusion

    It is necessary to exclude duplicate sessions, obvious test traffic, malicious prompts, empty tasks, non-model errors caused by network interruptions, and sessions lacking necessary context. Otherwise, evaluation results may reflect peripheral system factors rather than model capability.

    5. Privacy and Security Handling

    Real conversations often contain code, business documents, personal information, and internal configurations. Evaluation systems must perform de-identification, access control, and minimized storage, and avoid exposing raw content directly to unrelated personnel. For enterprise scenarios, data retention periods, usage scope, and audit mechanisms should also be clearly defined.

    4. Estimating Net Improvement Rate with Randomization

    Simply counting “success rate” is not enough to compare models fairly. Difficulty may vary significantly across users, tasks, and time periods. If a model happens to handle more easy tasks, its surface-level score may be overestimated.

    Therefore, randomized causal inference methods can be introduced to estimate the net improvement rate a model brings relative to a baseline level. The basic idea is to give different models comparable task samples under conditions that are as similar as possible, and then observe changes in positive and negative outcomes after model intervention.

    The outcome of a task can be simplified as:

    [
    Y \in {1,0,-1}
    ]

    Here, 1 indicates that the task is effectively improved, 0 indicates that the result is basically unchanged, and -1 indicates that the task is affected or clearly degraded. The net improvement rate of a model can be expressed as:

    [
    NIR = P(Y=1)-P(Y=-1)
    ]

    In practical systems, task difficulty, user type, task category, session length, and tool usage also need to be stratified or weighted. The resulting metric is not “how many questions the model answered correctly,” but “how much net benefit the model brings to real tasks compared with the average level.”

    The value of randomization lies in reducing selection bias. For example, complex tasks may be more likely to be assigned to more capable models, or experienced users may be more inclined to actively choose a certain model. Without controlling for these factors, final rankings may reflect user groups and task difficulty rather than true model differences.

    5. Key Metrics in Agent Evaluation

    1. Terminal Operation Reliability

    In software development and system operations tasks, models often need to execute terminal commands. Evaluation should not only check whether commands are generated, but also whether commands run successfully in a real environment.

    Observable signals include:

    • Whether command syntax is valid;
    • Whether paths, parameters, and file names are correct;
    • Whether operations that have already failed are repeatedly executed;
    • Whether error information can be read and the approach adjusted;
    • Whether necessary confirmation is obtained before executing high-risk operations;
    • Whether the approach is non-reusable due to environment differences.

    A highly reliable model does not necessarily succeed on the first attempt every time, but it should be able to quickly identify the source of errors and take effective recovery measures.

    2. Direction Adjustment and Correction Ability

    Agent tasks rarely proceed exactly according to the initial plan. Users may change goals temporarily, the environment may return new constraints, or the model itself may discover that previous assumptions are no longer valid.

    Strong correction ability has three levels: first, it can recognize that the current approach has deviated from the goal; second, it can accurately understand the user’s new requirements; third, it can stop ineffective paths and reorganize subsequent steps. If the model merely mechanically appends content to the original plan, the task often becomes more complex and may even produce cascading errors.

    3. Long-Horizon Task Stability

    Strong performance in short conversations does not mean that a model can handle long tasks. As interaction rounds increase, the model may forget early constraints, confuse file states, repeat completed steps, or gradually deviate from the user’s goal.

    Therefore, evaluation should record task duration, interaction rounds, state changes, and error recovery counts, and observe whether the model’s performance clearly declines in the latter part of a task.

    4. User Controllability

    Agents cannot pursue automation alone; they must also remain controllable. For operations that may produce external impact, such as modifying configurations, deleting data, or publishing content, the model should clearly explain risks and request confirmation when necessary.

    Controllability evaluation can focus on whether the model follows scope constraints, outputs in the specified format, stops when the user requests a pause, and whether it expands the task scope without authorization.

    6. Pitfalls to Avoid in Leaderboard Design

    First, a single score cannot be treated as the entire conclusion. A composite score is convenient for ranking, but it hides differences in model performance across task types. A leaderboard should show both overall results and scenario-specific results.

    Second, rankings should not be published without explaining the statistical methodology. Users need to know what kinds of tasks the samples come from, how outcomes are defined, whether filtering was applied, how confidence intervals are calculated, and whether differences between models are statistically significant.

    Third, time changes cannot be ignored. Model versions, user behavior, and task structures all change, so evaluation results should retain time windows and support rolling updates. For new models, early small-sample results should also be distinguished from long-term stable results.

    Finally, a leaderboard should not be the sole basis for procurement or launch decisions. Enterprises also need to consider data security, deployment cost, response latency, permission control, observability, and failure recovery mechanisms.

    Enterprises can build an internal agent evaluation system through the following steps:

    1. Define core tasks, such as code modification, document generation, data processing, or process automation.
    2. Define verifiable completion criteria for each task type, avoiding reliance only on subjective evaluation.
    3. Establish unified session logs and tool call records.
    4. Structurally annotate user feedback, environment errors, and task outcomes.
    5. Use random sampling or random assignment to reduce sample bias.
    6. Calculate overall metrics, scenario metrics, and risk metrics separately.
    7. Establish online monitoring to continuously observe changes after model upgrades.
    8. Conduct manual reviews of major failure cases and feed conclusions back into prompt design, tool orchestration, and model selection.

    Ultimately, the goal of agent evaluation is not to produce a seemingly precise number, but to establish an evidence system that supports engineering decisions. Only by combining real tasks, process signals, causal analysis, and safety constraints can one judge whether a model has the ability to create stable business value. For developers, this evaluation approach helps identify weaknesses in terminal operations, task planning, and correction. For enterprises, it provides a more reliable basis for model selection, version upgrades, and production deployment.