Evaluating AI Agents: A New Paradigm from Real Conversations to Causal Inference

Challenges and Paradigm Shifts in Agent Evaluation

With the rapid evolution of large language models, AI Agents have evolved from simple conversational systems into comprehensive systems capable of executing complex tasks, invoking external tools, and completing multi-step operations. However, how to scientifically and objectively evaluate the true capabilities of these agents remains a core challenge for the industry.

Traditional evaluation methods primarily rely on two approaches: static benchmark datasets based on manual annotation, and arena-style modes based on user voting. The former carries the risk of data contamination, where benchmark scores can severely diverge from actual user experience. The latter, while incorporating real user preferences, introduces subjectivity into voting behavior and struggles to cover long-term, multi-step complex agent tasks.

Against this backdrop, a new evaluation approach is emerging—automatically extracting behavioral signals from real user-agent conversations and estimating the model’s true improvement rate through causal inference methods. This evaluation system does not rely on any manual scoring or voting. Instead, it directly uses user interaction behaviors in real-world scenarios as the evaluation basis, significantly enhancing the ecological validity of the assessment.

Core Evaluation Methodology: Multi-Signal Extraction and Causal Tracing

Signal Mining: Discovering Quality Clues from Interactive Behaviors

The core innovation of this evaluation system lies in the fact that it does not ask users “which response is better.” Instead, it automatically extracts several discriminative behavioral signals by analyzing the complete interactive conversation between the user and the agent. These signals reflect the model’s actual performance during collaborative task execution, including:

• Explicit Task Confirmation: When a user expresses positive feedback regarding the model’s proposed solution or result, such as “this plan works” or “exactly what I needed,” it indicates the model accurately understood the user’s intent and provided an effective response.
• Praise and Complaints: Positive or negative emotional expressions naturally流露 by users during the session serve as the most direct quality feedback signals. Praise typically corresponds to high-quality solutions, while complaints indicate shortcomings in the model’s understanding, reasoning, or execution phases.
• Correction Instructions: When a user must redirect or correct the model, it implies the model’s initial output failed to fully meet expectations. The frequency and magnitude of corrections indirectly reflect the model’s “first-time success rate.”
• Shell Command Execution Errors: In task scenarios requiring specific command execution, such as programming or system operations, the command failure rate is a hard metric for measuring the model’s tool invocation capability and instruction comprehension accuracy.

These signals are designed as mediating variables highly correlated with task success. They can be continuously and cost-effectively obtained from existing session data without disturbing the user.

Causal Tracing: Estimating Net Improvement Rate via Randomized Inference

Merely extracting signals is insufficient; the second challenge is aggregating these signals into comparable model scores. This evaluation system introduces a randomized inference method called “Causal Tracing.” The core logic is as follows:

1. For each pair of model output and user feedback, a counterfactual framework is constructed: If the same user and the same task were handled by a different model, how would the user’s feedback change?
2. Through randomized experimental design (e.g., randomizing session routing), the model identity is treated as a treatment variable, and user feedback as the outcome variable. After controlling for confounding factors such as task difficulty and user preferences, the “net improvement rate” of each model relative to the average level of all evaluated models is estimated.
3. The net improvement rate is expressed in percentage points. A positive value means the model’s actual performance is above average, while a negative value indicates below-average performance.

The advantage of this method is that it breaks away from the traditional win-rate or score-ranking framework. It directly quantifies the incremental contribution of a specific model relative to a baseline, making evaluation results comparable across different times, batches, and task distributions.

Fine-Grained Metric System for Evaluation Dimensions

To help developers comprehensively understand the strengths and weaknesses of model capabilities, this evaluation system breaks down several sub-dimensional metrics alongside the overall score. Two key directions deserve attention:

• Execution Reliability: Measures the accuracy and stability of the model when issuing operational commands (e.g., writing and running scripts, calling APIs, modifying configurations). This metric directly relates to whether an agent can be trusted in a production environment.
• Direction Adjustment and Correction Capability: Measures whether the model can quickly correct its behavior and continue executing along a new direction after receiving user corrections or additional constraints. This reflects the model’s context understanding and dynamic adaptation ability, which are key characteristics of a mature agent.

This multi-dimensional evaluation design avoids the pitfall of a single total score masking specific shortcomings, enabling model developers to optimize and iterate more targetedly.

Practical Case: Performance Comparison Among Open-Weight Models

In actual evaluations based on the aforementioned methodology, a notable result comes from a large tech company. The company open-sourced the weights of a model with a parameter scale reaching 2.8 trillion, a highly competitive size among open-weight models. The model’s performance on the evaluation leaderboard sparked industry discussion:

• Ranked first among open-weight models and 13th among all evaluated models;
• Ranked at the top of open-weight models in three typical scenarios: Work (productivity assistance), Coding, and Chat;
• The rankings are based on over 17,500 real long-cycle agent task sessions from global users, providing strong statistical reliability through sample size.

The model’s comprehensive net improvement rate is +5.1%. Compared to its previous generation version (net improvement rate of +0.9%), it achieved a significant leap of 4.2 percentage points, indicating that the new generation model has seen a qualitative飞跃 in execution reliability and direction correction capabilities. This data also validates the sensitivity of the evaluation method from a side perspective: when a model’s actual capabilities undergo significant changes, measurements based on real conversation signals can promptly capture these differences.

Insights and Application Suggestions for Developers

For developers building agent applications, this evaluation system offers several actionable insights:

1. Establish a Capability Monitoring System Based on Real Logs

Rather than relying on periodic offline evaluations, embed evaluation capabilities into the product’s log collection and monitoring system. By defining key behavioral signals (such as user correction rate and task completion confirmation rate), teams can achieve continuous tracking of model capabilities and regression alerts.

2. Emphasize Counterfactual Evaluation

In A/B testing, use random routing as much as possible and record sufficiently rich conversational context for subsequent counterfactual analysis. This helps teams accurately attribute the true incremental benefits brought by model updates, without being interfered by traffic fluctuations or changes in user demographics.

3. Focus on Long-Tail and Multi-Turn Interaction Metrics

Simple first-turn response accuracy is insufficient to comprehensively measure agent capabilities. Focus on tracking long-cycle metrics such as correction costs in multi-turn dialogues, tool invocation failure rates, and instruction-following consistency. These are the critical factors determining user retention.

4. Leverage Open-Weight Models Rationally

For teams with data privacy requirements or those needing private deployment, large open-weight models provide a controllable foundational choice. However, note that benchmark rankings are only the starting point. Actual application performance still requires finer-grained validation and tuning tailored to specific business scenarios.

Conclusion

Agent evaluation is shifting from “subjective voting” to a new stage of “objective behavioral analysis.” The evaluation system based on real conversation signals and causal inference demonstrates how to extract scientifically rigorous quality metrics from massive user interactions. The popularization of this methodology will help foster a healthier, more transparent agent ecosystem—ensuring that improvements in model capabilities genuinely serve the successful completion of user tasks, rather than merely climbing ranks on static leaderboards. For developers, understanding and leveraging this evaluation mindset is a crucial step in building technical barriers in the highly competitive agent application track.