Causal Tracking for Real-World Agent Evaluation: Moving Beyond User Votes

Why Traditional Evaluation Paradigms Break Down in Agent Scenarios

Over the past few years, evaluation of conversational models has relied primarily on two approaches: static benchmark datasets and human preference voting. The former measures capability using fixed questions with standard answers; the latter asks users to choose between two anonymized responses. Both methods work reasonably well for single-turn Q&A, but they reveal structural weaknesses when applied to agent-based tasks.

Agent tasks are characterized by long horizons, multi-step execution, and heavy tool invocation. A typical task may involve dozens of interaction turns: reading files, executing commands, modifying configurations, verifying results, and rolling back on errors. In this context:

• Static benchmarks cannot capture the noise and unpredictability of real environments, and once questions become public, they can be optimized against.
• Double-blind voting offers too coarse a granularity—users typically see only final outcomes, not the reliability of intermediate steps.
• Manual annotation is expensive and difficult to scale across long-tail tasks.

More fundamentally, what user voting measures—“which answer looks better”—differs from what actually matters for agents: how much real-world effort this model saves me. This semantic gap has pushed the industry toward a new evaluation approach: instead of relying on explicit user votes, extract behavioral signals automatically from real sessions and use causal inference to estimate a model’s net contribution.

Designing Evaluation Signals: Mining “Implicit Votes” from Conversations

When users collaborate with an agent, they naturally leave behind a trail of feedback. These traces are not deliberate annotations but by-products of task progression. Four categories of signals are worth extracting:

1. Explicit Confirmation Signals

When users respond with phrases like “yes,” “that’s it,” “good,” or “continue,” it generally indicates that the previous step was accepted. These are positive signals, but care must be taken to distinguish polite acknowledgments from substantive confirmations. Contextual windows and task stage can be used for weighting.

2. Praise and Complaint Signals

Praise includes statements like “good approach” or “much faster than before”; complaints include “wrong again,” “not what I meant,” or “why isn’t this done yet.” These signals carry strong emotional weight and high informational density, but they are relatively sparse. They should be modeled jointly with task type to avoid amplifying isolated emotional expressions into general conclusions.

3. Corrective Instruction Signals

This is the most valuable category. When a user says “no, you should modify the config file first,” “you missed step three,” or “roll back to the previous version,” they are pinpointing specific deficiencies in the model’s output. Corrective instructions often contain clear error localization, enabling attribution to specific capability dimensions such as planning, tool-use accuracy, or context retention.

4. Command Execution Error Signals

For tasks involving command-line operations, signals like non-zero exit codes from the shell, error stack traces, and timeouts are the most objective negative indicators. They do not depend on subjective user judgment and directly reflect whether the model’s generated commands are executable, safe, and consistent with environment constraints.

After encoding these four categories, each session segment can be represented as a weighted signal vector. The evaluation system thus obtains a data source that requires no manual labeling and can be continuously updated.

Causal Tracking: From Correlated Signals to Net Improvement Rates

Simply counting signal frequencies is insufficient. A model might achieve higher confirmation rates simply because it was assigned easier tasks, or because its users happen to be more tolerant. To reach credible conclusions, confounding factors—task difficulty, user habits, session length—must be stripped away.

The core idea of the “causal tracking” method is randomized inference: in real traffic, tasks are assigned to different models according to a random policy, so that latent confounding variables such as task difficulty and user distribution become approximately balanced across models. On this basis, we compare each model’s signal performance against the population average to derive a “net improvement rate.”

A simplified form can be expressed as:

Net improvement rate (m) = E[signal score | model = m] - E[signal score | all models]

where the signal score is a weighted combination of the four signal types—positive signals contribute positively, negative signals negatively—after normalization. Because assignment is approximately random, the difference can be interpreted as a causal effect of the model itself, rather than as a task-selection bias.

Several practical issues need to be addressed in production:

• Sample imbalance: Different models may have vastly different session volumes, requiring weighted estimation and confidence intervals.
• Temporal drift: User behavior and task distributions change over time, requiring time windows and rolling updates.
• Signal sparsity: Some fine-grained dimensions may have very few signals, requiring hierarchical Bayesian or shrinkage estimation to avoid small-sample noise.
• Multiple comparisons: When evaluating multiple models and multiple metrics simultaneously, the false discovery rate must be controlled.

Metric Decomposition: From Composite Scores to Capability Profiling

A composite net improvement rate is suitable for leaderboard ranking, but what developers truly need is fine-grained metrics. A mature evaluation system typically attributes signals to several capability dimensions, such as:

• Command execution reliability: first-pass success rate of shell commands, error rate, and interception of dangerous operations.
• Direction adjustment and error correction: after a user issues a corrective instruction, how quickly the model understands and fixes the issue, rather than repeating the same mistake.
• Long-horizon consistency: whether the model maintains goal coherence and context without drift across multi-turn tasks.
• Tool invocation accuracy: correctness of parameters, logical ordering of calls, and sensible retry behavior after failures.

These fine-grained metrics can be combined into a capability radar chart, helping developers assess a model’s fit for their specific business scenarios. For example, an automation-heavy operations context should prioritize command execution reliability and error correction; a complex software development context should emphasize long-horizon consistency.

Engineering Implementation Recommendations

For teams looking to build a similar evaluation capability, the following practical path may serve as a reference:

1. Instrumentation and anonymization: Collect signals at the session layer, but strict privacy anonymization and compliance review are mandatory to avoid storing sensitive content.
2. Signal extraction pipeline: Use a combination of rules and lightweight models to extract the four signal categories—rules ensure interpretability, models improve recall.
3. Randomized assignment: Assign models to tasks randomly at the user or session level in real traffic, ensuring the precondition for causal inference holds.
4. Metrics as a service: Encapsulate net improvement rates and fine-grained metrics into queryable services that support slicing by task type, time period, and user segment.
5. Continuous calibration: Periodically use manual spot-checks to validate the quality of automated signals, preventing signal drift from distorting evaluation results.

Conclusion

Agent evaluation is shifting from “asking users to vote” to “inferring from behavior.” The significance of this shift lies not only in cost reduction but also in making evaluation results more aligned with real-world task value. By mining confirmation, complaint, correction, and execution-error signals, and by using causal tracking to strip away confounding factors, developers can obtain a sustainable, interpretable, and attributable capability profile of a model. For teams selecting or building agents, understanding this methodology offers more long-term value than memorizing a leaderboard ranking.