Causal Tracking: Evaluating AI Agents with Real-World Session Data

Why Agent Evaluation Is Far Harder Than Chat Evaluation

Over the past few years, the dominant paradigm for evaluating large language models has been “pairwise comparison + human preference voting”: users see responses from two models side by side, pick the better one, and a paired-comparison statistical model converts the votes into rankings. This paradigm works well for chat, but applying it directly to agent scenarios runs into three structural problems:

1. Tasks cannot be reproduced in parallel. Agent tasks typically involve real environment operations: reading and writing files, executing commands, calling APIs. Once two models each run through the task, the environment state has changed, and the results are no longer strictly comparable.
2. Sessions are too long, and users are unwilling to vote. A single agent task may involve dozens of tool calls stretched over hours. Users rarely have the patience to complete a “blind comparison” after it finishes.
3. Preference signals and reliability signals are misaligned. Chat votes measure “which answer is more likable,” while agent tasks are about “whether the job got done”: did the command run, was the goal achieved, how many rework cycles were needed. This information is hidden in the session process, not in a single click at the end.

As a result, the industry has gradually converged on a different evaluation route: instead of relying on human voting, use real users’ agent sessions themselves as the evaluation data source, automatically mining quality signals from sessions and then applying statistical inference to convert them into comparable model capability scores. These evaluations are usually presented as “agent leaderboards” covering multiple task domains such as work, coding, and chat, with rankings based on tens of thousands of real, long-horizon task sessions from around the world.

Four Key Signals Mined from Real Sessions

The most valuable thing in real sessions is the behavioral footprint users leave when they “vote with their feet.” Common key signals fall into four categories, differing in confidence level and coverage:

1. Explicit task completion confirmation. The user explicitly says something like “that works,” “exactly what I wanted,” or “let’s ship it.” These signals are semantically clear and low in ambiguity—the strongest evidence of a positive outcome.

2. Praise and complaints. Emotionally charged feedback such as “so efficient” or “you completely misunderstood my requirements.” Compared with confirmations, sentiment signals are denser but noisier, requiring context-aware sentiment analysis and denoising.

3. Correction instructions. The user interrupts the model and asks it to change direction—for example, “don’t touch that file” or “I said refactor, not rewrite.” Correction instructions are the core material for measuring a model’s steerability: after receiving a correction, whether the model accurately understands the intent and quickly gets back on track directly reflects human-AI collaboration efficiency.

4. Shell command execution errors. Tool calls returning non-zero exit codes, command not found, permission denied, missing dependencies, and so on. This is a fully objective, environment-level signal that requires no subjective judgment, yet it directly characterizes a model’s reliability in generating commands and its ability to self-verify.

Schematically, the signal extraction pipeline can be understood like this:

# Pseudocode: extract key signals from an agent session
def extract_signals(session):
    return {
        # User explicitly confirms task completion
        "confirmed": has_confirmation(session.user_msgs),
        # Sentiment feedback: praise / complaints
        "praise":    sentiment(session.user_msgs, kind="positive"),
        "complaint": sentiment(session.user_msgs, kind="negative"),
        # Number and positions of user corrections
        "corrections": count_corrections(session.user_msgs),
        # Tool call failures (e.g., non-zero exit codes)
        "shell_errors": count_tool_failures(session.tool_calls),
    }

These signals share two advantages: first, they are naturally real—they come from real users solving real problems, with no “evaluation-for-evaluation’s-sake” distribution shift; second, they are hard to game—fabricating massive volumes of high-quality sessions costs far more than overfitting to a static benchmark.

Causal Tracking: A Net Improvement Rate That Turns Correlation into Causation

Getting the signals is only the first step. If you directly compute “each model’s average session success rate,” you immediately fall into the confounder trap:

• Different models attract different task difficulty distributions—stronger models may be assigned more high-difficulty tasks;
• User populations, prompt styles, and context lengths vary enormously;
• The mix of task types (coding/office/chat) differs across models, potentially triggering Simpson’s paradox.

Randomized inference methods like “causal tracking” are designed to solve exactly this problem. The core idea can be summarized as: within statistical inference, treat “which model handled this session” as approximately a random assignment, and estimate each model’s net treatment effect against a virtual “average model” baseline.

Expressed intuitively as a formula:

Net improvement rate ≈ (the model’s metric performance on comparable sessions − the average performance across all models) / average performance

Using illustrative data: suppose a model achieves an overall net improvement rate of +5.1% on long-horizon agent tasks. This means that after removing confounders such as task difficulty and user habits, its overall performance is about 5.1% higher than the “average model.” If its previous generation scored only +0.9%, the generational gain is about 4.2 percentage points. Beyond the overall figure, breakdowns typically include command execution reliability (first-pass command success rate, recovery after errors) and steerability (improvement after receiving correction instructions), so teams can match dimensions to their needs.

Compared with simple mean comparisons, randomized inference brings three benefits:

1. Interpretable causal meaning: the score answers “if I switch to this model, how much will my session quality change,” not “this model got lucky with easy tasks”;
2. Quantified uncertainty: randomized inference naturally yields effect distributions and significance levels, so you can judge whether rank differences fall within noise;
3. Dynamic comparability: continuously updated with real traffic, allowing rolling comparison of new and old models under the same framework.

How Developers Should Read Agent Leaderboards

When facing agent evaluation leaderboards, it helps to build a three-layer reading habit:

• Look at breakdowns first, then the overall score. The overall ranking reflects the general level, but selection decisions depend on your workload: teams focused on terminal automation should prioritize command execution reliability; scenarios with frequent human-AI collaboration should prioritize steerability. A model ranked 10th overall but strong in coding may suit your scenario better than one ranked 5th.
• Track the catch-up pace of open-weight models. A clear recent trend is that leading open-weight models have entered the top tier on real agent tasks, with the gap to closed-source APIs narrowing significantly while remaining competitive across work, coding, and chat domains. For teams with private deployment, data compliance, or cost optimization requirements, the old assumption that “open-weight solutions won’t work” needs re-evaluation.
• Treat the leaderboard as a prior, and your own retest as the conclusion. Leaderboards reflect the task distribution of the “global average user”; your business distribution will inevitably differ. The right approach: use the leaderboard to narrow the candidate set, then run small-scale gray-release comparisons with your own traffic, and let business metrics make the final call.

Practical Advice for Building Your Own Agent Evaluation System

If your team builds agent products, you can fully adopt this methodology. Key steps include:

1. Instrument all sessions. Record user messages, model responses, tool calls and their return values (including exit codes), and session termination reasons, forming replayable session logs. Common cloud platform logging services and object storage can support the ingestion and retrieval of such data.
2. Layered signal extraction. Use rule matching for high-confidence signals (exit codes, explicit confirmations) and model-based judgment for ambiguous signals (sentiment, intent), tagging each signal with a confidence level to keep low-quality signals from polluting the statistics.
3. Gray-release routing and control groups. When launching a new model, split real traffic proportionally to form a natural control group; make sure traffic allocation is randomized, otherwise the premise of causal inference is broken.
4. Metric dashboards and regression alerts. Bring net improvement rate, command error rate, correction success rate, and similar metrics into monitoring dashboards, with automatic regression after every model or prompt change to prevent “improving scenario A while quietly degrading scenario B.”
5. Privacy and compliance up front. Session data masking, user consent, and retention period management must be addressed at the architecture design stage, not patched afterwards.

It is worth noting that model services on major cloud platforms (such as Baidu AI Cloud) generally ship with generic evaluation, monitoring, and logging capabilities. Teams can quickly assemble the above pipeline from these generic components and focus their energy on signal design and metric definition.

Looking Ahead

Agent evaluation is undergoing a paradigm shift: from static benchmarks to real traffic, from human preference to environment feedback, from single-turn scoring to session-level causal inference. For developers, the value of understanding this methodology goes beyond “reading leaderboards”—it is a transferable engineering capability. When your agent product faces real users, the same signal-mining and causal inference framework becomes the most reliable compass for continuous iteration.