Can AI Agents Really Do the Work? A Evaluation Framework for Data Tasks
Moving an AI assistant from a demo tool into production requires more than impressive-looking answers. This article presents a practical evaluation framework for data-analysis agents, covering semantic interpretation, data lineage, calculation consistency, adversarial test cases, runtime safeguards, observability, and staged production rollout. The central principle is simple: an agent should be trusted only when its results are verifiable, reproducible, and appropriately bounded.
To move an AI assistant from a demonstration tool into production, the key question is not whether it can generate polished conclusions. The real test is whether it remains reliable when faced with boundary conditions, ambiguous metric definitions, and abnormal data.
A typical failure occurs when an assistant is asked to calculate monthly “sales” for an industry. It may produce a total that is close to the correct value while misallocating large portions of the data across vehicle types, regions, or channels. An even less visible problem is mixing up manufacturer shipments, retail transactions, exports, and domestic sales. A correct total does not prove that the process was correct. If the process cannot be reproduced and verified, the agent should not be authorized to handle real business tasks.
Evaluation therefore needs to move beyond whether an agent “feels smart” and focus on whether its reliability can be demonstrated.
1. Define What “Doing the Work” Means
A data-analysis agent must get at least five things right:
- Understand business semantics.
- Locate trustworthy data.
- Perform correct calculations.
- Explain the boundaries and limitations of its results.
- Stop and ask for help when it is uncertain.
Business semantics are especially important because the same term can have different meanings across organizations. For example, “sales volume” might refer to a manufacturer’s shipments to distributors, transactions between distributors and end customers, or a broader figure that includes exports, goods in transit, pre-orders, or returns.
If an assistant guesses based on patterns in its training data instead of asking a clarifying question or consulting a metric dictionary, it has already introduced a source of systematic error.
In engineering practice, natural-language requests should be upgraded into a formal task contract. The contract should define:
- The target question
- The metric definition
- The time granularity
- The required dimensions
- Data-source priorities
- Missing-value handling
- Permitted error ranges
- The output schema
- Prohibited operations
For example:
{"metric": "sales_volume","definition": "terminal_retail_only","include": ["domestic_retail"],"exclude": ["factory_shipment", "export", "demo_car", "pre_order"],"granularity": "month","dimensions": ["region", "model_line"],"source_priority": ["governed_mart", "approved_file", "web_public"],"tolerance": {"absolute": 0, "relative": 0.0},"output": {"format": "table", "require_evidence": true}}
The value of this contract is not the format itself. It forces both parties to turn assumptions into conditions that can be checked. When a result is wrong, the failure can be traced to the semantic, data, calculation, or presentation layer.
2. Use Layered Evaluation: From Correct Totals to Correct Workflows
Many evaluations focus only on the final answer, allowing accidental correctness to hide serious problems. A better approach is to divide evaluation into four layers.
Semantic interpretation
When a prompt contains an ambiguous term, the agent must select the correct definition or ask for clarification. It should explicitly state which related concepts have been excluded.
Data lineage
Every number should be traceable to a dataset, field, filter condition, and time window. Values that cannot be traced should be marked as low-confidence rather than inserted into the main narrative.
Calculation consistency
Totals must equal the sum of their components. Year-over-year and month-over-month comparisons must use consistent baseline periods. Cross-dimensional tables must not conflict with row totals or summary tables.
A minimal validator can express the basic rule that a total inconsistent with its segments is a failure:
def reconcile(total, segments, keys, eps=1e-9):seg_sum = sum(s["value"] for s in segments)if abs(total - seg_sum) > eps:raise AssertionError("total mismatch with segment sum")for k in keys:if any(k not in s for s in segments):raise AssertionError(f"missing dimension: {k}")return True
A production system should also validate units, currencies, time zones, deduplication keys, slowly changing dimensions, and version snapshots.
It is particularly important to prevent metric drift. The same assistant might interpret “sales” as retail transactions in the morning and manufacturer shipments later in the day when given a similar prompt. One way to address this is to place metric definitions in a governed metadata layer. Prompts should reference metric IDs rather than inventing definitions dynamically.
Decision usability
An output should include limitations, anomalous samples, a review path, and recommended actions. It should not provide only a conclusion.
3. Design Adversarial Tests Instead of Running Only the Happy Path
A high-quality evaluation set should include at least three types of traps.
Near-synonym confusion
Sales, orders, deliveries, shipments, invoices, and registrations may be related without being interchangeable. Test cases should require the assistant to explain the differences.
Misleading data structures
Public websites often place wholesale, retail, and export figures in adjacent tables. Models can easily map the wrong column or apply an incorrect unit conversion. Evaluations should check column mapping and unit conversion explicitly.
False completeness
When a required dimension is missing, the assistant should reduce its coverage claim rather than fill the gap through interpolation and present the result as complete.
Each test case should record:
- The input
- The expected metric definition
- Permitted data sources
- The reference answer
- Common incorrect answers
- Deduction criteria
Scoring can use hard gates. A semantic-definition error should result in an immediate zero. Outputs without lineage should not proceed to human review. A numerical error beyond the permitted threshold should be marked as a failure.
Hard gates are often more representative of production risk than average scores. One incorrect critical field can distort procurement, inventory, or marketing decisions.
4. Embed Trusted Execution into Runtime Controls
Passing an evaluation only means that the agent is acceptable before deployment. Runtime safeguards are still required.
Data access should use governed data marts or metric services. Files and web pages should be treated as supplementary sources, and snapshots should retain their hashes.
Write operations should default to dry-run mode. Actual execution should require a second confirmation and use the principle of least privilege.
Every tool call should record its parameters, response summary, duration, software or data version, and retry reason. These records create a replayable execution trace.
For high-risk tasks, organizations can apply a two-person rule: the assistant produces a recommendation and a human approves it. Another option is to have the assistant generate only an executable plan, while a deterministic scheduler performs the actual operation. This prevents the model from directly modifying a database.
Observability metrics should also reflect business risk. Useful indicators include:
- Metric-definition accuracy
- Lineage coverage
- Total-to-segment reconciliation pass rate
- Human takeover rate
- Cost per task
- P95 latency
- Number of blocked unauthorized actions
- Number of rollbacks
Token cost alone is not a meaningful measure. The rework and business damage caused by one incorrect aggregation can far exceed the cost of model inference.
5. Organizational Rollout: From Pilot to Authorization
A three-stage rollout is a practical approach.
Stage 1: Shadow mode
The assistant produces conclusions without interacting with business systems. Its results are run in parallel with human-generated results for comparison.
Stage 2: Read-only and sandbox access
The assistant receives limited read-only and sandbox permissions. Every output must include an evidence package and a confidence level.
Stage 3: Limited write permissions
Only after the previous stages are successful should the assistant receive restricted write access. Tasks should be classified by risk:
- Low-risk formatting and aggregation tasks may be automated.
- Medium-risk tasks should undergo sampling or review.
- High-risk tasks must receive final human approval.
Teams should also maintain a metric dictionary, terminology glossary, and failure-case library. Every incident in which the total was correct but the breakdown was wrong should become a regression test.
The final standard is straightforward: does the agent reduce verifiable work, or does it create more work to explain and check its output?
A genuinely useful agent acknowledges its limits when uncertain, pauses to ask questions when definitions conflict, and provides evidence together with every important number. Only then does it begin to move from “able to answer questions” to “able to take responsibility.”
