Preserving Numerical Accuracy in AI Agent-Generated Research Reports
Report automation is moving from prompt-and-response workflows to AI agents that decompose tasks, call tools, and assemble deliverables. This article explains why precise calculations should be kept out of the generative model, proposes a workflow based on deterministic scripts, persisted results, and programmatic charting, and provides prompt design rules and delivery checks for keeping every percentage, data point, and conclusion traceable to the same source data.
Data analysis and research report writing are shifting from “a human writes the prompt and the model answers directly” to “an agent decomposes the task, calls tools, and then assembles the deliverable.” On the surface, a single conversation can ingest a table, calculate inventory and basis percentiles, plot trends, and export a document with text and charts. This is efficient. What really determines whether the output can be used in investment research, risk control, or operating reviews is not whether the prose is fluent, but whether every percentage, chart coordinate, and concluding sentence can be traced back to the same source data. If precise calculations are left to the generative model itself, the result is often only a plausible guess drawn from its training distribution, not a value computed by a formula.
Why Letting a Model Do Mental Arithmetic Is Unreliable
The generation mechanism of a large language model predicts the next token from context. It is good at summarizing definitions, organizing paragraphs, and surfacing anomalous clues, but it does not have a strict arithmetic engine built in. When it faces multi-column tables, rolling windows, percentiles, adjusted prices, or missing-value imputation, common problems include unstable rounding, inconsistent definitions for the same metric across sections, vague statements such as “about thirty percent,” and skipped rows or misaligned columns when the context is long. These errors do not always look like obvious nonsense. That makes them more dangerous: the report may have complete formatting and professional terminology, and a reviewer who does not independently verify the numbers against the table may treat an approximation as a fact.
A more reliable division of labor is therefore this: the model is responsible for understanding the business question, choosing definitions, writing and orchestrating scripts, and explaining results. All counting, sorting, percentile calculation, differencing, and aggregation are performed by scripts in a deterministic runtime. For the same input and the same code path, a script produces a reproducible result. There is no probabilistic recall that “remembers it this time and mixes it up the next time.” In this setup, the agent should act as a programmer and orchestrator, not as a calculator.
Recommended Task Decomposition
Do not compress a research report request into a single round of free-form generation. First create an inspectable to-do list, then execute it item by item. A typical sequence is as follows.
- Clarify inputs and definitions. Confirm the data file location, time range, field meanings, missing-value rules, percentile algorithm (including whether endpoints are included and which interpolation method is used), and comparison baseline. If the definitions are not written down before calculation begins, neither the later charts nor the conclusions can be audited.
- Parse the table read-only. Load the table with a script, and print the row and column counts, field types, duplicate keys, and time coverage. Complete data quality checks before calculating metrics.
- Compute and persist intermediate results. Write percentiles, basis, inventory changes, and other outputs as structured results, such as a table or key-value store. Keep enough decimal places and record the functions and parameters used. Numbers in the report body must be read from these results; the model must not paraphrase a “close enough” number separately.
- Generate charts programmatically. Every point, line, and annotation is driven by the values produced in the previous step. Do not use a text-to-image model to “draw something that looks like a research chart.” Pixel generation cannot guarantee that the coordinates match the table, and a professional reader will quickly notice when the axis, labels, and series do not line up.
- Keep retrieved background separate from numbers. Industry news and announcement summaries can enrich interpretation, but they must not overwrite or replace calculated metrics. Store citations and calculations in separate places.
- Synthesize the document and run self-checks. Combine the title, methodology notes, metric table, charts, and risk disclosures into a document with stable layout. Before export, use a script to check that every key number in the body can be found in the results table and that the length of each chart series matches the sample count.
The value of the to-do list is that failures can be localized. A table-loading failure, a missing field, a charting error, and a document-merge failure should be retried separately, rather than rewriting an entire section and quietly changing the numbers.
Programmatic Charts vs. Generative Visuals
A programmatic plotting library draws markers and axes from data arrays. Change one source data point, and the chart changes according to the same code logic. A generative visual samples pixels from a text description. It can produce an attractive layout, but it cannot promise that “this point on the chart equals this row in the table.” In research reports, daily reports, and quantitative appendices, charts are evidence, not decoration. Programmatic plotting should therefore be the default.
In practice, follow these rules: write the metric, frequency, and sample interval in the chart title; keep axis units consistent with the table; flag anomalies according to data rules rather than describing them only in prose; and include the plotting script and data version in the appendix. Then, when a reader questions a particular point, the team can rerun the script instead of arguing with the model about what it “probably meant at the time.”
The illustrative calculation and plotting flow can remain short. What matters is that results are reproducible and charts come from those results:
# Definition: in-sample linear-interpolation percentile; for structural illustration only.# The algorithm must be confirmed by the business.def percentile(values, q):xs = sorted(v for v in values if v is not None)if not xs:raise ValueError("empty series")pos = (len(xs) - 1) * qlo = int(pos)hi = min(lo + 1, len(xs) - 1)w = pos - loreturn xs[lo] * (1 - w) + xs[hi] * wstats = {"inventory_pctl": percentile(inventory, 0.5),"basis_pctl": percentile(basis, 0.5),}# Write stats to the results file, then have the charting program read the same file.
The example illustrates only the control flow: the percentile function is explicit, the results are persisted, and the charting program reads the same result file. In production, replace this with numerical and plotting libraries that the team has already validated, and add handling for time zones, adjusted prices, and trading calendars. Do not treat the return value of the illustrative function as a market conclusion for any real instrument.
Prompting Agents to Use Tools, Not Intuition
An effective prompt should constrain behavior rather than pile up adjectives. It should specify four things clearly.
- Output contract: The agent must first provide a to-do list, then execute it. The final document must include definitions, a metric table, programmatic charts, and a brief interpretation.
- Tool boundary: Numerical values may come only from script stdout or result files. If a script has not succeeded, the agent must not invent percentages.
- Precision requirements: Key ratios must retain the agreed number of decimal places. A phrase such as “close to thirty percent” must not replace an exact value unless the result file itself contains an interval.
- Failure strategy: If fields are missing, the sample is too small, or definitions conflict, stop generating conclusions and list the gaps.
The output requirement can state explicitly: “Use scripts to calculate metrics, generate charts programmatically, and then merge the body and charts into a document.” The purpose of this instruction is to move the agent from “writing an essay directly” to “writing code, running it, and citing the output.” If the runtime cannot execute code, the agent should downgrade to producing only a calculation plan for verification and clearly state that the numbers have not been verified. It should not pretend that the calculation has been completed.
When Results Disagree
When different agents, different prompts, or different script versions report different values for “the current inventory percentile” or “the current basis percentile,” that does not automatically show which one is smarter. The first checks should be: Are they using the same data snapshot, the same timestamp, the same missing-value handling, and the same percentile definition? If a model bypasses the script and fills in a percentage based on language intuition, the result is not comparable, even if the wording is professional. In review, place “can this be reproduced?” ahead of “does the prose sound like an analyst wrote it?”
At a minimum, the logs should retain input file checksums, script versions, key parameters, the raw result table, and the final document. If a definition later needs to be adjusted, the team can rerun the process instead of trying to reconstruct it from a chat history.
Quality Gate and Scope
Before delivery, use a short checklist to verify the following: the data range matches the title; each metric can be traced back item by item to the result table; each chart series matches the row count in the table; citations do not rewrite calculated results; and the document contains no untraceable “model-memory number.” When the sample is extremely small, field meanings are unclear, or the rules themselves are disputed, the agent should state the uncertainty rather than hide the gap in longer paragraphs.
Agents are useful for accelerating repetitive table reading, charting, layout, and first-pass interpretation. They do not replace the analyst accountable for signing off on definitions and conclusions. Keep calculations in deterministic programs, keep explanation in the model, and use human review as the final check. That is the precision practice worth preserving in automated research reporting.
