Automating Data Analysis Reports with AI Agents: Why Calculations Must Run in Code, Not in the Model
This article explains how to move data analysis from asking a large language model for a single number to having it write and execute code. It covers the difference between Agent mode and plain text generation, why visualizations drawn by a plotting library are traceable while image-generation models only approximate values, a reusable five-step workflow for producing an automated report, the prompt pattern that triggers full text-and-chart PDF output, and the central pitfall of "approximate numbers" from models. It closes with practical engineering advice: separating calculation from generation, pinning inputs for reproducibility, validating sensitive metrics, defining delivery specs in the prompt, and choosing a stable runtime. Use this article if you want to build reliable AI-driven reporting while keeping numbers accurate.
1. Data Analysis Is Shifting from “Give Me an Answer” to “Write Code and Run It”
In the past, we handed a table to a large language model and asked a direct question: “What is the inventory quantile?” It looks efficient, but the risk is high. A language model predicts the next token; it does not actually compute anything. It looks for the most probable value in its training corpus and in the current context. As soon as a metric requires a precise definition — a specific quantile, the decimal places of a ratio — the model tends to produce a number that merely looks plausible.
The safer approach is to put the AI in the role of a programmer: generate a runnable script, load the data into it, let code perform the calculation, and then organize the results, charts, and conclusions into a document. The key difference is that code does not tolerate “close enough.” Sorting, taking a quantile, and summing are deterministic operations — the same data and the same logic produce the same result no matter how many times you run them. This eliminates the probabilistic recall errors and hallucinations that are the most painful problem in structured data work.
2. The Agent Model: From Answering to Executing
Most mainstream LLMs now offer an Agent mode, and that is what makes “let the AI write code and run it” practical. Outside Agent mode, a model can only emit text or a code snippet that nothing will execute. In Agent mode, the model can plan tasks, call tools, read intermediate results, and keep going — breaking one complex goal into multiple steps and carrying them out on its own.
Visualization is the clearest illustration. When an analysis report needs figures, the Agent calls a data visualization library — a plotting library from a general-purpose programming language — to draw them. This is fundamentally different from “an image-generation model makes a nice-looking picture.” Every point, line, and axis tick produced by a visualization library comes from an actual calculation over the data, so it is exact, traceable, and reproducible. An image-generation model outputs probabilistic pixels: it may look polished, but it cannot guarantee the values are right. For professional data analysis, precision and traceability matter far more than appearance. In the same way, an Agent can generate formatted documents such as PDFs and tables directly from scripts, combining prose, data, and charts into a single illustrated deliverable.
3. Anatomy of the Workflow: How an Automated Report Is Produced
Unfold a typical “generate a report automatically” task and a clear, reusable pipeline appears:
- Task planning — the Agent creates a task list from the overall goal, breaking “produce the daily report” into subtasks such as read the data, compute the metrics, draw the charts, write the conclusions, and export the PDF.
- Data loading — scripts run as planned to read the data files (for example, tables), with supplemental information retrieved from the web or from a local knowledge base when needed.
- Metric calculation — quantiles, ratios, and period-over-period comparisons are computed with code.
- Visualization — the visualization library draws line, bar, and distribution charts that follow the logic of the metric definitions.
- Document assembly and export — the written conclusions and the generated charts are merged and exported to a final format such as PDF.
The reason this flow is stable is that every step is code-driven and every result can be inspected. The AI handles orchestration and judgment; the deterministic program does the actual calculating and drawing.
4. Prompt Design: One Sentence to Get a Text-and-Chart Report
An Agent’s behavior depends heavily on how the output requirement is stated. To have it finish the analysis, add charts, and produce a PDF in one pass, you only need to define the delivery standard in the prompt:
Analyze the data and produce a daily report; automatically call a visualization library to generate the figures; merge the report body with the generated figures and export a single PDF report.
Once the deliverable is clearly defined, the Agent fills in the intermediate steps itself — reading the table, computing, charting, laying out, and exporting — without step-by-step human guidance. The lesson: in Agent mode, writing a good task specification matters more than writing a good single instruction.
5. The Core Pitfall: LLMs and Approximate Numbers
This is the point to watch most closely in any AI data analysis practice: a large language model does not strictly follow mathematical rules, and the values it returns are usually approximations.
Take quantiles, a common metric. Ask different models to “mentally” compute the inventory quantile and the basis quantile of the same dataset and you will see clear disagreement — one model returns roughly 26%, another 21%, another 68%, and some return two values that are almost identical. The data is the same and the correct answer is unique, yet the model outputs are scattered across a fairly wide range. The reason is simple: the model is predicting, not computing. It never really sorts the data or locates the quantile position; it produces a number that looks right according to its training distribution and the context.
A truly accurate quantile has to be computed by deterministic code, as sketched below:
# Compute a quantile (illustrative, language-agnostic)sorted_data = sort(raw_values) # sort the data firstposition = ceil(p × len(raw_values)) # locate the quantile positionquantile = sorted_data[position - 1] # take the corresponding element
The conclusion is straightforward: wherever precise values are involved — quantiles, ratios, decimal places, period-over-period figures — the calculation must go to code and never to the model’s mental arithmetic. The right division of labor is to let the model understand the task, write the logic, and interpret the results, while code performs the exact computation. Handled this way, you get both the flexibility of AI and the rigor of a program.
6. Engineering Notes for Production
A few engineering details matter once you move this into a real production environment:
- Separate computation from generation. Give exact calculation to deterministic scripts and understanding and phrasing to the model; do not ask the model to do the numerical work it is bad at.
- Guarantee reproducibility. Pin the data version, the script logic, and the runtime environment so the same input yields the same output, which makes auditing and traceback easier.
- Validate key metrics. For sensitive values such as quantiles and ratios, add assertions or cross-checks — recomputing with a second algorithm, for instance — and block anomalous results.
- Define the delivery spec clearly. State the output format, fields, and file types in the prompt to reduce back-and-forth rework.
- Rely on a stable runtime. For long-running, batch report generation, the Agent’s computation and rendering work can be deployed to a cloud platform, using common capabilities such as containerized compute, object storage, logging, and monitoring with alerting to keep the pipeline stable and controllable (general-purpose cloud services, for example those offered by Baidu AI Cloud, can serve as the runtime foundation for this kind of task).
One more note on how these agents are hosted. Besides local environments, “cloud-hosted agent machines with a graphical interface” have appeared, where users drive the Agent directly in a browser without wiring up a model API themselves. These are ready to use and easy to start with, but they bring new considerations — for example, if their figures depend on an image-generation model rather than a visualization library, the accuracy and professional quality of the charts suffer. Limits on free quota and on task context length can also get in the way of completing long workflows. Choose based on how much data precision you need and how complex the task is, rather than simply going for whatever runs automatically.
7. Summary
Agent mode genuinely changes how data analysis is done: the model stops being “the person who hands you an answer” and becomes “the programmer who writes your code, runs your pipeline, and produces your report.” It is good at understanding tasks, orchestrating steps, and interpreting results, while precise numeric computation, reproducible charts, and properly formatted documents are guaranteed by deterministic code. Keep the most important rule in mind: anything that requires an exact number goes to code, not to a model’s guess. Get that right and you have the flexibility of AI together with the rigor of a program.
