Agent-Based Data Analysis and Automated Research Report Generation in Practice

    This article explains how to build an AI agent workflow for data analysis and research report generation. It covers why the language model should hand calculations to deterministic code, the six stages of the workflow, and practical methods for data validation, percentile calculation, chart generation, PDF report assembly, and controlling numerical errors.

    1. Background: Why Agents Use Code for Data Analysis

    Research reports, business analysis, and operational reviews often require analysts to read tables, calculate metrics, draw charts, and write conclusions. Over the past two years, more teams have tried to have AI agents handle most of this workflow. An agent can understand natural language requests, break a task into steps, and call external tools to carry out specific actions.

    A language model generates text by predicting the next token based on probability. This mechanism works well for understanding intent, organizing text, and summarizing key points. It is less reliable for reading large tables cell by cell, performing arithmetic with many decimal places, or strictly following mathematical definitions. When a model is asked to give the percentile of a metric directly, the result is often an estimate based on context rather than a calculated value.

    A safer approach divides the work. The model handles planning, writing code, and drafting text. Deterministic programs handle reading data, running calculations, and generating charts. Program outputs are reproducible and verifiable, and the model cites those results instead of filling in numbers on its own.

    2. Overall Architecture

    A typical agent workflow for research report generation includes six stages:

    1. Task decomposition: Generate a task list from the analysis goal, specifying the input and output of each step.
    2. Data loading: Load table files and check field structure, data types, and missing values.
    3. Metric calculation: Use scripts to compute statistics and save the results as structured files.
    4. Information retrieval: Collect background information from online sources or a local knowledge base, and record the sources.
    5. Chart generation: Draw charts from the calculation results, taking chart data directly from the output files of the previous step.
    6. Report assembly: Combine conclusion paragraphs, data tables, and charts using a template, then export the report as a PDF file.

    The stages pass intermediate results through files instead of relying on the model’s context memory. If one step fails, you can locate the problem and rerun only that step, without running the entire workflow again.

    3. Key Implementation Points

    3.1 Data Loading and Validation

    After loading data, run a structural check first. This prevents later calculations from silently running on bad data. Common checks include whether all required columns are present, whether dates parse correctly, whether numeric columns contain non-numeric characters, and whether the proportion of missing values exceeds a set threshold.

    1. def load_and_check(path, required_columns):
    2. rows = read_table(path) # Read the table as a list of records (illustrative)
    3. header = rows[0].keys()
    4. missing = [c for c in required_columns if c not in header]
    5. if missing:
    6. raise ValueError(f"Missing required columns: {missing}")
    7. bad = [r for r in rows if r["value"] in (None, "")]
    8. if len(bad) / len(rows) > 0.05:
    9. raise ValueError("Missing values exceed 5%. Stopping analysis.")
    10. return rows

    The example above treats validation as a hard requirement. When data fails a check, the process stops and writes the reason to the log. This is more reliable than producing a report that looks complete but is built on bad data.

    3.2 Metric Calculation: Precise Definition of Percentiles

    A percentile describes where a current value sits within a historical distribution. Take the historical percentile of the current value over a past period. The most direct definition is the proportion of historical samples that are less than or equal to the current value.

    1. def percentile_rank(history, current):
    2. n = len(history)
    3. not_greater = sum(1 for v in history if v <= current)
    4. return round(not_greater / n * 100, 1)

    Percentiles can be calculated in several ways. Linear interpolation and direct counting can give different results near sample boundaries, so the same data can produce different numbers under different methods. The report should therefore state the method, the sample period, and the sample size, and give the formula in the methodology section.

    The program should also control decimal places. The reporting layer should use one shared formatting function. This keeps the same metric from showing different values in different paragraphs because of inconsistent rounding.

    3.3 Chart Generation: Values Must Come from Calculation Results

    The value of a chart lies in showing the data faithfully. The plotting script should read the output files of the calculation step directly instead of hard-coding numbers. Every data point should trace back to a specific row in the original table. The script should also generate the chart title, axis units, and data source.

    This keeps charts consistent with the text. When data is updated, you only need to rerun the calculation and plotting pipeline. No one has to edit the numbers in the chart by hand.

    3.4 Report Assembly and PDF Export

    The assembly stage typically uses templates. Numbers in the body text are placeholders that the program fills in from the calculation results. Images are inserted by file path, and tables are rendered from data structures. Finally, a document rendering tool exports the report as a PDF.

    Templates make the division of labor clear. The model writes only the wording for trend descriptions and risk notes, while the program inserts the specific values. Checking the report then means comparing each placeholder with its data source, not reviewing every word by hand.

    4. Numerical Precision and Hallucination Control

    Language models do not perform strict arithmetic when they generate numbers. The numbers they output may not match the original data, and the error may not be visible in the wording. To control this risk, follow these rules:

    • All values must be calculated by scripts and written to result files. The model may reference only fields that already exist in those files.
    • Check that conclusion sentences match the data fields. For example, if the text says a value rose, the corresponding month-over-month change must be positive.
    • Keep intermediate values for key metrics so they can be recalculated and verified later.
    • Do not let the model add specific numbers or dates for historical events when no data source supports them.

    Floating-point numbers in computers are also approximations. When comparing two values for equality, use a tolerance, such as checking whether the absolute difference is smaller than a very small threshold, instead of using a direct equality test.

    5. Quality Checklist

    Before release, check the following:

    • Data: The sample period, sample size, and handling of missing values are documented.
    • Calculation: The metric definition matches the formula, and boundary values have test cases.
    • Charts: Axis units, legends, and data sources are complete, and chart values match the calculation output.
    • Text: Every number can be found in the result files, and the direction of each conclusion matches the data change.
    • Versioning: The report records the generation time, code version, and checksums of the input files so results can be reproduced.

    6. Scope and Practical Recommendations

    This approach suits scenarios with a large amount of structured data, relatively fixed report templates, and regular output. Examples include daily business briefings, inventory and price monitoring, and financial metric tracking. For analyses that depend on subjective judgment or lack reliable data sources, an agent can only produce a draft. Final conclusions still need human review.

    In practice, start with a single metric. Wrap its calculation logic in testable functions, and confirm the results are correct before gradually connecting task planning and report generation. Keep logs at every step, so that when a problem occurs, you can tell whether the error came from the data, the code, or the text.