Code Agents for Precise Data Analysis and Automated Research Report Generation

    Large language models excel at language and reasoning but lose credibility once precise numerical computation and structured data parsing enter the picture. This article explains why the code agent paradigm—where an agent writes and executes scripts instead of answering from memory—is the foundation of trustworthy data analysis and financial research reports. It walks through a typical agent workflow from task planning to final PDF, contrasts deterministic code-based plotting with probabilistic image generation, shows why percentiles and other exact calculations must be delegated to code, and lays out practical best practices for building reliable, auditable pipelines that separate language generation from numerical accuracy.

    Why LLMs Struggle to Be Trusted with Data Analysis

    When developers use large language models (LLMs) to process quantitative data or generate financial research reports, they often run into an awkward reality: the model excels at natural language understanding and logical reasoning, but its trustworthiness drops noticeably once precise numerical computation and structured table parsing come into play. It may calculate a percentile incorrectly, “invent” a conclusion that looks plausible but conflicts with the source data, or arbitrarily round off decimal places.

    The root cause lies in how LLMs generate output—they make probabilistic predictions based on training data and context rather than strictly executing mathematical rules. This leads to one basic principle: any step that requires getting the number right should not be delegated to the model’s direct generation, but to deterministic code. This is precisely where the value of the code agent paradigm lies.

    Core Idea: Have the Agent Write Code, Not Answer from Memory

    Mainstream AI agent patterns already broadly support tool calling. Unlike a simple “one question, one answer” conversational mode, the agent is cast in the role of a “programmer”: when facing a data analysis task, it does not output a conclusion directly. Instead, it first writes a Python script, executes it to read real data and perform the computation, and then organizes the results into prose.

    Three key advantages come from this paradigm:

    • Eliminating memory bias: Structured data is genuinely read from files or databases by the script, rather than guessed from the model’s “memory,” preventing data mismatch at the source.
    • Reproducible results: Running the same script on the same sample always yields identical results, making auditing and review straightforward—especially important in rigorous domains like finance and accounting.
    • Traceable logic: Every computation step is fixed in code, so when a deviation appears it can be traced to a specific line of the script, rather than a black-box “that’s just what the model answered.”

    Breaking Down the Agent Workflow: From Task Planning to Final Draft

    A mature code agent follows a workflow very similar to professional coding assistants, typically broken into these stages:

    1. Task decomposition: After receiving a request, the agent automatically generates a todo list, splitting “read data, compute factors, retrieve information, draw charts, merge into a document” into executable subtasks.
    2. Data loading: Following the list, it calls code item by item to load structured information such as tables and time series from local or cloud data sources.
    3. Information retrieval: It searches online or from a local knowledge base for relevant background material to add context to the report.
    4. Computation and plotting: It runs deterministic scripts to compute factors and calls a professional plotting library to generate charts.
    5. Draft delivery: It merges the body text with the charts and outputs a well-formatted document such as a PDF.

    An illustrative script structure looks like this (library names are described generically, focusing on logic rather than specific tools):

    1. # 1) Read structured data with a script, avoiding the model reading "from memory"
    2. data = read_table("holdings.csv")
    3. # 2) Compute factors with deterministic code; results are reproducible and auditable
    4. inventory_percentile = compute_percentile(data["inventory"])
    5. basis_percentile = compute_percentile(data["basis"])
    6. # 3) Call a professional plotting library; every data point comes from a real computation
    7. plot(data=data, chart_type="line", output="chart.png")
    8. # 4) Merge body text and charts into a research report
    9. generate_report(text=analysis, chart="chart.png", output="report.pdf")

    In practice, you only need to state the output requirements clearly in the prompt—for example, “automatically call the plotting library to generate charts, and merge the report body with the charts to produce a PDF report”—and the agent will autonomously complete the entire pipeline from computation to final draft.

    The Visualization Debate: Deterministic Plotting vs. Probabilistic Image Generation

    The chart-generation step reveals a fundamental difference between technical approaches, one that directly determines the professionalism of the result:

    • Code-based plotting (deterministic): In a professional plotting library, every point and every line is computed from concrete data. Axes, ticks, and value labels correspond strictly to the source data, making them precise and verifiable.
    • Image-generation models (probabilistic): Some cloud-based coding environments with a graphical interface generate charts through image-generation models. What they produce are probability-based pixels—visually attractive, perhaps, but the values in the chart are often not derived from real data, falling clearly short in professionalism and accuracy.

    For scenarios with extremely high demands on data accuracy, such as financial research reports and quantitative analysis, code-based plotting is the only reliable choice. A chart should not be “drawn”—it should be “computed.”

    The Numerical Precision Trap: Why You Shouldn’t Just Ask the Model for Percentiles

    A telling phenomenon: when you directly ask an LLM for “the current inventory percentile and current basis percentile,” different models often give contradictory answers, and none can guarantee accuracy. On the same dataset, for example, inventory percentiles from different agents may land anywhere in the 21%–30% range, while basis percentiles may vary from 30% to 68%—and this dispersion alone exposes the problem.

    The root cause is that an LLM does not handle exact decimal places. It merely approximates an answer based on training data and context rather than strictly following mathematical rules. Calculations like percentiles—which require sorting, interpolation, and precise definitions—are almost bound to introduce error once the model does the math in its head and bypasses code.

    The correct approach is to hand this kind of computation entirely to scripts. Given the same data and the same percentile algorithm, code produces a single, exact result; handed to the model for a direct answer, you get only a high-probability distorted estimate. This once again confirms the principle above: leave precise computation to code, and language organization to the model.

    Implementation Best Practices

    Overall, building a reliable automated data analysis agent can follow these best practices:

    • Enforce code execution: Fix every must-be-accurate step—data loading, factor computation, plotting, and drafting—as scripts, and never let the model directly produce numerical conclusions.
    • Define a clear output contract: State the output format explicitly in the prompt (e.g., “merge into a PDF report”) to reduce the agent’s room for improvisation.
    • Prefer code for plotting: When charts are needed, favor the deterministic route via a professional plotting library and avoid the numerical distortion that probabilistic image generation can introduce.
    • Establish a review mechanism: Introduce sampling checks or cross-validation with a second script for key values, so results are auditable and traceable.
    • Allocate compute resources sensibly: An agent’s code execution, online retrieval, and document generation often require sustained runtime resources. General-purpose cloud offerings such as cloud container platforms and object storage can support this, ensuring stable task execution and persistence of results.

    Ultimately, the value of a large language model lies in understanding intent, planning tasks, and organizing language, while data accuracy and chart rigor should be backed by deterministic code. Clarifying this division of labor is what makes AI agents a truly trustworthy productivity tool for data analysis and research report generation.