How LLM Agents Automatically Generate Illustrated Quantitative Research Reports

    This article explains why trustworthy quantitative research reports need a "model plus code" approach rather than letting a large language model compute numbers directly. It covers the three key advantages of code-driven analysis, the typical workflow of an Agent that decomposes tasks, runs code, generates charts, and self-checks its output, and the reasons quantile results can differ significantly across models. It also compares three automation paths, outlines the security, validation, and data-governance requirements for production, and closes with a practical deployment checklist. The goal is to keep computation in code and leave orchestration and writing to the model, so efficiency and credibility can both be achieved.

    How LLM Agents Automatically Generate Illustrated Quantitative Research Reports

    Part 1: Why Quantitative Reports Need a “Model + Code” Dual Engine

    1.1 The Core Value of a Quantitative Report Is Trust

    A solid quantitative daily or weekly report usually contains four elements:

    • Data: structured data such as inventory, basis, price, volume, and open interest;
    • Metrics: statistical results such as quantiles, year-over-year and month-over-month comparisons, and correlation coefficients;
    • Charts: trend curves, quantile distributions, basis structure diagrams, and so on;
    • Conclusions: interpretation of metrics, changes in risk, and tracking recommendations.

    Whether a report is accepted depends first on whether its numbers and charts can be reproduced. No matter how fluent the prose is, a single mismatched quantile undermines the credibility of the entire report.

    1.2 Why Letting the Model “Compute in Its Head” Is Unreliable

    A large language model is fundamentally a probabilistic generative model. At each step it “guesses” the most likely next token based on training data and context, and it does not perform strict mathematical operations. On numerical tasks, this shows up as three risks:

    1. Approximate output: for complex calculations, the model may produce a value that is close but not exact, with no guarantee of correctness after the decimal point;
    2. Context drift: in long-report tasks, values are easily influenced by the surrounding text or by “common values” seen in the training corpus;
    3. Nondeterministic results: running the same input again may produce slightly different output.

    A more insidious problem is that a wrong number is presented in exactly the same format as a correct one. Without automatic validation, it is very hard to catch by manual reading alone.

    1.3 Let the Model Write Code and Let the Code Compute

    A more robust approach is to have the model automatically generate and execute analysis scripts that read tables and compute metrics. Here the model plays the role of a “programmer”: understanding requirements, writing code, calling tools, and checking outputs. The actual numerical computation is done by script libraries in a deterministic execution environment, which avoids probabilistic memory bias.

    Part 2: Three Key Advantages of a Code-Driven Approach

    2.1 More Reliable Structured Data Parsing

    Business tables often contain multi-row headers, merged cells, mixed units, and missing or anomalous values. Reading the data purely from context tends to lose or misinterpret information during conversion. A script, in contrast, can:

    • locate columns precisely by header hierarchy;
    • perform type conversion, missing-value handling, and anomaly checks;
    • output the number of parsed rows, field types, and sample intervals as verifiable intermediate artifacts.

    These “verifiable intermediate artifacts” are something probabilistic parsing cannot provide.

    2.2 Higher Chart Precision and Professionalism

    In Agent mode, charts are drawn step by step by plotting scripts based on the data. Every point in a chart is derived from specific data, and the axes, ticks, and legend correspond one-to-one with the source data, so readers can trace back layer by layer through “chart → calculation logic → raw data.”

    By contrast, an image generation model is essentially a “probabilistic pixel generator.” It learns the distribution of images and does not understand the true values of the data. When used to draw statistical charts, the bar heights, line trends, and axis ticks only look right visually; the actual numbers are usually untrustworthy and cannot meet the precision requirements of quantitative reports.

    2.3 Reusable, Accumulable, and Deliverable

    The generated scripts are reusable analysis assets:

    • after changing the cycle or parameters, you only need to adjust the configuration to rerun the entire pipeline;
    • recurring analysis logic can be solidified into modules and gradually accumulated into a team’s analysis toolkit;
    • scripts can call document generation tools to merge prose and figures into a report, delivering an illustrated PDF directly.

    Part 3: The Typical Workflow of an Agent

    Modern mainstream large models generally support an Agent mode. In a conversation, the model no longer only outputs text; it can autonomously plan steps, call tools, read execution results, and adjust its next action accordingly. Taking “generate a commodity inventory and basis daily report” as an example, the typical flow is:

    1. Task decomposition: after receiving a goal, first generate a to-do list, such as “read inventory data → read basis data → compute quantiles → draw figures → write the text → merge and generate PDF”;
    2. Data reading: write and execute a script to read local tables, performing calculations on given intervals and given fields;
    3. Information retrieval: when news or market context is needed, first search the web, or retrieve relevant material from a local information library, citing the source and time;
    4. Figure generation: call plotting tools according to the established analysis logic to produce chart files;
    5. Document generation: merge the text and figures into a report document;
    6. Self-check and revision: verify that files were generated, that metrics are within a reasonable range, and that charts match the text, then fix problems automatically.

    Step 6 determines whether an Agent can deliver reliably. An Agent with self-checking ability is closer to a “trustworthy assistant,” while an Agent that only produces output in one pass is more like an “intern who needs human review.”

    A typical prompt looks like this (illustrative only):

    1. Task: Generate today's commodity daily report based on local inventory and basis data.
    2. Requirements:
    3. 1. Use linear interpolation for all quantiles, sorted in ascending order, with percentages kept to one decimal place;
    4. 2. All values must be computed by code; direct mental estimation or experience-based guessing is prohibited;
    5. 3. Automatically call the visualization tool to draw figures, and merge the text and figures into a PDF report;
    6. 4. Before delivery, verify: the sample interval and data row count used for metric calculation.

    Part 4: Why Quantile Results Differ Significantly

    4.1 The Phenomenon

    Running several different LLM Agents on the same batch of data to generate the daily report produced noticeably different quantile results (the table below is illustrative test data, used only to show the phenomenon):

    Run method Current inventory quantile Current basis quantile
    Method A 26.3% 55.0%
    Method B 21.2% 55.2%
    Method C 27.4% 68.0%
    Method D ~30% ~30%

    The two metrics behave differently:

    • The gap in the current inventory quantile is about 6 percentage points;
    • For the current basis quantile, the relative gap among the first three methods is under 1 percentage point, while Method D’s “close to 30%” differs from the others by about 25 percentage points;
    • Method D’s two results are not only too close to each other; the round-number expression “about 30%” also shows traces of direct estimation.

    4.2 Common Causes

    1. Code not actually invoked: in long contexts, the model may “mentally compute” using language ability directly, producing approximate values;
    2. No unique quantile definition: algorithms such as linear interpolation, nearest-rank, and midpoint can differ by several percentage points when the sample size is limited and the distribution is uneven;
    3. Different sorting and percentile conventions: ascending or descending, percentage or decimal, and “which end to compute from” all affect the value;
    4. Inconsistent sample scope: whether missing values, extremes, or suspended-trading data are excluded, whether a rolling window is used instead of the full sample, and how rounding is done all introduce bias;
    5. Unit and rounding errors: mixing ten-thousand-ton and thousand-ton units, or rounding too early in intermediate steps;
    6. Context input bias: when data enters the context only as text fragments, the model may read the wrong row or omit rows.

    4.3 Improvement Suggestions

    1. Fix the calculation convention: specify the sort direction, interpolation method, sample window, and number of decimals in the prompt or configuration, and have the code read the configuration directly, eliminating ambiguity in interpretation;
    2. Always let code compute metrics: require all values to be output by scripts, and attach the sample interval and sample size to the results;
    3. Independent verification: recompute key metrics with a separately written piece of code and compare the error against the expected range, to prevent the same code from being “consistently wrong”;
    4. Keep records: include the source script, version, and run log of key metrics in the report appendix or metadata.
    1. # Example: differences between two common quantile methods (generic form)
    2. values = [10, 12, 14, 16, 18] # sample size of 5, sparse distribution
    3. pos = (len(values) - 1) * 0.30 # linear interpolation: the 15% position falls between index 0 and 1
    4. q_linear = values[0] + (values[1] - values[0]) * pos # result: 10.6
    5. idx = int(len(values) * 0.30) # nearest-rank / floor: index = 1
    6. q_nearest = values[idx] # result: 12
    7. print(f"Linear interpolation: {q_linear}, floor: {q_nearest}, difference: {abs(q_linear - q_nearest):.2f}")

    This example shows that with a sparse sample, the “value-taking method” alone can make results differ by 1.4 under the same stated convention. So “using some library’s default algorithm” does not mean “the convention is correct.” Production environments should specify and lock the convention explicitly over the long term.

    Part 5: Comparing Three Automation Approaches

    In practice, letting a large model participate in generating illustrated content usually follows one of three paths:

    Dimension Direct model text Image generation model Code-driven visualization
    Numerical accuracy Probabilistic bias Not applicable Guaranteed by computation code
    Chart readability No charts, or text description only Visually close to charts, numbers untrustworthy Accurate scales and legend
    Reproducibility Weak Weak, differs across generations Strong, same input yields same output
    Auditability Weak Weak Strong, traceable to scripts and data
    Use cases Qualitative overviews, summaries Illustrative, decorative figures Formal deliverables such as reports

    Code-driven visualization is best suited to formal delivery scenarios such as quantitative reports. Image generation can be used for covers or conceptual illustrations, but it should not take on the job of expressing data.

    Part 6: From a One-Off Experiment to Production

    6.1 Security Isolation for Code Execution

    Letting the model write and execute code is essentially running unreviewed instructions in the environment, with potential risks including:

    • the model mistakenly generating dangerous commands, such as deleting files or accessing credentials;
    • “prompt injection” from malicious data or external web pages that lures the model into unauthorized operations;
    • vulnerabilities or resource exhaustion in the generated code.

    Recommended safeguards:

    1. Run code in a container or isolated sandbox, with limits on CPU, memory, and runtime;
    2. Mount only the required data in the working directory, keep the input side read-only where possible, and write output to a separate directory;
    3. Prohibit access to credentials and sensitive system paths, and disable inbound network access by default; when online retrieval is truly needed, use a controlled egress and restrict the reachable scope;
    4. Pin dependency versions, and prefer a reviewed dependency list;
    5. Record complete execution logs for later traceability.

    6.2 Automated Validation and Scheduling

    1. Front-load validation rules: before delivery, automatically check that key metrics fall within a reasonable range (for example, quantiles between 0 and 100%), that the number of charts matches the references in the text, and that the document opens normally;
    2. Scheduled triggering: based on the data update frequency, generate reports daily or weekly, with automatic retry and alerting on failure;
    3. Versioned artifacts: tag each generated report, chart, and script with a version identifier to support traceability.

    6.3 Data Governance and Traceability

    1. Establish a unified source and field-definition specification for the data;
    2. Record data intervals, calculation conventions, script versions, model versions, and generation time in the report metadata;
    3. For external information cited during retrieval, note the source and time so readers can verify it.

    6.4 Integration with Cloud Capabilities

    When this pipeline moves into production, it can rely on general cloud capabilities to improve stability:

    • use object storage to centrally store raw data, charts, and report artifacts;
    • use a container platform to run the code execution environment, ensuring isolation and elasticity;
    • use a message queue to decouple tasks and absorb traffic spikes;
    • use logging, monitoring, and alerting services to track task success rate, duration, and anomalies.

    For example, on Baidu AI Cloud, developers can combine general cloud products such as object storage, containers, and log monitoring to build the runtime and operations foundation for this kind of automated pipeline.

    Part 7: Deployment Checklist

    1. Are all values computed by code and accompanied by their sample interval?
    2. Is the quantile convention explicitly stated and locked over the long term?
    3. Have key metrics been independently recomputed?
    4. Are figures generated from real data?
    5. Does code execution run in an isolated sandbox with time and resource limits?
    6. Are externally retrieved sources cited, and is untrusted content filtered?
    7. Does the delivery document pass automated validation?
    8. Can artifacts be traced back to the script, data, and model versions?

    Part 8: Conclusion

    The value of Agent mode is not that the model becomes “better at arithmetic,” but that the model learns to work like an engineer: decomposing tasks, writing code, calling tools, and verifying results. In quantitative report scenarios, leaving “computation” to code and “orchestration and expression” to the model is what allows both efficiency and credibility. For metrics that easily cause disagreement, such as quantiles, explicitly declaring the calculation convention and insisting on independent verification is the key to making results reproducible and auditable.