How LLM Agents Improve Quantitative Analysis and Research Report Generation

    This article examines how LLM agents can support quantitative analysis and financial research report generation by planning tasks, executing code, generating charts, and validating outputs. It explains why deterministic computation should be handled by programs rather than language inference, outlines a practical workflow from data reading to PDF output, compares models across data processing, code execution, mathematical accuracy, chart quality, report structure, and auditability, and provides production-oriented recommendations for secure, reproducible, and reviewable agent-based analysis.

    1. From “Answering Questions” to “Completing Tasks”: How LLM Agents Change the Workflow

    Traditional large language models mainly generate text, code, or analytical conclusions through conversation. After a user asks a question, the model organizes an answer directly from the context. This approach is suitable for explaining concepts, drafting initial content, and generating simple code. However, in quantitative analysis, financial data processing, and similar scenarios, relying solely on natural language reasoning carries clear risks.

    Financial analysis often involves multiple tables, many fields, different time periods, and precise decimal calculations. For example, when calculating inventory percentiles, basis percentiles, period-over-period changes, or historical rankings, a model needs to identify the correct data range, handle missing values, standardize date formats, and perform deterministic mathematical operations. If any one of these steps is inaccurate, the final conclusion may be wrong.

    The core change in the agent pattern is that the model no longer only “generates an answer directly.” Instead, it can break a goal into tasks, call a code execution environment, file-reading tools, web search tools, or document-processing tools, and then summarize the results. At this point, the LLM is closer to an analytical assistant that can write programs, invoke tools, and check results.

    It is important to note that agents do not automatically eliminate errors. They only add the ability for the model to perform external operations. The reliability of the final result still depends on data sources, code logic, tool permissions, exception handling, result validation, and human review mechanisms.

    2. Why Quantitative Code Analysis Is Well Suited to Agents

    In quantitative analysis, the most reliable approach is usually not to let the model “guess” the result directly. Instead, the model should read the data first, generate and run a script, and then write the analytical conclusion based on the program output.

    A typical workflow can be divided into the following steps:

    1. Read the original file and identify worksheets, field names, data types, and time ranges.
    2. Standardize dates, numeric values, and missing values.
    3. Filter the sample according to the task requirements.
    4. Perform calculations using explicit statistical formulas.
    5. Generate intermediate result tables.
    6. Create trend charts, distribution charts, or comparison charts.
    7. Check the reasonableness of the calculation results.
    8. Organize the results into a daily report, weekly report, or research report.

    For example, percentile calculation should not rely on the model’s language inference. It should be performed by a statistical program. An illustrative code snippet is shown below:

    1. import pandas as pd
    2. df = pd.read_excel("market_data.xlsx")
    3. df["date"] = pd.to_datetime(df["date"], errors="coerce")
    4. df["inventory"] = pd.to_numeric(df["inventory"], errors="coerce")
    5. df["basis"] = pd.to_numeric(df["basis"], errors="coerce")
    6. df = df.dropna(subset=["date", "inventory", "basis"])
    7. current_inventory = df.iloc[-1]["inventory"]
    8. current_basis = df.iloc[-1]["basis"]
    9. inventory_percentile = (
    10. (df["inventory"] <= current_inventory).mean() * 100
    11. )
    12. basis_percentile = (
    13. (df["basis"] <= current_basis).mean() * 100
    14. )
    15. print(f"Inventory percentile: {inventory_percentile:.2f}%")
    16. print(f"Basis percentile: {basis_percentile:.2f}%")

    The value of this kind of code is that every result can be traced back to specific data and formulas. Even if the final conclusion is problematic, it is possible to determine whether the issue came from data reading, filtering logic, or statistical definitions.

    By contrast, if a model is asked directly, “What historical percentile is the current inventory in?”, it may make an approximate inference based on context or even provide a plausible number that cannot be verified. For financial tasks, such a result cannot be used directly as a basis for decision-making.

    3. Automatic Chart Generation: Code-Based Plotting Is Better for Analytical Tasks

    Another important capability of agents in data analysis is automatic visualization. For financial research reports and operational daily reports, charts are not merely decorative. They are used to show trends, compare differences, and support judgment.

    If charts are generated by code, every point, line, and bar in the chart corresponds to specific data. Users can modify parameters to redraw the chart and can also check the data filtering range and calculation logic. For example:

    1. import matplotlib.pyplot as plt
    2. df = df.sort_values("date")
    3. plt.figure(figsize=(10, 5))
    4. plt.plot(df["date"], df["inventory"], label="Inventory")
    5. plt.title("Inventory Trend")
    6. plt.xlabel("Date")
    7. plt.ylabel("Value")
    8. plt.grid(alpha=0.3)
    9. plt.legend()
    10. plt.tight_layout()
    11. plt.savefig("inventory_trend.png", dpi=200)
    12. plt.close()

    This approach has three clear advantages.

    First, results are reproducible. The same data and the same code can generate consistent charts.

    Second, modification costs are low. Users can adjust the time window, colors, ticks, titles, and legends without regenerating the entire image.

    Third, the meaning of the data is clearer. The chart is not a visual effect generated from probabilities; it is determined jointly by statistical data and plotting instructions.

    Of course, code-based plotting does not guarantee absolute correctness. Common issues include incorrect date sorting, mixed units, unreasonable axis scaling, missing legends, and unhandled outliers. Therefore, after an agent generates a chart, it is still necessary to check whether the chart is consistent with the original data, especially the direction of time, units, and the location of extreme values.

    4. The Complete Task Chain for Agent-Generated Research Reports

    A relatively mature automated research report workflow can usually be divided into four stages: planning, execution, validation, and output.

    1. Planning Stage

    The agent first converts the user’s natural language requirements into a task list, for example:

    • Read tables in the specified directory.
    • Identify core indicators and their historical ranges.
    • Calculate current values, change rates, and historical percentiles.
    • Search for relevant industry information.
    • Generate trend charts and comparison charts.
    • Write conclusions and risk disclosures.
    • Combine text and charts into a PDF file.

    The purpose of the task list is to prevent the model from missing steps and to help users understand the current execution progress.

    2. Execution Stage

    The agent can call Python scripts to process tables, use search tools to obtain public information, and generate charts based on analysis results. For multiple indicators, it is better to establish a separate calculation process for each indicator rather than concentrating all logic in a single unauditable code block.

    At this stage, the model may also dynamically adjust subsequent actions based on the output of the previous step. For example, if field names do not match expectations, it can first inspect the headers. If the data contains empty values, it can first calculate the missing ratio and then decide whether to remove, fill, or retain them.

    3. Validation Stage

    Validation is the most easily overlooked yet most critical step in an automated research report workflow. It should include at least the following checks:

    • Whether the file was read successfully.
    • Whether the number of data rows and the date range meet expectations.
    • Whether key fields exist.
    • Whether numeric fields were incorrectly parsed as text.
    • Whether percentiles fall between 0% and 100%.
    • Whether the data used in charts is consistent with the conclusions.
    • Whether dates, sources, and facts in the information are traceable.
    • Whether the numbers in the research report match the program output.

    For important results, the agent can be asked to generate both a “calculation detail table” and a “conclusion summary,” and then automatically compare the two.

    4. Output Stage

    After completing the analysis, the agent can combine text, tables, and charts into a PDF document. To improve readability, it is recommended to standardize heading levels, fonts, page margins, chart numbering, and data source notes.

    A research report should not contain only conclusions. It should also clearly state:

    • Data sources.
    • Statistical period.
    • Calculation definitions.
    • Indicator definitions.
    • Outlier handling methods.
    • Possible limitations.
    • The time range to which the conclusions apply.

    Only then will the generated document be closer to professional material that can be reviewed, archived, and reused.

    5. How to Compare Different Models on Agent Tasks

    In practical testing, the same dataset, the same set of prompts, and the same output requirements can be used to compare multiple models. However, the comparison should not only examine whether the article is fluent. It should also evaluate the following dimensions:

    Data Processing Capability

    Whether the model can correctly identify fields, dates, and data types, and whether it misses worksheets or misreads column names.

    Code Execution Capability

    Whether the scripts generated by the model can run correctly, whether they include exception handling, and whether the model can locate problems after execution fails.

    Mathematical Accuracy

    Whether the model strictly follows program output, whether it arbitrarily changes decimal places, and whether it presents approximate results as precise conclusions.

    Chart Quality

    Whether charts accurately reflect the data, whether axes, units, legends, and titles are complete, and whether there is visual misrepresentation.

    Report Structure

    Whether the report includes data notes, core conclusions, driving factors, risk disclosures, and follow-up indicators to monitor.

    Auditability

    Whether users can obtain intermediate data, execution logs, code, and cited sources rather than only the final text.

    In one set of example tests, different models may produce different outputs for the current inventory percentile and the current basis percentile. For example, some results may be close to 20%, 30%, or a higher range. Such differences cannot simply be attributed to whether a model is “smart” or “not smart.” They may also come from differences in sample range, percentile definitions, missing value handling, duplicate value handling, and the selection of the row containing the current value.

    Therefore, the correct comparison method is not to record only the final number. It is to fix the statistical definitions and require the model to output the complete calculation process.

    6. Why Large Models Are Not Suitable for Direct Precise Calculation

    A large language model is essentially generating text sequences with high probability based on context. It can explain formulas, write programs, and organize conclusions well, but it is not inherently equivalent to a strict mathematical calculator.

    Directly asking a model for an answer is especially risky in the following scenarios:

    • Continuous calculations with multiple decimal places.
    • Statistics on large tables.
    • Multi-condition filtering.
    • Comparisons across multiple date ranges.
    • Percentile and ranking calculations.
    • Unit conversion.
    • Financial data that requires item-by-item verification.

    A more reliable division of labor is as follows: the model understands the task, designs the algorithm, writes code, and explains results; the program reads data, performs calculations, and outputs precise values; humans confirm business definitions, check anomalies, and make final decisions.

    7. Practical Recommendations for Production Environments

    If agents are to be used for daily data analysis or enterprise research report generation, the following constraints are recommended:

    1. Limit file and network access scope to avoid unnecessary data leakage.
    2. Set timeouts, permissions, and resource limits for executed scripts.
    3. Save code, input file summaries, run logs, and output results.
    4. Important conclusions must cite original data or calculation details.
    5. Conduct human review before external publication.
    6. Write indicator definitions and statistical definitions into fixed templates.
    7. Preserve error information for failed tasks to avoid silently generating incorrect reports.
    8. Verify the sources of information summaries generated by the model.
    9. Check charts in the PDF page by page.
    10. Periodically use fixed samples for regression testing.

    Overall, the real value of agents is not to replace analysts in completing all work. It is to automate data reading, code writing, repetitive calculations, chart creation, and document formatting. For quantitative analysis and financial research report scenarios, the most reliable solution is not to let the model “guess the answer” directly. It is to let the model work around executable code and verifiable data.

    When task planning, program calculation, result validation, and human review form a closed loop, a large model can evolve from a text generation tool into an intelligent assistant capable of executing analytical workflows.