Large Language Model Agents for Quantitative Analysis and Financial Report Generation
This article explains how to build reliable LLM Agent workflows for quantitative analysis and financial report generation. It focuses on assigning planning and orchestration to the model while using deterministic scripts for calculations, data-driven libraries for chart rendering, and validation modules for accuracy, reproducibility, and auditability. It also covers task orchestration, report generation, practical limitations, and an implementation checklist.
Large Language Model Agents for Quantitative Analysis and Financial Report Generation
In quantitative analysis and financial reporting, the value of an LLM Agent is not limited to its ability to write text. More importantly, it must be able to break complex tasks into executable steps: reading structured data, generating and running scripts, calling visualization libraries, retrieving supporting information, and combining the results into a report with text and charts.
At the same time, allowing a model to calculate numbers mentally, generate charts directly, or produce conclusions without verification introduces risks to accuracy and reproducibility. A more reliable engineering approach is to let the model handle planning and orchestration, deterministic scripts handle computation and rendering, and validation modules provide safeguards.
1. Core Principle: Let Scripts Calculate and Models Explain
Common quantitative analysis tasks include percentile calculations, basis percentiles, year-over-year and month-over-month changes, ranking, and grouped statistics. LLMs are good at understanding requirements and organizing language, but they are not precision calculators. A model may infer decimal values from training data and context, producing results that appear reasonable but do not follow the underlying mathematical rules.
This risk becomes more pronounced in multi-turn conversations, long contexts, and complex tables, where approximate outputs can develop into hidden errors.
An Agent pipeline should therefore establish a clear division of responsibilities:
- The model interprets objectives, generates task lists, selects tools, and explains results.
- Scripts read tables, execute calculations, and produce intermediate outputs.
- Visualization libraries render charts from defined data points.
- Document-processing modules combine content into an archivable report.
- Validation modules compare key metrics, check for outliers, and maintain audit logs.
The workflow can be abstracted as follows:
task_list = plan(goal)data = read_table(input_path)clean_data = validate_and_clean(data)stats = compute_quantile(clean_data, target_columns)chart = render_chart(stats, chart_spec)text = summarize(stats, context)report = merge_document(text, chart)save_pdf(report, output_path)audit_log(stats, script_version, input_hash)
The key point is that compute_quantile is executed by a script. The model receives the calculated result and explains it. This preserves the Agent’s automation capabilities while preventing it from freely generating critical numerical values.
2. Why Mental Calculation by Models Is Unreliable: An Anonymous Comparison
In one anonymous experiment, several Agent approaches were asked to generate an analysis report from the same dataset and extract inventory percentiles and basis percentiles. The approaches are labeled A, B, C, and D only to illustrate the difference between model-generated estimates and script-based calculations. They are not intended as a product evaluation or selection guide.
| Approach | Inventory Percentile | Basis Percentile | Observation |
|---|---|---|---|
| Approach A | 26.3% | 55.0% | Deviated from the script results |
| Approach B | 21.2% | 55.2% | Showed a relatively large percentile deviation |
| Approach C | 27.4% | 68.0% | Showed a significant deviation in the basis percentile |
| Approach D | Approximately 30% | Approximately 30% | Used a relatively coarse output granularity |
The common problem was not the models’ ability to express themselves. Rather, they tended to produce approximate values, and the number of decimal places could vary depending on the context.
The correct approach is to implement percentile calculations in a fixed script and specify in the prompt that all numerical values must come from script output. The model should be prohibited from calculating values independently. Reports should also retain the calculation methodology, sample period, and script version. Only in this way can a financial report remain reproducible.
3. Agent Task Orchestration: From To-Do Lists to Auditable Pipelines
A mature Agent workflow typically includes the following stages:
- Task decomposition: Break “generate a daily report” into tasks such as reading data, calculating metrics, retrieving relevant information, generating charts, merging documents, and validating the output.
- Tool invocation: Provide controlled tools for reading tables, executing scripts, rendering charts, and merging documents. Limit accessible directories and executable operations.
- Persistence of intermediate artifacts: Save calculation results, chart files, and retrieval summaries as intermediate artifacts for traceability.
- Result validation: Recalculate or cross-check key metrics. If an anomaly is detected, stop the workflow and request human review.
- Report generation: Combine text, charts, data methodology, and risk disclosures into a general document format such as PDF.
Prompts should define an explicit output contract, for example:
All numerical values must be calculated by scripts and written to a result file. Charts must be rendered from calculated results. The report must list data sources, calculation methodology, generation time, and any anomalies.
This is more important than simply instructing the model to “write professionally.”
4. Chart and Report Generation: Data-Driven Rather Than Pixel-Based
Analytical charts should be rendered from specific data points, with every point and line derived from calculated results. This differs from image-generation models that create pixels probabilistically. Data-driven charts are suitable for presenting percentiles, distributions, trends, and comparisons with precision and professional consistency. Image-generation approaches are better suited to illustrative visuals and should not be used to carry key numerical information.
In practice, an Agent can be instructed to call a general-purpose visualization library and generate bar charts, line charts, or distribution plots from predefined metric templates.
The following details require particular attention:
- Chart axes, units, and sample periods must match the underlying data.
- Legends and annotations should be generated by scripts to prevent mismatches caused by manually written labels.
- Key values in the report and the chart data should come from the same result file.
- The output document should retain the script version, input file hash, and generation logs.
5. Practical Experience and Limitations: Automation Does Not Eliminate Validation
In practice, the Agent approach provides a coherent workflow. It can automatically create a task list, run scripts to read data, retrieve supporting information, generate charts, and merge the results into a report. For recurring daily reports, weekly reports, and standardized financial research documents, this can significantly reduce manual work.
However, the limitations are equally clear:
- The model may still introduce uncalculated numbers while explaining results in natural language. It should therefore be required to reference the result file.
- When resource quotas are exhausted or tools fail, iterative revisions may be restricted. Checkpoint-based execution, resumable workflows, and caching should be considered.
- If charts are generated as images rather than from data, their precision and professional quality may be inadequate. Data-driven rendering should be preferred.
- Multiple tool calls increase audit complexity. Inputs, outputs, versions, and failure reasons must be recorded.
6. Implementation Checklist
When implementing quantitative analysis and financial report generation, consider the following checklist:
- Script all numerical calculations and prohibit mental calculation by the model.
- Require prompts to produce structured result files, and make the report reference those files.
- Add parameter validation, exception handling, and unit tests to scripts.
- Generate charts from data and preserve intermediate datasets.
- Run cross-checks on key metrics before merging the report.
- Retain audit logs to support reproducibility and human review.
- When anonymously evaluating different approaches, compare workflow capabilities and sources of error rather than promoting or ranking specific products.
Overall, LLM Agents are well suited to serve as analyst assistants and workflow orchestrators, but they should not replace deterministic calculation modules. Separating calculation, chart rendering, document merging, and validation into a clear pipeline makes it possible to improve efficiency while preserving the accuracy and traceability of financial reports.
