Reliable Reproduction of Research Indicators With AI Code Generation
This article provides a practical framework for using AI-assisted programming to reproduce technical indicators from financial research reports. It outlines a structured workflow covering requirement decomposition, mathematical specification, environment capability checking, layered validation, and audit-ready execution logging, with the goal of turning code generation into a verifiable engineering process rather than a one-shot output.
The Real Challenge: Delivery Reliability for Complex Tasks
The value of AI-assisted programming is not limited to producing syntactically correct code. For tasks such as reproducing indicators from financial research reports, the system must continuously handle document understanding, mathematical modeling, code generation, environment adaptation, and result verification. A deviation at any stage can cause the final indicator to diverge from the original research.
For example, a research report may describe a robust moving average in prose, define its outlier handling using formulas, and then present timing performance through charts. If a developer extracts only one formula, ignoring the sample window, initial values, and trading timestamps, the code may run but still cannot be considered a successful reproduction.
Therefore, instead of judging a model based on a single run, it is more useful to establish a reusable evaluation framework: Did it understand the complete definition? Did it identify target environment constraints? Did it make localized corrections based on error feedback? Did it provide a verifiable evidence trail?
Decompose the Task First, Then Generate Code
“Read the research report and implement the indicator” sounds like a single instruction, but it actually contains at least six subtasks:
- Locate the indicator definition and its context.
- Extract formulas, variables, windows, and parameters.
- Confirm input data and time-alignment rules.
- Check the capability boundaries of the target formula language.
- Implement the computation logic and handle compatibility.
- Verify the output and save the execution record.
These six subtasks should not be compressed into a single unexamined code generation pass. A more reliable approach is to define stage-wise deliverables: first produce a definition checklist, then an implementation plan, and finally executable content with a verification report.
A neutral task template might look like this:
Objective:Reproduce the specified moving-average indicator based on the provided research document.Stage 1: Extract the DefinitionList relevant page numbers, mathematical expressions, variable meanings,window length, initialization method, and missing-value handling rules.Mark any content that cannot be confirmed as "to be confirmed";do not fill in gaps on your own.Stage 2: Check the EnvironmentReview the target formula engine documentation to verify functions,loop capabilities, array capabilities, and historical-reference rules.Stage 3: Implement and VerifyGenerate indicator code, necessary comments, test samples, and execution logs.Report run status, numerical consistency, and known limitations separately.
The core purpose of such a prompt is not to add text length, but to clarify what “done” means. If the acceptance criterion is only “no errors,” the system will tend to fix syntax rather than actively prove the mathematical meaning is correct.
Document Extraction Must Preserve Mathematical Structure
Formulas in research documents are rarely plain text. Superscripts, subscripts, fractions, parentheses, absolute values, and piecewise conditions can all be lost during extraction. Tables that span pages, poor scan quality, or multi-column layouts can also misalign variables and their explanations.
Therefore, a “mathematical structure review” should follow document parsing. A record table is recommended for each key variable:
| Item | What to Confirm |
|---|---|
| Input series | Close price, returns, or other derived variables |
| Window definition | Fixed length, expanding window, or recursive state |
| Central statistic | Arithmetic mean, median, or other estimator |
| Outlier handling | Truncation, winsorization, down-weighting, or removal |
| Output method | Independent calculation or dependent on prior values |
| Boundary rules | Handling of insufficient samples, missing values, and zero denominators |
In particular, it is important to distinguish between what the original text explicitly specifies and what the engineering implementation must add. The former should be preserved faithfully; the latter should be recorded separately as implementation assumptions, not presented as conclusions of the original research.
If the original document only provides a chart without a complete definition, this should be clearly stated as an inability to reproduce precisely. Guessing formulas from the shape of a curve can at best form an approximation.
Write the Mathematical Specification First, Then Map to the Formula Language
Before code generation, the document content should be organized into a mathematical specification independent of the runtime environment.
The following uses a generic rolling median only to demonstrate the specification format; it does not represent the real definition of any specific research indicator.
Let the input sequence be (x_t) with window length (n). The current window is:
[
Wt={x{t-n+1},\ldots,x_t}
]
Sort the window elements in ascending order as (z_1,\ldots,z_n). The median is then:
[
mt=
\begin{cases}
z{(n+1)/2}, & n\text{ odd}\
(z{n/2}+z{n/2+1})/2, & n\text{ even}
\end{cases}
]
This definition looks simple but contains several constraints that code can easily obscure: the window includes the current value, an even window requires averaging the two middle elements, sorting must not alter the original time series, and the output rule for insufficient historical data must be separately agreed upon.
Once the mathematical specification is clear, the developer can judge whether the target environment can express it exactly. If the target environment lacks sorting, array, or order-statistic capabilities, the median cannot simply be replaced by the mean just because a mean function exists. The two differ in sensitivity to outliers, which changes the nature of the indicator rather than completing a compatibility fix.
When a Function Is Missing: How to Handle Compatibility Correctly
The target formula engine lacking a required function is a common obstacle in reproduction tasks. A reasonable processing order is: verify, substitute, validate, disclose.
First, check whether a mathematically equivalent native capability exists. For example, some environments lack a direct median function but may offer order-statistic operations that satisfy the requirement. Equivalence must be verified by checking interpolation methods, sort positions, and missing-value rules — not by looking at function names alone.
Second, evaluate whether self-implementation is feasible. A small fixed window may allow expansion into comparison logic, but as the window grows, code length, maintainability, and execution overhead all rise significantly. If the formula engine does not support the necessary data structures, it is unwise to force an unmaintainable expression.
Third, adjust the computation boundary when permitted by the rules. Where the rule set allows it, the indicator can be pre-computed in a general-purpose environment and then used for display or strategy validation. This is an architectural adjustment that requires clear statements about data-update frequency, timestamp mapping, and result import.
Fourth, when only an approximation is possible, the divergence must be disclosed. Approximate algorithms should be named separately, with an explanation of the direction of deviation from the original definition, test coverage, and potential impact. Producing a curve that looks similar cannot be called strict reproduction.
Close the Loop Among Reasoning, Tool Calls, and Verification
Complex tasks typically require repeatedly reading the document, checking function documentation, executing tests, and fixing errors. An effective workflow is not a one-time plan followed mechanically, but one that updates the approach each time new evidence is obtained:
Requirement decomposition → Document evidence collection → Mathematical specification → Capability verification → Code generation → Test execution → Divergence localization → Localized repair.
Throughout this process, the system should output concise decision rationales, evidence locations, and items requiring confirmation, rather than relying on unverifiable self-assessment.
For example, when a statistical function is found to be unavailable, the system should answer three questions: Which capability is missing? Does the proposed substitute remain mathematically equivalent? What tests are still needed to prove equivalence? This is far more reliable than rewriting the entire code block.
Context management is equally important. A long document should not be treated as uniformly important. Key formulas, variable tables, environment constraints, and test conclusions should be organized into structured records. This reduces interference from irrelevant content and helps maintain definitional consistency across multiple correction rounds.
Verification Needs Layers: It Cannot Stop at “Compilation Passed”
Verification is recommended in five layers.
The first layer is syntactic verification. Check variable definitions, bracket matching, comment formatting, function arguments, and output statements against the target environment’s requirements. This only proves the code has a runnable foundation.
The second layer is boundary verification. At minimum, cover: window size of one, odd windows, even windows, insufficient samples, duplicate values, missing values, and extreme values. For recursive indicators, check whether the initial state affects downstream results.
The third layer is numerical comparison. Use an independent reference implementation or small manually computable samples to compare point-by-point outputs on identical inputs. For example:
[10, 11, 12, 13, 100]
The mean is 29.2, while the median is 12. This sample quickly catches the error of writing the median as the mean. Test cases should specifically target properties that are easy to confuse, rather than choosing smooth market data.
The fourth layer is time-consistency verification. Confirm that each time point uses only data available at that time. Computing a global statistic over the full historical sample and backfilling it to earlier time points can introduce look-ahead bias. A shifted display curve must also be checked separately from the actual calculation dependency.
The fifth layer is strategy-level verification. If the indicator is used for trading research, clarify when signals generated after market close are executed, and take into account transaction costs, slippage, trading halts, and fill constraints. Correct indicator reproduction does not imply that a strategy’s returns hold; historical results also cannot substitute for out-of-sample testing.
Execution Logs Should Serve Auditing, Not Accumulate Output
A useful log records inputs, changes, and evidence — not large volumes of irrelevant text.
At minimum, the log should include:
- Document version, relevant page numbers, and location of extracted definitions;
- Data source description, time range, and processing rules;
- Mathematical specification and implementation assumptions;
- Function-support status of the target environment;
- Reason and impact scope for each code modification;
- Test input, expected result, actual result, and error;
- Unresolved issues and applicable boundaries.
The log should also avoid recording access credentials, personal information, or unauthorized sensitive data. In enterprise scenarios, traceability and data governance should be designed together, not patched in after the task is complete.
How to Objectively Evaluate the Value of AI-Assisted Programming
When evaluating such tasks, initial generation speed should not be the only metric. More meaningful indicators include: whether definition extraction is complete, whether error fixes preserve semantics, the number of human interventions, numerical verification coverage, and whether the final deliverable is maintainable.
When comparing different approaches, use the same document, the same environment constraints, and the same acceptance samples, and clearly state the statistical scope. A fast initial draft in a single demonstration only indicates preliminary efficiency on one case; it cannot be extrapolated into stable production capability.
In practice, deliverables can be divided into three states:
“Initial implementation” means code has been generated; “runtime verification passed” means the code executes in the specified environment; “reproduction verification passed” requires evidence for the definition, boundaries, and numerical results alike. Only by distinguishing these states clearly can teams avoid mistaking development progress for research conclusions.
Conclusion: Upgrade Auto-Generation into Verifiable Engineering
From research document to indicator code, the hardest part is not typing in a formula — it is preserving semantic meaning across the document, the mathematical model, and the execution environment.
AI-assisted programming can shorten the time spent on data preparation and implementation iteration, but reliable delivery still depends on clear specifications, capability verification, layered testing, and human review. What is most worth building is not a “generate and use directly” pipeline, but an engineering mechanism in which every step has supporting evidence, every modification is explainable, and every result is verifiable.
