From PDF Research Reports to Backtestable Scripts: A Full-Workflow Guide to Reproducing Quantitative Indicators with LLMs

    This article presents a complete, practical workflow for turning quantitative indicators from PDF research reports into backtestable scripts using modern LLMs and coding agents. It explains why vague "one-click" prompts tend to fail, breaks the task into four distinct capability stages—document parsing, semantic extraction, formal conversion, and a validation loop—and walks through a real case of implementing a complex moving-average timing indicator. It also covers context management, prompt engineering, common pitfalls, efficiency gains, and enterprise adoption advice. The core takeaway: the real competitive advantage comes from engineering a reusable process around the model, not from raw model capability alone.

    1. Introduction: An Undervalued, High-Value Scenario

    In financial engineering, research digitization, and quantitative trading, there is a task that looks simple but is in fact extremely labor-intensive: accurately and quickly turning the indicators in a research report into backtestable scripts.

    A mature quantitative timing report often contains dozens of indicator definitions: moving-average systems, range statistics, volatility adjustments, and signal thresholds. Each definition mixes three kinds of information:

    • Natural-language descriptions;
    • Formula screenshots embedded in charts;
    • Parameter settings tightly coupled to a specific backtesting framework.

    The traditional workflow is “read carefully, transcribe manually, translate line by line, then debug repeatedly.” The barrier is high and the process is unstable. Even an experienced quant researcher often needs several days to reproduce a single high-complexity indicator, and errors are easy to make on the details.

    By late 2025, a clear trend had emerged in the LLM space: leading vendors released new flagship models and open-sourced their weights, while the coding capabilities of open-source models collectively approached those of closed-source flagships. At the same time, intelligent coding tools evolved from “code completion engines” into “code agents” capable of planning, tool calling, and multi-step execution.

    For developers, the significance is this: these tasks are shifting from “grinding through them by hand” to “driving them with a process.”

    Using “reproducing a complex moving-average timing indicator from a research report as an executable formula script in a market terminal” as the main thread, this article breaks down a complete, deployable workflow. The views here are not tied to any specific model or platform, and all methods can be transferred to mainstream intelligent coding tools and model services.

    2. Task Decomposition: Why “One-Click Generation” Inevitably Fails

    Many developers go straight to vague instructions like “read this report and write the formulas into a script.” The result is usually a script that “looks about right”: the syntax is mostly correct, but the logic does not hold up under scrutiny.

    The reason is that this kind of task involves at least four relatively independent capability stages:

    Stage Input Output Main failure points
    Document parsing PDF report, attachments Structured text and formulas Layout misalignment, garbled formula-to-text conversion
    Semantic extraction Structured text Definitions, parameters, boundary conditions Confusing similar indicators, missing constraints
    Formal conversion Definition + script spec Script that conforms to the DSL syntax Mismatch between mathematical notation and operators
    Validation loop Script + historical data Indicator curves and signals Hallucinated functions, hidden errors

    Key conclusion: the stages closer to “human judgment” should be led by people or strongly validated; the stages closer to “mechanical conversion” are better handed off to the agent.

    So don’t treat this as a single “ask-and-answer” exchange. Design it as an engineering process of “task specification + staged acceptance.”

    3. Practical Case: Full Workflow for Reproducing a Complex Moving-Average Indicator

    3.1 Task Definition

    Assume the task is: from a securities research report on a “moving-average timing strategy,” extract the complete definition of its core ICU moving average (a composite moving average with range-statistics properties) and implement it using the formula language of a market terminal.

    Rather than pasting the real report filename and prompts, we abstract it into the following structured task specification:

    1. Task: Reproduce an indicator from a research report
    2. Inputs:
    3. - Research report file (PDF, with charts and formulas)
    4. - Sample formula script for the target terminal (with syntax conventions and comment style)
    5. Outputs:
    6. - Indicator script that conforms to the target DSL
    7. - Execution log (commands, errors, fix records)
    8. - Assumptions list (explanation and uncertainties for each definition)
    9. Constraints:
    10. - All "speculative interpretations" must be marked as assumptions
    11. - If an operator does not exist in the target language, provide an equivalent implementation first
    12. - Log every step to allow traceability

    This specification is itself an exercise in “requirements engineering.” The more specific it is, the more controllable the agent’s output.

    3.2 Stage One: Document Parsing and Semantic Extraction

    The agent first invokes a document-parsing skill to read the report. The key here is not “converting the PDF to text” but building a searchable semantic structure.

    Ask the model to output a structured summary like the following:

    1. {
    2. "indicator_name": "ICU moving average",
    3. "core_definition": "Compute a rolling median over the price series within a fixed window, then compare it against a moving average via a ratio test",
    4. "parameters": {
    5. "window": 20,
    6. "median_type": "rolling median",
    7. "threshold": [0.98, 1.02]
    8. },
    9. "edge_cases": ["limit-up/limit-down day handling", "placeholder strategy when the window is insufficient"],
    10. "assumptions": ["close price by default", "no adjustment for splits/dividends"],
    11. "source_refs": ["page 4, Figure 2", "page 6, formula (3)"]
    12. }

    The most important outputs at this stage are the assumptions and edge_cases. They determine whether the resulting script is actually usable, not just whether it “looks right.”

    3.3 Stage Two: Formal Conversion and Operator Equivalence Substitution

    This is the step most likely to go wrong in the whole process.

    Take the “median” as an example: many classic market-terminal formula languages do not have a built-in median function (for instance, MEDIAN(X,N) may not exist). If the model simply “hallucinates” a nonexistent function, the script will throw an error as soon as it is loaded into the terminal.

    The right approach is to first identify the missing operator, then provide an equivalent implementation.

    For a “rolling median,” if the terminal provides no median function, you can approximate it using a sorting approach:

    1. // Pseudocode: equivalent rolling median implementation
    2. MED_N(X, N) :=
    3. // If the terminal supports SORT or PERMUTATION-like functions, prefer those
    4. // Otherwise use the "k-th smallest value" approach
    5. k = FLOOR(N/2) + 1
    6. return KTH_SMALL(X, N, k)

    If even the “k-th smallest value” is unsupported, fall back to an approximation:

    • Use a moving average to approximate the median (acceptable error, but it must be flagged in the assumptions);
    • Or switch to a “deviation of price from the moving average” signal, bypassing the median entirely.

    The engineering point of this step is: don’t chase “mathematical equivalence”; instead, aim for “implementable within the target language’s operator set, with controllable error.”

    3.4 Stage Three: Iterative Correction—One Failure Is Not the End

    Scripts generated by an agent often have three typical kinds of problems:

    1. Syntax level: nonexistent function names, wrong parameter order, variable scope violations;
    2. Semantic level: writing “median” as “mean,” or writing an “interval test” as a “threshold test”;
    3. Boundary level: insufficient window, division by zero, null-value propagation.

    The correction strategy is not “regenerate” but “fix at the exact point.” An effective approach is:

    • Have the agent read the terminal’s error messages (rather than guessing);
    • Modify only the smallest faulty fragment and preserve the rest of the logic;
    • Re-execute after each change until it passes;
    • If two attempts fail in a row, ask the agent to produce a “root-cause hypothesis” rather than continuing to change things blindly.

    One developer’s real experience: the first version of the script failed to load because it used a nonexistent median function, and it passed testing after a single correction. The key is not “getting it right the first time” but “failing fast and fixing precisely.”

    3.5 Stage Four: The Validation Loop

    A script that runs is not necessarily a correct indicator. At least three layers of validation are recommended:

    Layer Method Pass criterion
    Syntax Terminal load No errors; indicator displays
    Numerical Sample comparison Deviation < 1e-6 vs. manual/reference implementation
    Behavioral Signal consistency Signal trigger points match the report’s description

    The behavioral layer is the most easily overlooked. Many scripts “run,” but their signals fire 3 days earlier than the report describes—usually because of a different window-alignment method or split/dividend adjustment handling.

    4. Context Management: An Engineering Variable More Important Than “How Strong the Model Is”

    In complex tasks, context strategy often determines success more than model choice does.

    4.1 Don’t Stuff the Entire Report In at Once

    A common mistake is feeding the agent the full text of a hundred-page PDF. The results:

    • Key information gets diluted;
    • The model’s attention scatters, and edge conditions get missed;
    • Output length is limited, and the script gets truncated.

    Recommended approach:

    1. First have the agent produce a “table-of-contents-level summary” to locate the relevant sections;
    2. Feed only the relevant sections (usually 5–15 pages) as context;
    3. Extract the formula portions separately into structured text;
    4. Interact in rounds, solving one sub-problem per round.

    4.2 Use “Skills + Tools” Instead of “Ultra-Long Context”

    Modern code agents support mounting skills (such as document parsing, code execution, and terminal interaction). Compared with cramming all information into the context, a more robust approach is:

    • Let the agent call the document-parsing skill on demand, rather than pre-reading the full text;
    • Let the agent execute scripts in a real terminal, using errors to feed corrections;
    • Treat the “execution log” as persistent memory, rather than relying on conversation history.

    This is a typical “interleaved thinking” paradigm: think—act—observe—think again. It is more robust than one-shot long-chain reasoning, and closer to how a human engineer actually works.

    4.3 Longer Context Isn’t Always Better

    Over the past couple of years, the industry has seen a trend: flagship models no longer blindly pile on ultra-long context, but instead return to windows of 128K or even shorter, while improving output stability and resistance to distraction.

    For developers, this sends a clear signal: context length should match the task’s needs, not be as large as possible. A short, precise context is often more effective than a long, messy one.

    5. Prompt Engineering: Making the Agent “Work Like an Engineer”

    5.1 A Structured Prompt Template

    The following template can be reused directly:

    1. # Role
    2. You are a quantitative engineer familiar with financial time series and market-terminal formula languages.
    3. # Task
    4. Extract the complete definition of [indicator name] from the given research report and reproduce it as a script executable in [target terminal].
    5. # Inputs
    6. 1. Research report file (PDF)
    7. 2. Sample formula script for the target terminal (with syntax conventions and comment style)
    8. # Constraints
    9. 1. All inferences must be marked as "assumptions"
    10. 2. If an operator does not exist, provide an equivalent implementation first, then code
    11. 3. Log every step to the execution log
    12. 4. Output must include: indicator script, assumptions list, validation suggestions
    13. # Acceptance
    14. - Script loads in the terminal without errors
    15. - Sample 10 data points and compare with manual calculations; deviation < 1e-6
    16. - Assumptions list contains at least 3 items

    5.2 Three High-Leverage Techniques

    1. Provide a “positive example” rather than a “description”: A sample script that conforms to the spec is more effective than ten lines of syntax explanation.
    2. Force an “assumptions list”: This compels the model to expose its uncertainty and avoid “making mistakes confidently.”
    3. Require a “traceable log”: Recording each command and error makes manual auditing and further correction easier.

    6. Common Pitfalls and Avoidance Strategies

    Pitfall Symptom Avoidance strategy
    Hallucinated functions Calling nonexistent operators Check the operator list first; provide an equivalent if missing
    Semantic drift Writing median as mean Force a definition summary for manual confirmation
    Missing boundaries Outputting NaN when the window is insufficient State the placeholder strategy explicitly in the assumptions
    Overfitting Parameters look “perfect” on the backtest Validate on out-of-sample data
    Missing logs Unable to trace after an error Auto-log every execution

    7. Efficiency Comparison: From “Days” to “One Hour”

    One developer’s real experience is representative: reproducing a high-complexity report indicator used to take a researcher with a financial-engineering background one to two weeks; with a new-generation code agent, a first reproduction can be completed within an hour.

    The efficiency gain doesn’t come from a single model capability, but from the stacking of three factors:

    1. Improved model capability: planning ability, tool-calling stability, and output consistency have all improved markedly;
    2. Upgraded interaction paradigm: shifting from “Q&A” to “task delegation + staged acceptance”;
    3. Accumulated engineering process: document parsing, operator mapping, and the validation loop have formed reusable templates.

    This also explains a phenomenon: with the same model, different developers can achieve results that differ by several times. The gap usually lies not in the model, but in process design.

    8. Enterprise Adoption Advice

    For teams hoping to promote similar capabilities internally, focus on three points:

    8.1 Build an “Indicator Library + Assumptions Library”

    Turn each reproduced indicator script, assumptions list, and validation result into structured assets. This has more long-term value than merely accumulating prompts.

    8.2 Embed “Human Review” into the Process, Not as an Afterthought

    Setting a human-confirmation checkpoint at the semantic-extraction stage costs far less than reworking during validation.

    8.3 Support the Engineering Chain with General Cloud Capabilities

    General cloud products—model services, object storage (for report and script versions), log services (for capturing execution traces), and monitoring/alerting (for tracking task success rates)—can be combined to support the workflow above, without locking into any specific vendor.

    9. Conclusion: The Competition in Coding Capability Is Shifting to “Engineering Capability”

    Back to the original question: how much has the coding capability of these models actually improved?

    From an engineering perspective, the answer isn’t a benchmark score, but three perceivable changes:

    • Faced with a complex task, the model proactively plans rather than piling up code;
    • Faced with a missing operator, it offers an equivalent approach rather than hallucinating;
    • After an error, it makes precise fixes based on the error message rather than blindly rewriting.

    For developers, the real opportunity is this: package model capability into a reusable engineering process. Whoever standardizes “document parsing—semantic extraction—formal conversion—validation loop” will gain the upper hand in the next round of the efficiency competition.

    If you’re working on report reproduction, indicator migration, or automatic script generation, start with a “structured task specification”—not with a vague prompt.