Reproducing Research Report Indicators with AI Coding Tools: A Practical Workflow

    This article walks through the practical process of turning sell-side quantitative research reports into backtestable trading indicators using AI coding tools. It explains why this task is a good fit for language models, breaks down the five capabilities needed for reliable reproduction, and lays out a repeatable workflow—from preparing materials and prompt design to compilation, targeted fixes, and numerical alignment. It also covers common blockers and how to handle them, a set of metrics for objectively evaluating "good enough," guidance on cost and compliance, and the boundaries that still demand human sign-off.

    1. The Problem: The “Last Mile” Between a Research Report and a Backtestable Indicator

    Quantitative research reports from sell-side firms carry a high density of value, but they are written to be read by humans: narrative arguments, formatted equations, charts, footnotes, and appendix tables of parameters. Turning one into an indicator that compiles in a charting platform and produces historical signals requires making a series of implicit conventions explicit:

    • Data conventions: whether prices are adjusted for splits/dividends, whether to use closing price or average price, and how to handle suspended or limit-up/limit-down days;
    • Parameter definitions: whether periods are counted in trading days or calendar days, and whether a window “includes the current day” or is “lagged by one day”;
    • Syntax differences: the report uses mathematical notation, while the charting platform’s formula language is a separate, restricted domain-specific language (DSL) with an incomplete function set;
    • Silent failures: the code compiles and a chart renders, but the values are systematically off—these are the hardest issues to catch.

    The traditional path demands both a quantitative background and fluency in formula syntax, concentrating the labor cost in “translation plus debugging.” What is genuinely scarce here is not compute, but the patient time needed to run a debugging loop against an unfamiliar DSL to completion.

    2. Why This Task Is a Good Fit for AI Coding Tools

    To decide whether a task can be handed to AI, the key question is whether it has the property of a “closed, verifiable loop.” Research report indicator reproduction meets this on four points:

    1. Inputs and outputs are both text. PDF extraction, terminology mapping, and formula generation never require cross-modal alignment, which naturally suits language models.
    2. Acceptance criteria are objective. Whether the code compiles, and whether sampled K-line indicator values match a hand calculation, are binary judgments that need no subjective review.
    3. Domain corpus is abundant. The formula languages of mainstream charting platforms have over two decades of public accumulation; function names, syntax conventions, and common patterns all fall within knowledge the model has seen.
    4. Failures localize quickly. A compile error reports a line number and a cause directly, so iteration costs far less than writing business code.

    By contrast, tasks with vague requirements and subjectively judged acceptance—for example, “design a good-looking frontend page”—are far more prone to repeated rework.

    3. Capability Breakdown: What a Successful Reproduction Actually Requires

    Making this task reliable requires a model to hold five capabilities at once; none can be missing:

    • Document extraction and structuring: locating the paragraph that defines the indicator in the PDF, correctly parsing equations from tables, footnotes, and images, and distinguishing the “core definition” from “strategy discussion.”
    • Task planning and proactive thinking: for multi-step tasks, decompose before acting—read the document → extract the definition → map it to the target DSL → generate → self-check undefined behavior → output an explanation. A common shortcoming of earlier models in such scenarios was “write first, think never,” which leaves complex tasks without global planning, drives up the error rate, and burns developers’ time on bug fixes.
    • Interleaved reasoning and tool calls: weaving reasoning steps in between file reads and writes during a long task, rather than dumping everything at once and patching afterward. This directly affects success rates in multi-file, multi-round-fix scenarios.
    • Effective context convergence: more context is not always better. Holding the full report plus format samples plus several rounds of fix history (on the order of a hundred thousand tokens) is enough; an overly long context dilutes the attention weight on key constraints.
    • Low hallucination and output stability: the most critical of all. When the target function library lacks a needed function, the correct behavior is to explicitly declare the gap and offer alternatives, not to invent a nonexistent function name that fails at compile time.

    4. A Reproducible Workflow

    Step 0: Prepare the Materials

    Three inputs set the ceiling: the report PDF (the original source), a format sample file (defining comment style, parameter conventions, output line colors, and so on), and a glossary (aligning the report’s Chinese phrasing with the formula variables). The value of the sample file is often underestimated—it is the least-effort way to turn “the style I want” from natural language into a hard constraint.

    Step 1: Prompt Design

    A workable prompt skeleton is shown below (replace the specific file names with actual values):

    1. @<report file>.pdf
    2. Use the document-parsing skill to extract the definition of <indicator name> from the
    3. document, and reproduce it in <target platform> formula syntax.
    4. Follow @<sample file>.txt for the formula format; use a // prefix for all comments.
    5. Save the execution process and key decisions to a log.

    Four constraints are indispensable: explicit input, explicit output format, explicit comment convention, and an explicit requirement to keep a trail. The last one matters especially—an execution log lets you trace back “which paragraph it read, why it chose this function, and where it made a compromise,” which is the foundation for later auditing and second-pass fixes.

    Step 2: Compile and Verify Manually

    Paste the generated code into the charting platform and compile. This step cannot be automated, nor does it need to be: the compiler’s error messages are themselves the most precise feedback signal.

    Step 3: Targeted Fixes, Not Task Restarts

    When you find a capability gap, throw the problem back as an incremental instruction rather than opening a new conversation. A typical example is that ranking/quantile functions (such as the median) are not built into some classic formula languages. Here, do not accept the model’s first “workaround”; instead, ask for a comparison of options and their costs:

    1. The current formula language does not provide a MEDIAN function. Give three alternative
    2. approaches, each with its precision cost and implementation complexity, then choose the
    3. most accurate one and rewrite, noting the approximation in a comment.

    Step 4: Numerical Alignment

    Sample 20–30 trading days, compute the indicator values independently with an external script, and compare them point by point against the platform’s output, setting the allowed error to around 1e-6. An illustrative independent verification implementation:

    1. import pandas as pd
    2. def rolling_median(series: pd.Series, window: int) -> pd.Series:
    3. """Independent reference implementation used to check whether the approximate
    4. formula-language写法 introduces any deviation."""
    5. return series.rolling(window=window, min_periods=window).median()
    6. # Comparison logic: subtract the exported indicator column from the reference point by point
    7. # diff = exported["icu"] - rolling_median(exported["close"], 20)
    8. # assert diff.abs().max() < 1e-6, "Systematic deviation found; check window alignment and adjustment convention"

    A schematic formula-side version (pseudocode that only expresses structure; use the target software’s documentation for exact function names):

    1. // ===== Parameter definitions =====
    2. N1 := 5; // short period
    3. N2 := 20; // medium period
    4. N3 := 60; // long period
    5. // ===== Center series: rolling median =====
    6. // The target language provides no MEDIAN, so use an approximate window-sorting approach
    7. CENTER := <implemented with window functions supported by the target language>;
    8. // ===== Moving average family =====
    9. ICU_S := MA(CENTER, N1);
    10. ICU_M := MA(CENTER, N2);
    11. ICU_L := MA(CENTER, N3);
    12. // ===== Timing signal =====
    13. BUY := CROSS(ICU_S, ICU_M) AND CLOSE > ICU_L;
    14. SELL := CROSS(ICU_M, ICU_S);
    15. // Note: the window includes the current day; suspended/adjustment handling is not applied;
    16. // see the log for differences from the report's conventions

    5. Common Blockers and How to Handle Them

    Blocker type Typical symptom Suggested handling
    Missing function set Sorting/quantile functions not built in Ask for an options comparison, pick the best, and require the approximation convention to be noted in a comment
    Syntax version differences The same function name means different things across platforms Lock “target platform + version” in the prompt and use the sample file as a hard constraint
    Parameter convention drift Window boundaries off by one day; including vs. excluding the current day Compare point by point against a reference implementation in the verification stage to pinpoint the exact line
    Hallucinated function names Calling a nonexistent function, which fails at compile time Ask the model to “declare uncertainty first if unsure whether a function exists”
    Attention decay on long tasks Losing the original comment convention after several rounds of fixes Re-paste the core constraints each round, or narrow the context scope

    6. How to Objectively Evaluate “Good Enough”

    Do not draw conclusions from a single experience. Build a small internal evaluation set for quantitative measurement—10–20 reports spanning different complexity levels—and record the following dimensions:

    • First-pass rate: the proportion of first-round generations that compile on the first try;
    • Fix-round distribution: median and P90; a P90 above 3 rounds means interaction costs are high;
    • Hallucination rate: the frequency of three error types—invented functions, invented parameters, and ignored comment conventions;
    • Numerical consistency: whether sampled-point errors fall within tolerance;
    • Stability: rerun the same prompt 5 times and check whether output structure and conclusions converge;
    • Effective-context behavior: whether the metrics above degrade significantly as input length grows.

    The value of this metric set is turning vague judgments like “it feels like it has caught up with the international frontier” into comparable, regression-testable numbers.

    7. Cost, Compliance, and Engineering

    On cost: individual subscription-based coding assistants currently commonly use monthly/yearly tiers, with the annualized yearly cost typically in the low hundreds of yuan. Measured against the one to two weeks of expert labor saved per report reproduction, the return on investment is clear; but the real cost item is the time humans spend verifying—do not count only the subscription fee when evaluating.

    On compliance: sell-side research reports are usually bound by copyright and usage-scope restrictions, and unpublished institutional reports additionally involve confidentiality obligations. For production, three layers of handling are recommended: data de-identification (removing institution names, ratings, and client identifiers), access auditing (recording who sent which documents to what environment and when), and output trail retention (keeping the complete chain of prompts, generated artifacts, and execution logs). For highly sensitive scenarios, choose a private or dedicated-cloud deployment form rather than using a public service directly.

    On engineering: this task can be fully batched. On Baidu AI Cloud, you can store report samples in object storage, use a container platform to host batch jobs and isolate dependencies, write each generation’s prompt and artifacts to a logging service for auditable trace-back, and use monitoring and alerting to track anomalies in quality metrics such as first-pass rate and fix-round count. This turns the manual “paste the formula and test” step into a gate in the pipeline rather than a one-off manual effort.

    8. Boundaries: What Still Requires Human Oversight

    The pitfalls of financial time-series scenarios are more insidious than in general engineering, and the following four categories must be confirmed by a human:

    1. Look-ahead bias (future functions): does the indicator reference data only available after the current day’s close, inflating backtested returns;
    2. Adjustment and ex-rights handling: unhandled dividends and stock splits produce false signals, especially on long-period windows;
    3. In-sample overfitting: report parameters are often tuned on a specific interval, so the indicator itself may already be overfit;
    4. Distinguishing fidelity from numerical consistency: producing a chart does not mean the report’s original logic was reproduced—you must go back and check the definition line by line.

    AI’s role here is as an accelerator: compressing “one to two weeks of heads-down translation and debugging” into hours, but the fidelity of the indicator logic, the reasonableness of the parameters, and the credibility of the backtest conclusions still require a named owner to sign off. Writing the verification criteria into the prompt upfront is the single most effective lever for raising AI output quality—more effective than switching to a stronger model.

    Action checklist: prepare the format sample file and glossary → run 10 reports with a fixed prompt template → establish baselines for first-pass rate and fix-round count → add an automated numerical verification script → then decide between a subscription or batch private deployment.