AI Coding Assistants for Data Insights: A Complete Engineering Practice

1. Turn Vague Requests into Verifiable Engineering Tasks

Phrases like “take a look at the market and find a few topics that could go viral” are typical open-ended requests: the goal is fuzzy, success criteria are missing, and data boundaries are unclear. Handing such a request directly to an AI coding assistant usually yields an analysis that looks complete but can’t actually be verified.

A more reliable approach is to run a requirements-engineering pass first, converging the open question into three clearly defined deliverables:

• Raw layer: A reproducible snapshot of source data, append-only and never modified, so any conclusion can be traced back to a specific file.
• Metric layer: A derived-metrics table with unambiguous definitions, where every column has a calculation formula and a denominator definition.
• Conclusion layer: A topic shortlist with evidence chains and risk warnings, where every conclusion is backed by supporting data.

These three layers are connected by file-level contracts. For example, you can agree on three directories — raw/, curated/, and report/ — where the file format and primary key within each directory stay fixed. The value here is that when an AI coding assistant runs a long task, having intermediate outputs written to disk means every step can be re-run and validated on its own. If one step fails, it won’t contaminate the whole pipeline, and you won’t need to feed the entire context back to the model.

The task contract can be written as a machine-readable manifest that the assistant executes step by step:

{
  "task_id": "content-insight-2026-08",
  "steps": [
    {"id": "S1", "name": "Export own historical content data", "output": "raw/own_articles.jsonl", "assert": "row count == authorized export record count"},
    {"id": "S2", "name": "Collect public platform leaderboard data", "output": "raw/platform_rank.jsonl", "assert": "field completeness >= 0.95"},
    {"id": "S3", "name": "Clean and compute metrics", "output": "curated/metrics.parquet", "assert": "no empty denominators, no non-negative constraint violations"},
    {"id": "S4", "name": "Layered comparison and attribution", "output": "curated/insights.json", "assert": "each conclusion must cite step and sample size"},
    {"id": "S5", "name": "Generate visualization report", "output": "report/final.md", "assert": "body numbers align one-to-one with the metrics table"}
  ],
  "guardrails": {
    "network": "read-only, rate-limited, no reuse of login sessions",
    "writes": "restricted to the workspace directory",
    "credentials": "injected from environment variables, never written to scripts or logs"
  }
}

2. Data Collection: Compliance and Stability Before Speed

The collection stage is where problems most often arise, and those problems are usually not technical. The correct order is: first confirm whether the data source is authorized, then think about scraping efficiency.

Compliance boundaries. Prefer the platform’s official open APIs, the creator console’s authorized export features, or data-export files from accounts you own. For third-party public content, respect the platform’s terms of service and crawling rules, throttle request frequency, don’t bypass rate limiting or anti-bot mechanisms, don’t collect personal information, and don’t use content for purposes beyond what’s authorized. Even when source material is used only for internal statistics and topic selection, avoid republishing others’ original text directly in the report.

Engineering stability. Scraping failures mainly come from three sources: rate limiting, structural changes, and network jitter. There are three corresponding responses: backoff retries, field tolerance, and resumable runs.

import json, time, random
from pathlib import Path
from urllib import request, error
RAW_DIR = Path("raw")
RAW_DIR.mkdir(exist_ok=True)
MIN_INTERVAL = 1.2   # Minimum request interval (seconds); adjust per platform terms and rate-limiting policy
MAX_RETRY = 3
_last_call = 0.0
def polite_get(url: str, timeout: int = 10) -> bytes:
    """Read-only fetch: polite interval + exponential backoff retries."""
    global _last_call
    last_err = None
    for attempt in range(MAX_RETRY):
        gap = MIN_INTERVAL - (time.time() - _last_call)
        if gap > 0:
            time.sleep(gap)                 # Client-side throttling; no concurrent flooding
        _last_call = time.time()
        try:
            req = request.Request(url, headers={"User-Agent": "internal-insight-bot/1.0"})
            with request.urlopen(req, timeout=timeout) as resp:
                return resp.read()
        except error.HTTPError as e:
            last_err = e
            if e.code in (429, 503):        # Rate limited / temporarily unavailable: back off and retry
                time.sleep((2 ** attempt) + random.random())
                continue
            raise                            # 4xx errors are mostly unrecoverable; raise immediately
        except OSError as e:
            last_err = e
            time.sleep((2 ** attempt) + random.random())
    raise RuntimeError(f"Fetch failed after {MAX_RETRY} retries: {url}") from last_err

Incremental and idempotent. Computing a content hash as the primary key for each record, and de-duplicating before writing to disk, makes repeated runs produce no duplicate data and turns “re-running once” into a zero-risk operation. It’s also worth archiving a copy of the raw snapshot to object storage, partitioned by task ID, to ease later audits and cross-task comparisons.

3. Metric Definitions: Consistent Definitions Matter More Than Complex Algorithms

Most analysis conclusions don’t hold up not because of model capability, but because the denominators are inconsistent.

• Engagement rate: Commonly defined as (likes + comments + saves + shares) / views. If some data sources only have likes and comments, comparing across sources directly will systematically underestimate engagement.
• Save rate: saves / views, reflecting the “retainable value” of content, and closer to long-tail traffic characteristics than engagement rate.
• Completion and drop-off: Requires duration data. When it’s missing, it should be explicitly marked as “not computable” rather than filled with 0.

The correct way to handle missing values is to distinguish three cases: a true value of 0, not collected, and collection failed. Mixing the three will distort every downstream percentile.

def ratio(numer: dict, denom, fields) -> float | None:
    """Return None if any numerator field is missing; never fill 0. Return None if the denominator is non-positive."""
    if not denom or denom <= 0:
        return None
    total = 0
    for f in fields:
        v = numer.get(f)
        if v is None:
            return None          # If any required field is missing -> this sample is excluded from the ratio
        total += max(v, 0)
    return total / denom

Before running statistics, you should also add data-quality assertions, writing validation into the pipeline instead of relying on the naked eye: whether record counts match the export, whether primary keys are unique, whether metrics fall within valid ranges, and whether extreme values exceed reasonable quantile thresholds. For outliers, it’s better to winsorize (truncate) than to delete them outright, and to disclose the handling rules in the report.

4. Analysis: From Descriptive Statistics to Testable Hypotheses

Layered analysis is more informative than an overall average. You can segment by topic, duration bucket, headline pattern, posting time, whether content is part of a series, and so on, then observe how the metric distributions differ across groups. Also pay attention to the dispersion of the distribution — the variance of viral content is usually far larger than that of ordinary content, which means “average performance” has limited value for topic-selection decisions.

The key methodological discipline: correlation is not causation. Observing that “a certain topic has high engagement” can only form a hypothesis; it doesn’t prove that producing that topic will work. To turn it into a basis for decisions, you need to design a small-scale verifiable experiment: fix the posting cadence, control a single variable (such as cover style or headline structure), set a minimum sample size and observation window, and write down the decision threshold in advance to avoid cherry-picking favorable data after the fact.

Attribution conclusions are best output in a three-column “evidence – strength – counterexamples” format, with each conclusion annotated with sample size and confidence level. The structure of this output determines whether the report can be trusted.

5. Tool Execution and Context Engineering: Let People Make the Call

The value of an AI coding assistant is compressing repetitive labor (writing scripts, editing fields, running batches, drawing charts) down to minutes, but it has three stable failure modes that need engineering safeguards.

Permissions and security. Letting the assistant write code is low risk; letting it execute destructive commands in a real environment is high risk. Work outside the working directory should be read-only, commands should produce a dry-run and wait for confirmation before executing, and credentials should be injected from environment variables or a secrets-management service and never appear in scripts, logs, or reports. Operations involving deletion, overwriting, or bulk updates should not be auto-approved by default.

Context management. In long tasks the model forgets early conventions, so the rules need to be externalized: put a project description file at the repository root that spells out the directory structure, naming conventions, metric definitions, and prohibited actions; then maintain a progress file that’s updated after each completed step. This is essentially swapping the “model’s short-term memory” for “persistent memory on disk,” so a task can resume from its last checkpoint after an interruption.

Sharding and acceptance. Keep the input and output of a single step at a size that can be read in one pass; when it exceeds that, slice it by time or category, run each slice, and then merge. Run the validation script at the end of every step — if it doesn’t pass, don’t proceed to the next step.

6. Reporting and Engineering: From One-Off Scripts to Reusable Pipelines

The report structure should lead with conclusions: first the three most important judgments, then the evidence tables and charts, and finally the risks and unverified items. Every number in the body must align item by item with the detail tables, and charts should annotate their definitions and sample size.

Once this workflow proves valuable, upgrade it from “manually triggered scripts” to a routine pipeline:

• Collection tasks are triggered on a schedule, with results written to object storage and partitioned by date;
• Long-running tasks are smoothed by a message queue, with worker-side containerized deployment and locked dependency versions;
• When processing or validation fails, use a logging service to search and monitoring alerts to notify by rule, instead of people repeatedly watching the terminal;
• Key intermediate artifacts are versioned to support recomputing historical metrics for any date;
• On cost, restrict large-model calls to the “needs judgment and generation” steps, hand pure computation to deterministic code, and cache the results of repeated requests.

This structure can run on common capabilities like Baidu AI Cloud’s container platform, object storage, message queue, logging service, and monitoring/alerting, without being tied to a specific implementation.

7. Boundaries: Which Judgments Shouldn’t Go to the Model

Three kinds of things must be handled by people: compliance judgments about data and content use, business trade-offs for conclusions, and the final wording of anything involving external publication. Models are good at structured, verifiable tasks and poor at taking on untraceable responsibility.

A pragmatic division of labor: the model produces “hypotheses to be validated + evidence + experiment design,” while people decide whether to do it, how much to invest, and when to cut losses.

Action Checklist

1. Define the three-layer deliverables and directory contract first, then start coding;
2. Prioritize authorized exports and official APIs: client-side throttling + backoff retries + hash de-duplication;
3. Write metric definitions into documentation and assertions; don’t fill missing values with 0;
4. Do the analysis in layers, annotate conclusions with sample size, and validate hypotheses with controlled small experiments;
5. Externalize rules and progress as files, apply least privilege, and require confirmation for destructive operations;
6. Lead the report with conclusions and keep numbers traceable; once stable, upgrade to a scheduled, monitorable pipeline.