Natural Language Data Analytics Platform: Technical Architecture and Practice

1. The Starting Point: What Enterprises Need Is Not “a Model That Writes SQL,” but “Answers They Can Trust”

Over the past decade, enterprises have consolidated their data into analytical data warehouses but have not consolidated their metric definitions along with it. The result is predictable: the same “active customer count” is calculated by the marketing team based on registrations, by the finance team based on payments received, and by the operations team based on logins. Three reports show three different numbers, and meetings are spent arguing over who is right. Counterintuitively, the more self-service analytics tools you add, the worse this definition fragmentation becomes.

A natural language data analytics platform (also referred to in the industry as conversational analytics or conversational BI) exists to solve exactly this problem. Its goal is not to “let users learn less SQL,” but to converge three capabilities behind a single entry point:

• Unified query entry: describe the question in business language instead of recalling table and field names;
• Unified semantic definitions: metric definitions, dimensions, time granularity, and join paths are provided uniquely by a governed semantic layer;
• Expanded delivery formats: extending from static dashboards to written data reports that can be cited directly, and to narrated data videos.

One boundary needs to be stated up front: the ceiling on the output quality of such a platform is determined by the level of governance in the data and semantic layers, not by the model’s capability. Treating the semantic layer as “documentation bolted on afterward” is the single most common reason these projects fail.

2. Overall Architecture: A Six-Layer Structure

A data analytics platform that can go into production typically includes the following layers:

1. Data source access layer: connects to the enterprise data warehouse, analytical databases, CRM systems, and semi-structured data in object storage. The principle is “connect, don’t move” — push computation down to the source as much as possible, and keep only metadata on the platform side.
2. Metadata and semantic layer: table structures, field types, primary/foreign keys, partition keys, metric definitions, dimension hierarchies, synonym vocabularies, and metric owners. This is the trust boundary of the entire system.
3. Intent understanding and retrieval layer: intent recognition (aggregation, comparison, trend, drill-down, anomaly attribution), time-expression parsing, entity linking (mapping “East China region” to concrete enum values), multi-turn context completion, and ambiguity clarification.
4. Query compilation layer: instead of generating raw SQL directly, it produces a structured logical query plan (measures, dimensions, filters, sort order, time window, join paths), which a deterministic compiler then translates into SQL for the target dialect.
5. Execution and governance layer: permission pushdown (row-level/column-level), read-only accounts, query timeouts and cost budgets, result caching and incremental materialization, and execution auditing.
6. Delivery layer: dashboards, written data reports, and narrated data videos, along with the shared insight-generation and narrative-generation pipelines reused underneath them.

3. The First Hard Problem: Mapping Natural Language to a Logical Query Plan

Letting the model output SQL directly is engineering laziness. The maintainable approach is “two-stage compilation”: semantic understanding produces a plan object, and the compiler handles dialect differences and correctness.

def to_answer(question: str, user: UserContext):
    q = normalize(question)                       # normalize: full/half-width chars, units, time base
    q = resolve_time(q, today=user.today)         # "last month" -> absolute range
    q = link_entities(q, semantic_layer)          # dimension-value linking and synonym disambiguation
    plan = compile_plan(q, semantic_layer)        # produce a logical plan (not SQL)
    plan = apply_policies(plan, user)             # push down permission and cost boundaries
    if plan.ambiguities:
        return Clarify(plan.ambiguities)          # better to ask than to guess
    df = execute(plan, dialect=user.dialect)      # read-only account + timeout + row cap
    return Render(df, evidence=plan.provenance)    # result + evidence chain

The compiler must handle several high-frequency traps:

• Unspecified time granularity: “growth” without saying whether it is sequential (period-over-period), year-over-year, or cumulative — the default must be specified by the metric definition;
• Missing default filters: “sales” without stating whether it includes refunds or tax — this should be carried by a certified metric;
• Cardinality explosion: a full-table aggregation without filters should first be estimated; when it exceeds a threshold, switch to a sampled preview or prompt the user to add narrowing conditions;
• Grain mismatch: directly adding order-grain amounts to month-grain metrics is a classic incorrect join.

The corresponding metric definition should be a declarative configuration that is reviewable and diffable:

metric: net_revenue
version: 3
owner: finance-analytics
grain: [order_id]
aggregation: sum
expr: order_amount - refund_amount - tax_amount
required_filters: [status = 'paid']
dimensions: [order_date, region, channel, product_category]
time_policy: { default_grain: month, compare: [mom, yoy] }
certified_by: metric-review-board

4. The Second Hard Problem: Turning “Dare to Ask” into “Dare to Allow”

The first hard constraint in enterprise deployment is security: the model must not become a channel for bypassing permissions. Key engineering points:

• Least-privilege account: the platform uses an independent read-only account with DDL/DML disabled; views and stored procedures are authorized separately.
• Semantic permission pushdown: compile row-level/column-level policies into the WHERE and SELECT clauses, rather than filtering at the application layer after retrieval — this avoids both the privilege escalation and the waste of “fetch everything, then trim.”
• Cost guardrails: per-query scan volume, concurrency, timeouts, and result-row caps; route high-cost requests through pre-aggregation or caching.
• Pre-aggregation first: build materialized aggregate tables for high-frequency dimension combinations, have the query planner prefer hitting the aggregate layer, and fall back to detail tables when the hit rate is insufficient.
• Audit trail: record the original question, the generated plan, the SQL actually executed, execution time, the accessing user, and the data domains accessed, forming a traceable chain.

At the same time, a verifiability mechanism is needed: answers carry evidence (which metric version was used, which filters were applied, and the data cutoff time); run consistency checks on key metrics and cross-reconcile them against existing authoritative reports, downgrading any result to an “unconfirmed conclusion” when the deviation exceeds a threshold; and clearly flag results as unreliable when the sample size is too small or the ratio denominator is too small.

5. Delivery Formats: From Dashboards to Written Reports and Narrated Videos

A single analysis result should support multiple deliverables that share the same computation and insight pipeline.

1) Dashboards:固化固化固化 Fix one-off questions into reusable boards, with configurable refresh cycles and subscription pushes. The value lies in letting repeatedly asked definitions settle down, rather than re-asking them each time.

2) Written data reports: on top of charts, add structured narrative following a “conclusion — evidence — boundary” structure. The key design is slot binding — every number in the narrative template must be injected by the computation pipeline; the model is responsible only for wording, not for arithmetic:

{
  "headline": "East China region gross margin fell 2.1 percentage points month-over-month",
  "facts": [
    {"slot": "delta",  "value": -0.021, "source": "metric:gross_margin"},
    {"slot": "period", "value": "2026-10 vs 2026-09", "source": "time_policy"},
    {"slot": "driver", "value": "product_category=hardware", "source": "attribution:delta_decomp"}
  ],
  "caveats": ["The hardware category definition changed in week 3; cross-period comparison requires caution"]
}

For attribution, deterministic methods (dimension decomposition, contribution decomposition, anomaly detection) should be preferred, with the model restricted to explanation and expression. This is what determines whether a report can be accepted by finance and audit.

3) Narrated data videos: the pipeline runs “chart orchestration → key-finding extraction → narrative script → speech synthesis → video composition.” Technically it is not mysterious; the hard part is pacing control and factual consistency: each voiceover segment corresponds to a chart slice, the speech rate is aligned with subtitles, and numbers are strictly sourced from the same origin as the labels in the chart. It suits scenarios like management weekly reports and monthly business reviews — cases where “the message must be clear even when the presenter is absent” — but it is not suitable as a formal audit-trail source document.

6. Collaborative Governance: Metric Certification, Real-Time Sharing, and Version Tracking

What truly turns a platform from a “toy” into “infrastructure” is the governance mechanism:

• Metric certification: metrics in the semantic layer should have a clear owner, certification status (draft / under review / certified / deprecated), and approval records. Only certified metrics may appear in external reports and videos; uncertified metrics must be explicitly labeled.
• Metric reuse retrieval: before creating a new metric, search existing definitions first to avoid building duplicate synonyms; definitions that are too similar are directly flagged for merging.
• Version tracking: metric definitions, dimension mappings, and report configurations are all brought under version control, retaining the changer, change time, change reason, and approval chain. Example:

metric: gross_margin   v2 → v3
- exclude: refund_channel = 'internal_test'
+ exclude: refund_channel in ('internal_test', 'gray_release')
impact: average change of +0.3pp in this metric's absolute value over the past 12 months
reason: including gray-release refunds in main business gross margin violates financial recognition principles
approved_by: finance-analytics-lead

• Impact analysis: before changing a metric definition, automatically list the affected dashboards, subscriptions, report templates, and downstream data-fetching jobs, avoiding the situation where “changing one field silently corrupts ten boards.”
• Snapshot and replay: when sharing a board, carry a data snapshot and the semantic-layer version number so the exact content seen at that time can be reproduced afterward — extremely useful for resolving cross-period disputes.

7. Implementation Roadmap and Acceptance Metrics

It is recommended to proceed in four phases, each with a decidable exit condition:

• P0 Metadata and semantic layer (2–4 weeks): connect read-only data sources, collect table structures and lineage, and produce a core metric dictionary (a first batch of 20–40 metrics covering 80% of questions is recommended).
• P1 Q&A and dashboards (4–8 weeks): launch the logical plan compiler, permission pushdown, caching, and aggregation, and get the “ask → board” loop working end to end.
• P2 Reports and governance (4–8 weeks): report templates, slot binding, the metric certification process, and version and impact analysis.
• P3 Video and scaling: narrated videos, subscription distribution, and rollout across multiple business domains.

Do not judge acceptance by “accuracy” alone; use a combination:

Category	Metric	Description
Understanding quality	Intent-recognition accuracy, ambiguity-clarification rate	A high clarification rate signals insufficient synonym and dimension coverage in the semantic layer
Generation quality	Logical-plan review pass rate, first-attempt SQL execution success rate	Reflects the correctness of the compiler
Result trustworthiness	Reconciliation deviation vs. authoritative reports, manual correction rate	The core threshold for external delivery
Performance and cost	P95 response latency, cache hit rate, per-query scan volume	Strongly correlated with the aggregation strategy
Business adoption	Weekly active questions, number of boards built, metric reuse rate	Judges whether the platform is accumulating assets or consuming resources

8. Common Failure Modes

• Bypassing the semantic layer: letting the model generate SQL directly against hundreds of physical tables causes accuracy and maintainability to collapse at the same time.
• Dual-track definitions: old and new metrics coexist with no deprecation process — “both are queryable” in the short term, but no one dares to use them in the long term.
• Application-layer permission filtering: fetching everything first and then trimming carries both a privilege-escalation risk and inflated cost.
• Free-form narration: when the numbers in reports and videos do not come from the computation pipeline, conclusions that “look reasonable but are wrong” appear — a single incident can destroy all trust.
• Q&A without accumulation: without固化固化固化 consolidating high-frequency questions into boards and subscriptions, users keep asking repeatedly and the payoff never compounds.

9. Conclusion

The competitiveness of a natural language data analytics platform lies not in how large the model is, but in “how clean the semantic layer is, how deterministic the compilation chain is, and how closed-loop the governance process is.” Only by treating metric definitions as first-class citizens under version control, and treating answer generation as a verifiable engineering pipeline rather than a generative black box, can an enterprise truly gain the ability to “ask a question in one sentence and deliver in multiple formats.” If you are planning a similar platform, you can combine Baidu AI Cloud’s general-purpose capabilities in data warehousing, object storage, message queues, container platforms, and monitoring and alerting; prioritize completing the two foundations — the semantic layer and permission pushdown — and then gradually expand the delivery formats.