LLM Document Intelligence Implementation Guide: From Extraction to Automated Tables

1. Start With the Real Cost: Copying Is Not the Main Expense

A typical request sounds simple: “Extract the key information from these ten invoices and put it into a table.” But the manual work behind it is long: opening files one by one, scrolling to locate fields, selecting and copying text, switching windows to paste, aligning columns, standardizing date and amount formats, checking results, and reworking missed items.

The cost comes from four sources, none of which is typing speed:

• Context-switching cost: Every time a person switches documents, they must rebuild the short-term memory of which fields are being searched for.
• Format-alignment cost: Different sources use different field names, such as “unit price” versus “price per unit,” or “tax-inclusive amount” versus “total amount including tax.” Dates may appear as 2024/3/1 or March 1, 2024, requiring manual normalization.
• Error-rework cost: A misread decimal, a missing row, or a shifted column often surfaces only during downstream aggregation, making root-cause analysis expensive.
• Non-reusable cost: The same work is repeated next month, and the experience is not converted into any durable asset.

The right engineering goal is therefore not “faster copy and paste.” It is to build a two-way pipeline: documents → structured data that can be calculated, validated, and stored, and structured data → compliant documents that can be delivered, preserve formatting, and support auditing. Large language models and their AI-agent forms are valuable because they can handle both ends: understanding unstructured content and generating structured artifacts, while compressing the glue code in between.

2. Capability Breakdown: A Four-Layer Model for Document Intelligence

Use the following four layers to evaluate any solution and locate its capability boundary. Most failures do not occur because the model is “not smart enough.” They occur because one layer is skipped.

L1 Layout Understanding and Text Acquisition

This layer determines “what can be seen.” It includes OCR for scanned documents, table-line reconstruction, reading-order detection, multi-column layout segmentation, header/footer/page-number removal, and cross-page table stitching.

The key output of this layer is not only text. It should also include positional anchors, such as page numbers, block coordinates, and cell addresses. All downstream review and traceability depend on these anchors.

L2 Semantic Extraction and Normalization

This layer determines “whether the understanding is correct.” The core work is not just extracting fields, but handling three practical problems:

1. Synonym consolidation: Map headers from different sources to a unified field dictionary.
2. Unit and dimension normalization: Unify currency, tax rates, weight units, and Chinese large-number units equivalent to 10,000 and 1,000.
3. Explicit missing-value and conflict handling: When a field is empty, output a null value with the reason. When multiple sources conflict, output candidates and supporting evidence rather than asking the model to “guess a plausible value.”

L3 Structured Generation and Writing

This layer determines “what is delivered.” Tables are the simplest form. Word-processing documents require consistent styles, numbering, table of contents, and revision marks. Spreadsheets require live structures, such as formulas, charts, and pivot tables, rather than static formatted numbers.

A frequently overlooked principle is that the generation side should be as declarative as possible. Let the model output instructions such as “make which edit at which anchor,” and let deterministic code perform the actual write. This provides far better format fidelity and traceability than having the model emit an entire document directly.

L4 Validation and Write-Back Loop

This layer determines “whether the system can go live.” At minimum, it should include field-level rule validation, cross-document consistency checks, such as totals equaling the sum of details, sampling-based human review, failed-run retries, and a manual fallback queue. Without L4, the accuracy figures from the first three layers are only marketing claims.

3. Three Typical Scenarios and Implementation Approaches

Scenario A: Extract Key Fields From Multiple Invoices and Aggregate Them Into a Table

Start with the schema. Before writing the prompt, define the target table structure. Use it to constrain model output, drive code-based validation, and generate the database table:

{
  "vendor_name": "string, required",
  "quote_no":    "string, required, must exist in the original document",
  "unit_price":  { "type": "number", "currency": "ISO 4217", "tax_included": true },
  "quantity":    { "type": "number", "unit": "string" },
  "total_amount":{ "type": "number", "rule": "abs(total_amount - unit_price*quantity) <= 0.01" },
  "valid_until": "string, normalized to YYYY-MM-DD",
  "_evidence":   [{ "field": "string", "page": "int", "snippet": "string" }],
  "_confidence": "high | medium | low"
}

Pay attention to the two non-business fields, _evidence and _confidence. They allow every row to be traced back to its source and turn untrusted model output into an operable system.

Chunking and retrieval. Do not put an entire long document into the context window. Split it by page or semantic section while preserving page numbers. First use lightweight methods, such as keywords, regular expressions, or small-model classification, to locate candidate pages. Then perform deep extraction only on those candidate pages. This reduces input volume and also reduces errors caused by confusing similar passages.

Do not guess conflicts. If two amounts appear in the same invoice, prioritize a computable relationship such as “tax-inclusive total > tax amount > sum of line items.” For cross-source conflicts, rank sources by trustworthiness and write low-confidence results into a separate “pending confirmation” column rather than mixing them into formal data.

Scenario B: Batch-Generate and Revise Word-Processing Documents

There are three implementation paths, ordered from highest to lowest determinism:

1. Template + placeholders: The model produces only field values, and code performs replacement. This suits contracts, reports, notices, and other documents with fixed formats, with zero formatting drift.
2. Edit instruction sequence: The model outputs instructions such as [{ "action": "replace", "anchor": "paragraph 3, sentence 2", "text": "..." }], and code locates and applies them in the document object model. This suits changing several places in an existing document. Revision marks, comments, and styles are guaranteed by the write layer.
3. Full-document generation: This is suitable only for first drafts or sketches, after which human or rule-based formatting is still required.

If the requirement includes “track changes,” “preserve comments,” or “must not break existing numbering,” choose path 1 or 2.

Scenario C: Spreadsheet Analysis and Chart Generation

The order matters: explore first, calculate second, then write formulas.

• Explore: Read headers, infer column types, calculate missing rates and anomalies, and identify merged cells.
• Calculate: Ask the model to output analytical intent and calculation steps rather than directly producing conclusion numbers.
• Land: Write results as cell formulas or pivot-table configurations rather than hard-coded values.

# Example: write analysis results as live formulas instead of static numbers
plan = model.plan_analysis(header_row, sample_rows)   # output steps and fields
for step in plan:
    if step["kind"] == "formula":
        sheet.write_formula(step["cell"], step["expr"])   # e.g., =SUMIFS(D:D,A:A,"East China")
    elif step["kind"] == "pivot":
        sheet.add_pivot(step["rows"], step["values"], step["agg"])
    elif step["kind"] == "chart":
        sheet.add_chart(step["range"], step["chart_type"])

The benefit is auditability: business users can inspect the logic behind a cell, results recalculate automatically after data updates, and the model does not need to be rerun.

4. Reliability Core: Constrained Output and Dual Validation

Treat model output as untrusted external input. This is the dividing line between a demo and a production system. A recommended loop is as follows:

def extract_with_retry(doc_pages, schema, max_rounds=3):
    history = []
    for i in range(max_rounds):
        raw = model.extract(doc_pages, schema, strict=True, examples=FewShot)
        data, errors = validate(raw, schema)          # structure + business rules
        if not errors:
            return data
        history.append((raw, errors))
        # Feed the previous errors back and request targeted fixes instead of a full rewrite
        doc_pages = augment_with_feedback(doc_pages, errors)
    return escalate_to_human(doc_pages, history)      # fallback queue

validate should check at least four layers:

1. Structure: fields are complete and types are correct.
2. Business rules: totals equal the sum of details, and tax rates are within a reasonable range.
3. Cross-document consistency: aggregate values equal the sum of detail rows, and identifiers are unique.
4. Format compliance: dates and currencies have been normalized.

Validation rules themselves should be configurable data, not scattered judgment branches in code.

5. Orchestration and Engineering: Make It a Real Pipeline

The difference between a single-file demo and production batch processing is engineering infrastructure:

• Separate files from metadata: Store original files in object storage and pass only file identifiers and task states through the queue, avoiding bloated message payloads.
• Decouple and buffer with message queues: Split extraction, validation, and writing into independent consumers so that one slow stage does not bring down the whole pipeline.
• Containerize compute and dependencies: Put heavyweight dependencies, such as document parsing, OCR, and headless browser rendering, into a container platform and scale them on demand. This keeps cold starts and version consistency manageable.
• Ensure idempotency and resumeability: Use “file content hash + prompt version + model version” as the idempotency key. For long tasks, record page-level checkpoints and resume from the checkpoint instead of rerunning the entire document.
• Provide observability: Logging should capture input summaries, latency, token usage, and validation failure reasons for each call. Monitoring and alerting should focus on three metrics: field-level accuracy drops, timeout rates, and fallback-queue backlog. The first is quality; the latter two are availability.
• Version and regression-test: Put prompts, schemas, and validation rules under version control. Every change should be compared against a fixed evaluation set, otherwise online quality can quietly degrade.

On Baidu AI Cloud, the architecture requires general-purpose capabilities such as object storage, message queues, container platforms, logging, and monitoring/alerting. These can support the relevant stages without introducing document-specific components.

6. Cost and Performance: Do Not Use One Path for All Tasks

A recommended optimization order is: reduce call volume → lower per-call cost → increase concurrency.

1. Reduce call volume: Do not use a model when a deterministic rule will suffice, such as regular expressions, templates, or header-mapping tables. Use lightweight classification first to determine document type, and send only difficult samples to the heavier pipeline. Cache results for the same file hash and prompt.
2. Lower per-call cost: On the input side, filter pages and remove duplicates. Do not send repeated headers, page headers/footers, or irrelevant appendices. On the output side, enforce a compact schema and prohibit the model from repeating the source text. Keep examples to 1–3 and cover major edge cases.
3. Increase concurrency: Apply token-bucket rate limiting according to downstream quotas, schedule batch jobs during off-peak periods, and parallelize long documents at the page level before merging. Note that inference services often price input and output tokens separately. Long context and highly redundant output can significantly increase the bill.

7. Security and Compliance: Define Boundaries Before Automating

• Minimize data sent externally: Mask locally where possible, such as ID numbers, bank account numbers, and contact phone numbers. Keeping computable amount and quantity fields is sufficient for most analytical needs.
• Choose deployment modes: For scenarios with strict data-residency requirements, use dedicated deployment or inference within a private network, and restrict outbound traffic at the network layer.
• Keep audit trails: Preserve the relationship between source snippets, extraction results, validation records, and modifying personnel to support post-incident accountability and external audits.
• Set human-review thresholds: Amount-related content, contracts, and externally disclosed materials should require mandatory human confirmation. Automation should reduce the work from “entering every item” to “confirming every item.”
• Isolate permissions: Separate file reading and result writing by project or tenant to prevent one task from reading directories belonging to others.

8. Model Selection Checklist: Compare Capabilities, Not Brands

The relative strength of large models depends heavily on the task. It is common for one model to lead on table reconstruction but lag on long-contract clause comparison. Instead of making broad rankings, score models with your own evaluation set. Suggested dimensions:

Dimension	Validation Method
Field-level extraction accuracy	200+ real samples, manually labeled; measure field-level precision and recall
Table and layout reconstruction	Accuracy on merged cells, cross-page tables, and multi-column layouts
Structured-output stability	Repeated runs on the same input; Schema compliance rate and result variance
Long-document processing	Context limit and recall of middle-section information in long documents
Document-writing capability	Whether it supports style preservation, revision marks, formula writing, and chart writing
Tool calling and file I/O	Whether agents can complete multi-step operations reliably without drifting
Deployment and compliance	Support for dedicated or private deployment and data-retention policies
Cost and throughput	Overall cost per document, peak throughput, and rate-limiting strategy

The evaluation set must cover the real long tail: handwriting, blurry scans, cross-page tables, missing fields, and dirty data with conflicts. Results tuned only on toy samples will fail in production.

9. Common Pitfalls and How to Avoid Them

• Mistaking fluency for extraction accuracy: Field-level validation and traceability are mandatory. Humans reviewing only summaries will never catch a misplaced decimal.
• Missing page or coordinate anchors: Once an error occurs, it cannot be located precisely, forcing a full rerun and greatly increasing operational cost.
• Letting the model overwrite formulas and formatting: Replacing formulas with static numbers removes recalculability. Write formulas or pivot-table configurations instead.
• Overloading examples: Too many few-shot examples multiply input cost quickly, while benefits saturate.
• Launching all scenarios at once: First use a single scenario, such as invoice aggregation, to close the loop across evaluation, validation, and alerting, then replicate it to other scenarios.
• Ignoring document-template drift: Upstream teams may change templates without notice, causing rules to fail silently. Add detection and alerting for template changes.

10. Recommended Implementation Roadmap

1. Step 1 — Define: Define the schema, field dictionary, and acceptance criteria for the first scenario. Build a labeled evaluation set of 100–300 samples.
2. Step 2 — Close the loop: Get the minimal pipeline working: parse → extract → validate → write → human confirmation. Focus on validation and traceability rather than throughput.
3. Step 3 — Engineer it: Add queues, object storage, containerized dependencies, idempotency, and resumeability. Complete logging, monitoring, and alerting.
4. Step 4 — Optimize: Use token- and latency-based routing, caching, and page-level parallelism to reduce the cost per document.
5. Step 5 — Expand: Abstract the evaluation set and validation rules into platform capabilities, then replicate horizontally to new document types.

Conclusion: The value of document intelligence is not making a single operation faster. It is converting the hidden cost of “someone must do this every time” into reusable rules, schemas, and evaluation assets. In the four-layer capability model, semantic extraction determines the upper bound of value, the validation loop determines whether the system can go live, and declarative writing determines delivery quality. Once these three things are solid, replacing a model becomes a configuration change rather than a redesign.