Long-Document AI Office Test: Reading 582 Pages in 8 Minutes, Presentation Generation Remains the Bottleneck

In intelligent office scenarios, expectations for multimodal agents have shifted from “can answer questions” to “can complete tasks.” For example, a system must not only read a prospectus of hundreds of pages but also extract the business structure, identify key financial metrics, summarize risk factors, and finally generate a presentation suitable for reporting. This process involves long-document understanding, table parsing, chart reconstruction, content planning, visual layout, and result validation. It is a representative task for measuring an agent’s overall capabilities.

A test on long public documents shows that an agent can complete the initial reading and information extraction of hundreds of pages in a relatively short time, but final presentation generation takes noticeably longer. The problem is not simply that the model “cannot understand.” It is concentrated in content organization, layout decisions, chart expression, and repeated checking. Therefore, evaluating an intelligent office system cannot rely only on response speed; it must also consider the complete chain from input material to final deliverable.

1. Long-Document Processing Is Not Just About a Larger Context Window

Traditional text question answering usually centers on a passage or a few pages. The system can place the full text in context and then generate an answer. But for prospectuses, annual reports, contracts, or technical specifications of hundreds of pages, simply expanding the context window does not automatically solve all problems.

Long documents often have the following characteristics:

1. Complex section hierarchy: tables of contents, body text, appendices, footnotes, and cross-references are interconnected.
2. Diverse information carriers: text, tables, flowcharts, financial charts, and scanned pages coexist.
3. Scattered key conclusions: the same topic may appear separately in business descriptions, risk factors, and financial sections.
4. Uneven content density: some pages contain core data, while others contain only legal statements or repetitive explanations.
5. Long semantic dependencies: a conclusion may depend on earlier definitions, later notes, and data across multiple tables.

Therefore, a reliable long-document processing flow usually needs to establish document structure first, then perform semantic retrieval and cross-section reasoning. A generic pipeline can be abstracted as:

Raw file
  ↓
Layout recognition and text extraction
  ↓
Structure sections, page numbers, tables, and charts
  ↓
Split content by semantics and hierarchy
  ↓
Build retrieval indexes and citation relationships
  ↓
Generate an evidence set for the task
  ↓
Summarize conclusions and output deliverables

Structuring is the step most easily overlooked. If the system converts pages only into continuous text, the row-column relationships in tables, the correspondence between headings and body text, and the numerical positions in charts may all be lost. Even if the model has strong reasoning capabilities later, distorted input can still lead to incorrect conclusions.

2. Why “Reading in Minutes” Still Does Not Mean True Understanding

During testing, the system could quickly complete an initial scan of long documents and return relatively complete answers to questions about business, finance, and risk. This shows that long-context retrieval and information summarization can already cover many office tasks.

But “finishing reading” has at least three different meanings:

• Completing content access: the system has processed all pages or relevant fragments;
• Completing information location: the system can find the sections, tables, and evidence relevant to the question;
• Completing task understanding: the system can organize scattered information into executable, verifiable conclusions.

The first two are not equivalent to the third. For example, the system can accurately find revenue changes, user scale, and business segment data, but if it cannot explain the relationships among these data points, the generated material is only an information excerpt, not an analytical report.

To reduce the risk of misjudgment, it is advisable to enforce an “evidence–conclusion” chain in long-document question answering. Every important conclusion should be linked to at least the following information:

Conclusion: A certain business revenue has changed significantly
Evidence: Corresponding section, page number, table number
Calculation: Change ratio or period difference
Interpretation: Possible reasons and supporting text
Confidence: High / Medium / Low

This design can effectively distinguish “facts explicitly disclosed in the original text” from “opinions inferred by the model from context.” For financial, legal, and investment materials, speculative content especially must not be written directly as definitive conclusions.

3. Data Understanding Determines Analysis Quality

The value of a long-document agent lies not only in answering “what is a certain number” but also in completing analysis across tables, sections, and time periods.

A qualified data analysis process includes at least four steps:

1. Identify Data Semantics

The system needs to determine the units, currency, time range, statistical scope, and whether special items are included in a table. For example, the same “revenue” may represent annual revenue, quarterly revenue, segment revenue, or adjusted revenue. If headers and footnotes are ignored and calculations are performed directly, the results can easily be distorted.

2. Align Dimensions

Different sections may use different names to describe the same business, or use similar names to represent different scopes. The system needs to establish a unified metric dictionary and align business names, time, region, product category, and statistical dimensions.

3. Perform Calculations

For metrics such as growth rates, proportions, and marginal changes, it is better to use a deterministic calculation module rather than leaving everything to the language model. Example formulas are as follows:

Growth rate = (current value - previous value) / previous value
Business share = segment value / total value
Profit margin = profit value / revenue value

The model is responsible for understanding the question and selecting data, the calculation module is responsible for executing formulas, and finally the model generates a natural-language explanation. This combination of “model planning, program calculation, and model expression” is usually more stable than relying solely on model mental arithmetic.

4. Run Consistency Checks

If the sum of segment data does not match the total, the system should automatically indicate possible rounding, scope differences, elimination items, or missing data instead of forcing a seemingly complete conclusion.

4. Why Presentation Generation Becomes the Bottleneck

Compared with reading and question answering, presentation generation is more complex because it is not a single task but a chain of multiple subtasks:

1. Determine the reporting goal and audience;
2. Extract core messages;
3. Plan the page sequence;
4. Choose suitable chart types;
5. Generate titles, body text, and notes;
6. Handle page layout;
7. Unify fonts, colors, and visual hierarchy;
8. Check for overflow, occlusion, and data errors;
9. Revise repeatedly based on check results.

In actual use, the system frequently checks and retries in the latter half of presentation generation, increasing overall time. This phenomenon is representative: text generation is often linear, while page generation is often a cycle of “generate—render—detect—fix.”

If a presentation contains more than a dozen pages, and each page has multiple text boxes, charts, and decorative elements, a minor layout change may affect the positions of other components. The system must constantly confirm:

• Whether titles are fully displayed;
• Whether body text exceeds text boxes;
• Whether chart labels overlap;
• Whether pages contain too much whitespace;
• Whether color contrast is sufficient;
• Whether styles are consistent across pages;
• Whether key data is obscured by visual elements.

Therefore, presentation generation speed cannot simply be equated with language model output speed. It is also affected by the rendering environment, page checking strategy, retry count, and execution framework efficiency.

5. Engineering Methods for Building a Stable Presentation-Generation Pipeline

1. Create a Content Outline First, Then Design Pages

Do not let the system generate the final presentation directly from the entire document. A more reliable approach is to execute in two phases:

Document analysis → Topic extraction → Presentation outline → Page draft → Visual layout → Quality check

The first phase focuses only on “what to say,” while the second phase focuses on “how to present it.” This avoids having the model analyze content and adjust layout at the same time, reducing repeated planning.

2. Build Page-Type Templates

Common pages can be abstracted into several types, such as:

• Cover page;
• Executive summary page;
• Data overview page;
• Trend analysis page;
• Comparative analysis page;
• Process explanation page;
• Risk notice page;
• Conclusion and recommendation page.

Each page type predefines a title area, body area, chart area, and notes area. The model only needs to fill in content and select layout parameters instead of designing from scratch every time.

3. Separate the Fact Layer from the Expression Layer

The system can maintain a structured fact table internally:

{
  "topic": "business performance",
  "metric": "a certain metric",
  "value": "original value",
  "period": "reporting period",
  "source": "section and page number",
  "confidence": "high"
}

Titles, summaries, and charts in the presentation are all generated from the fact layer. This prevents the same number from appearing inconsistently across pages and makes later source tracing easier.

4. Set a Limited Number of Validation Rounds

Unlimited retries do not necessarily improve quality; instead, they may cause uncontrolled time consumption. In engineering practice, clear check indicators and maximum revision counts can be set for each page:

if overflow == false and occlusion == false and data_validation == true:
    page passes
else:
    make targeted corrections based on error type

Compared with “regenerating the entire page,” targeted correction of titles, tables, or chart areas is more efficient and makes it easier to preserve the stability of parts that have already passed.

6. How to Comprehensively Evaluate Multimodal Office Agents

A single metric cannot reflect real office capabilities. It is advisable to build an evaluation system from the following dimensions.

Long-Context Capability

Assess whether the system can find distant information, handle cross-section references, and accurately return evidence locations. Tests should avoid asking only about local facts; they should also include questions that require synthesizing multiple sections.

Data Analysis Capability

Focus on unit recognition, scope alignment, formula calculation, anomaly detection, and conclusion explanation. In particular, distinguish between “the number is correct” and “the analytical logic is correct.”

Collaboration and Task Planning Capability

For complex tasks, whether the system can split steps, invoke different capability modules, record intermediate results, and recover after failure is an important indicator of agent maturity.

Deliverable Quality

Whether the final result is readable, editable, and reusable is more important than whether a single answer is fluent. For presentations, it is necessary to assess structural completeness, visual consistency, information density, and suitability for reporting.

Stability and Explainability

If the same input is executed multiple times, whether the results are broadly stable; whether key conclusions can be traced back to the original text; and whether the system can explain the cause when errors occur. These indicators directly determine whether the system can enter enterprise workflows.

7. Recommendations for Enterprise Deployment

When building intelligent office systems, enterprises should not only pursue larger models or faster first-token response speed. They should optimize from the perspective of the complete task chain.

First, establish a unified document parsing and data governance layer to ensure that the relationships among text, tables, images, and page numbers are not broken. Second, split retrieval, calculation, rendering, and validation into relatively independent modules for easier replacement and troubleshooting. Third, set human review nodes for key business areas, especially financial data, legal judgments, and externally released materials. Finally, continuously collect failure cases and build evaluation sets covering factual accuracy, page quality, and task completion.

Overall, long-document intelligent office systems already have a strong practical foundation. They can significantly reduce the cost of initial reading, information retrieval, and first-draft organization, but there is still a clear gap between “quickly understanding” and “reliably delivering.” A truly high-quality solution cannot rely only on the model itself; it needs to combine long-context processing, structured data, deterministic computation, page rendering, and quality control into a complete engineering system. Only then can an agent gradually evolve from a tool that answers questions into a productivity system capable of taking on real office tasks.