Office AI Agent Hands-On: 582 Pages in 8 Minutes, but Stalling on PPT Generation
This article examines why office AI agents can read long documents quickly yet still struggle to deliver finished presentations. Based on a hands-on test in which an agent read 582 pages in eight minutes but repeatedly checked its slide output, it breaks down five capability layers, common evaluation pitfalls, failure modes, and engineering practices such as structured outlines, design tokens, deterministic validators, task budgets, idempotent actions, and acceptance gates for reliable production use.
Introduction: Reading Is Not the Same as Delivering
After a multimodal office AI agent was released, a hands-on test was conducted on the desktop experience. The results showed that the agent read through a 582-page prospectus in eight minutes and produced a presentation and a poster with a consistent visual language. However, during the later stage of presentation generation, it repeatedly checked the output, and the time cost increased noticeably.
These two observations are not contradictory. “Reading long documents” and “producing finished deliverables” are fundamentally different tasks:
- Long-document speed reading is an information-compression problem. The model locates key passages within a long context, summarizes them, and cites them. Input and output boundaries are relatively clear, and the result is easy to evaluate automatically.
- Deliverable generation is a multimodal delivery problem. The model must satisfy several requirements at once: content completeness, reasonable layout, traceable data, and consistent visual style. These requirements are coupled. If any one fails, rework is required, and the signal for rework is often sparse or unreliable.
Setting aside the specific product used in the test, this article addresses a more general question: Why can office agents “read everything” yet still fail to “deliver”? How should developers and enterprises solve this at the engineering level? This is a key step for AI agents moving from “demo-ready” to “production-ready.”
1. Capability Layers: What Office Agents Actually Need
Before discussing the bottlenecks, it helps to break office tasks into five capability layers. Clear layering turns vague ideas such as “usable” into engineering problems that can be located and optimized.
| Capability layer | Core question | Typical failure mode | Engineering countermeasure |
|---|---|---|---|
| Long-context retrieval | Can it accurately locate and cite content in long documents? | Missing key clauses, incorrect page references | Section structuring, retrieval augmentation, source labeling |
| Cross-modal understanding | Can it read charts, layouts, and images? | Misreading units or scales, charts disconnected from conclusions | Chart-specific extraction, secondary data verification |
| Task execution | Can it reliably call tools and advance multi-step workflows? | Repeated actions, stalls, timeouts | Task budgets, idempotent tools, state machines |
| Visual expression | Can it produce layouts and visuals with consistent style? | Text overflow, theme drift, inconsistent style | Structured outlines, design tokens, layout libraries |
| Delivery acceptance | Can it verify that results meet requirements and finish on time? | Repeated self-checking, failure to converge | Deterministic validators, stop rules, quality gates |
A single capability is necessary but not sufficient. A longer context window cannot replace task planning or delivery acceptance. A high benchmark score also does not guarantee end-to-end delivery quality.
2. How to Read Evaluations: From Benchmark Rankings to Engineering Choices
2.1 What Mainstream Benchmarks Examine
Office-agent benchmarks generally cover the following dimensions. Developers can treat them as a “capability radar chart” when selecting tools.
| Test dimension (neutral description) | Main focus | Relevance to enterprise office scenarios |
|---|---|---|
| Team collaboration | Multi-role division of labor, task allocation, and coordination | High: cross-department collaboration projects |
| Data understanding and result delivery | Data analysis, reasoning, and conclusion output | High: business analysis, finance, and due diligence materials |
| Automated task execution | Tool calling and multi-step task progress | High: batch processing and workflow automation |
| Work assistant collaboration | Everyday office assistance | Medium-high: information organization, meeting notes |
| Presentation generation | Content organization and visual expression | High: reports, roadshows, and training materials |
| Web browsing and retrieval | Online search and page understanding | Medium: industry research, fact-checking |
| Long-text memory | Long-term memory across turns | Medium: long-term project tracking |
| Long-context understanding | Overall understanding of very long documents | High: prospectuses, annual reports, contract review |
| Role memory and personalization | Remembering user preferences and role settings | Low-medium: personalized assistants |
| General reasoning and planning | Reasoning and planning for complex tasks | Medium: strategy formulation, solution design |
2.2 Three Common Misreadings
Misreading 1: A single-dimension ranking represents overall capability.
Different models perform differently across dimensions. One model may tie or closely compete with others on several tests. Choosing only the highest score on one benchmark can easily overlook capabilities that matter more for the business.
Misreading 2: Public benchmark scores equal production performance.
Benchmark datasets are usually decontaminated, with clear scenario boundaries and relatively fixed scoring standards. Real office tasks are open-ended, multi-constraint, and time-sensitive. The correlation between the two is limited.
Misreading 3: End-to-end latency equals model inference speed.
In hands-on tests, the extra time caused by “repeated checking in the later stage” often comes largely from tool calls, rendering, and evaluation framework overhead. A model may generate text quickly, but the overall task can still be slow.
2.3 A More Reliable Approach: Build a Scenario-Based Regression Set
A more reliable method is to use public benchmarks for initial screening, then build a regression set based on real enterprise tasks:
- Sample real tasks. Cover high-frequency scenarios such as long-document summarization, data analysis, and presentation generation, while preserving original attachments and layout requirements.
- Define acceptance rules. Each task should specify pass criteria, such as accurate page references, consistency between charts and conclusions, and no text overflow.
- Use independent annotation by multiple people. Each task should be scored independently by at least two people. Disputed samples should go to arbitration to improve scoring consistency.
- Run tasks multiple times. Each task should be executed 3–5 times. Evaluate both average pass rate and stability (standard deviation), because office scenarios are especially sensitive to “works sometimes, fails other times.”
- Record layered metrics. Separately measure end-to-end latency (P50/P95), model generation time, tool and rendering overhead, task completion rate, and rework count.
Only when “capability” is broken down into measurable indicators can evaluation results be converted into selection decisions and tuning directions.
3. Failure Mode Analysis: Why Agents Get Stuck on Presentation Generation
3.1 Self-Check Loops: Missing a Decidable Stop Condition
Presentation quality involves multiple subjective judgments: whether the layout looks good, whether information density is appropriate, and whether charts are readable. The model often cannot produce a clear “done” signal, so it keeps checking and making small adjustments:
Read content → Plan outline → Render first draft → Self-check finds issues → Revise → Self-check again → New issues appear↑ │└────────────────────────── No clear stop rule ←────────────────────────────┘
Compared with coding tasks, the difference is clear. Code has compilers and unit tests, and “pass” is binary and decidable. Layout problems, however, are often only perceived as “something looks off,” and there is no deterministic mapping between the fix and the goal.
3.2 Environment and Pipeline: Rendering and Orchestration Overhead Counted as Task Time
- Rendering is expensive. Each check may require re-rendering the result as an image before the model can “see” it. Font loading, graphics library initialization, and image decoding all add latency.
- Orchestration constraints are often ignored. Timeout settings, retry policies, and context truncation in the evaluation framework or runtime environment directly affect agent behavior.
- Context grows longer. Long document content plus multiple rounds of revision history dilutes constraints, causing fixes to solve one issue while worsening another.
Therefore, latency must be interpreted carefully. It reflects the overall behavior of “model + environment + toolchain” and should not be directly attributed to slow model inference.
3.3 Multimodal Perception: Layout Issues Are Hard to Detect Without Rendering
Text overflow, overlapping elements, and misalignment are essentially two-dimensional spatial relationships. They are difficult to express directly in plain text. This creates a dilemma: without rendering, the problem cannot be seen; with frequent rendering, the entire workflow slows down. If there is no lightweight, deterministic checking method, the agent can only move back and forth between “render” and “judge.”
3.4 Convergence Strategy: Check Granularity and Depth Lack Constraints
Checking only one item at a time, such as color scheme, versus checking everything at once, such as color scheme, font size, overflow, and charts, creates very different costs in context and time. Agents usually do not specify a maximum number of check rounds or a per-step time limit. This can lead to a “local optimum trap”: each revision improves one aspect but worsens another, and the overall output still fails to meet acceptance standards.
3.5 Attribution: Four Root Causes
The bottleneck comes from the combination of four layers, not a single model defect:
- Goal layer: Delivery standards are hard to convert into machine-decidable conditions.
- Perception layer: Layout problems depend on rendering, making feedback expensive.
- Execution layer: There is a lack of explicit task budgets and stop rules.
- Environment layer: Orchestration and rendering overhead lengthen end-to-end latency.
4. Engineering Practices: Making Agents Deliver Reliably
4.1 Structure First, Render Later
The core idea is to change presentation generation from “free-form creation” to “constrained template filling”: the model only outputs the content structure relevant to the business, while the theme, layout, and coordinates are handled entirely by the rendering engine.
Example (JSON):
{"title": "2024 Annual Business Review","theme": "corporate_blue","slides": [{"layout": "metric_cards","title": "Full-Year Performance Overview","cards": [{"label": "Revenue", "value": "8.64 billion yuan", "delta": "+12.3%"},{"label": "Gross Margin", "value": "41.2%", "delta": "+1.5pp"}],"source": {"doc": "2024 Annual Report.pdf", "page": 38}},{"layout": "two_column","title": "Risks and Challenges","left": {"heading": "Key Risks","bullets": ["Raw material price volatility", "Uncertain overseas market demand"]},"right": {"heading": "Mitigation Measures","bullets": ["Lock in long-term procurement agreements", "Expand into diversified markets"]}}]}
The benefits are:
- The model only needs to focus on “what to say,” not “where to put it.”
- Layout overflow and overlapping elements are avoided at the rendering layer.
- Each point can include a source document and page number, making review and verification easier.
4.2 Design Tokens and Visual Language Consistency
In the hands-on test, the presentation and poster maintained a consistent style. This is a practice worth preserving. Its essence is “one set of design tokens drives multiple deliverables”:
theme:name: corporate_bluecolors:primary: "#1F4E79" # Primary coloraccent: "#ED7D31" # Accent colorbackground: "#FFFFFF" # Backgroundtext: "#333333" # Textfonts:heading: {family: "SimHei", size: 32, bold: true}body: {family: "SimSun", size: 18}layout:slide_size: [1280, 720]margin: 60 # Page margin (px)line_height: 1.4 # Line-height multiplier
Two points need to be emphasized:
- The same tokens can be used for slides, posters, long images, and other deliverables. This is key to visual language consistency.
- Tokens should be centrally maintained by designers and placed under version control. The agent should only “fill in content” and should not be allowed to modify the theme. This avoids style drift caused by multiple rounds of revisions.
4.3 Layered Generation Pipeline
Break the task into four stages. Each stage should have a single responsibility and produce inspectable output:
Parse long document → Retrieve key sections → Generate structured outline (JSON) → Render engine lays out (PPTX/HTML/poster) → Export deliverables
- Parsing stage: Preserve page numbers and section hierarchy to support citation and localization.
- Retrieval stage: Combine keyword and semantic retrieval to narrow the context the model needs to focus on.
- Outline stage: Completed by the model and used directly as rendering input.
- Rendering stage: Use a headless browser or document generation library inside an isolated container.
A headless browser approach usually first builds HTML pages at fixed sizes, then exports them to PDF or images through printing or screenshot capabilities. General office document processing libraries can then convert the output into slide formats. The exact choice depends on the enterprise’s existing technology stack.
4.4 Deterministic Validators: Turning “Looks Good” into Decidable Checks
The following example contains three lightweight validators that do not depend on a large model. Each outline check can run at millisecond level and can be executed frequently.
import mathTOKENS = {"colors": {"primary": "#1F4E79","accent": "#ED7D31","background": "#FFFFFF","text": "#333333"},"fonts": {"heading": {"family": "SimHei", "size": 32, "bold": True},"body": {"family": "SimSun", "size": 18}},"layout": {"slide_size": [1280, 720],"margin": 60,"line_height": 1.4}}PALETTE = set(TOKENS["colors"].values())def _linearize(channel: float) -> float:return channel / 12.92 if channel <= 0.04045 else ((channel + 0.055) / 1.055) ** 2.4def contrast_ratio(fg: str, bg: str) -> float:def luminance(hex_color: str) -> float:r, g, b = (int(hex_color[i:i + 2], 16) / 255 for i in (1, 3, 5))return 0.2126 * _linearize(r) + 0.7152 * _linearize(g) + 0.0722 * _linearize(b)l1, l2 = sorted((luminance(fg), luminance(bg)), reverse=True)return (l1 + 0.05) / (l2 + 0.05)def check_palette(deck: dict) -> list[dict]:"""Theme consistency: all color values must come from design tokens."""def walk(node):if isinstance(node, dict):for value in node.values():yield from walk(value)elif isinstance(node, list):for item in node:yield from walk(item)elif isinstance(node, str) and node.startswith("#"):yield node.upper()allowed = {p.upper() for p in PALETTE}return [{"code": "OFF_PALETTE", "slide": None, "detail": color}for color in set(walk(deck))if color not in allowed]def check_text_fit(deck: dict) -> list[dict]:"""Text fit: estimate width as 1em for Chinese and 0.55em for ASCII."""width, height = TOKENS["layout"]["slide_size"]usable_w = width - 2 * TOKENS["layout"]["margin"]body_size = TOKENS["fonts"]["body"]["size"]line_h = body_size * TOKENS["layout"]["line_height"]issues = []for i, slide in enumerate(deck["slides"]):for key in ("title", "subtitle"):text = slide.get(key, "")if not text:continuesize = (TOKENS["fonts"]["heading"]if key == "title"else TOKENS["fonts"]["body"])["size"]width_em = sum(1.0 if ord(ch) > 255 else 0.55 for ch in text)lines = math.ceil(width_em * size / usable_w)if lines * size * TOKENS["layout"]["line_height"] > height / 3:issues.append({"code": "TEXT_OVERFLOW", "slide": i, "detail": key})return issuesdef check_contrast(deck: dict) -> list[dict]:"""Contrast: body text >= 4.5, large text >= 3.0."""bg = TOKENS["colors"]["background"]pairs = {"body": (TOKENS["colors"]["text"], 4.5),"heading": (TOKENS["colors"]["primary"], 3.0)}return [{"code": "LOW_CONTRAST", "slide": None, "detail": name}for name, (fg, minimum) in pairs.items()if contrast_ratio(fg, bg) < minimum]
4.5 Task Budgets and Explicit Stop Rules
To address self-check loops, the key is to give the task hard boundaries and define clear finishing conditions:
import timefrom dataclasses import dataclass, field@dataclassclass TaskBudget:max_steps: int = 8 # Maximum execution stepsdeadline_s: float = 180.0 # Overall time limitstep_timeout_s: float = 60.0 # Per-step timeoutmax_rounds: int = 2 # Maximum self-check rounds@dataclassclass RunState:steps: int = 0rounds: int = 0started_at: float = field(default_factory=time.monotonic)def next_action(state: RunState, budget: TaskBudget, issues: list[dict]) -> str:elapsed = time.monotonic() - state.started_atblocking = [issue for issue in issuesif issue["code"] in {"OFF_PALETTE", "LOW_CONTRAST"}]if not issues:return "finish" # All checks passed -> deliverif elapsed >= budget.deadline_s or state.steps >= budget.max_steps:return "degrade" # Budget exhausted -> degraded deliveryif state.rounds >= budget.max_rounds and blocking:return "degrade" # Blocking violation cannot be fixedreturn "revise" if blocking else "polish" # Fix blocking issues first; polish others later
agent_runtime:default: strict # Strictly enforce stop rules by defaultpilot: relaxed # Relax policies for pilot tasks to observe self-check benefits
A few notes:
- Timeout takes priority over perfection. Delivering a usable draft with complete content and acceptable layout within the deadline is better than delivering a “perfect” draft after timeout. When performing degraded delivery, revision records should be preserved for human handoff.
- Handle issues by severity. Color deviations and insufficient contrast are blocking issues and must be fixed. Subjective issues such as whitespace ratio and icon placement can be delivered with annotations for manual fine-tuning.
- Instrument and observe. Record time spent, token consumption, and trigger reasons for each check round. This makes it possible to judge whether checks actually improve quality. If most rounds do not change the final score, reduce the depth of checking promptly.
4.6 Idempotent Actions and Operation Primitives
One major reason repeated checks slow down the process is that actions are not idempotent: every “fine adjustment” by the model may trigger a full reflow. It is better to reduce operations to a small set of primitives and ensure that repeated execution produces consistent results:
set_slide_layout(slide_id, layout): set the layout of a slide.set_text(slide_id, key, text): write text and automatically adapt font size and line breaks.trim_text(slide_id, key, max_chars): truncate by semantic completeness while preserving sentence integrity.split_bullets(slide_id, max_items): automatically split a slide when there are too many bullet points.
Three principles apply:
- Make targeted edits; do not trigger full reflows.
- Change only one thing at a time and record the reason.
- After each operation, re-render only the affected slide, not the entire document.
4.7 Acceptance Gates
Set quality gates before delivery: if any blocking issue remains unresolved, export is blocked. In addition, track the following metrics:
- Source traceability rate;
- Chart-conclusion consistency rate;
- First-pass success rate.
When validation fails, the system should output a structured issue list rather than a vague evaluation, making it easier for humans to locate problems quickly.
5. Enterprise Adoption: From Trial to Scale
5.1 Typical Scenarios and Required Capabilities
| Scenario | Key capabilities | Main risk points | Suggested acceptance criteria |
|---|---|---|---|
| Prospectus/annual report speed reading | Long-context understanding, information extraction | Missing risk factors, incorrect citations | Key clause recall rate, citation page accuracy |
| Business analysis report | Data analysis, conclusion delivery | Inconsistent data definitions | Key numbers match source tables, conclusions supported by data |
| Presentation material generation | Content organization, visual expression | Text overflow, style drift | Theme consistency, no overflow, first-pass success rate |
| Batch content production | Task orchestration, parallel execution | Resource contention, task timeouts | P95 latency, retry success rate |
5.2 System Architecture Recommendations
Large-scale enterprise deployment usually includes the following common modules:
- Access layer: Unified identity authentication and permission control, with data visibility scoped by department and project.
- Document layer: Use object storage to save original long documents and intermediate artifacts, managed by version.
- Retrieval layer: Chunk sections and vectorize them, build a full-text index, and support hybrid keyword and semantic retrieval.
- Execution layer: Use message queues and container platforms to handle rendering and parsing tasks. Isolate priorities by queue so long tasks do not block short tasks.
- Observability layer: Use log services to collect full-link tracing data. Combine monitoring and alerting to set threshold alerts for task latency, failure rate, and budget overruns.
- Security layer: Validate file type and size upon upload, execute rendering in an isolated sandbox, and disable dynamic execution for formula cells by default. Apply masking and permission isolation to sensitive documents, and retain operation audits.
Baidu AI Cloud offers corresponding capabilities in common cloud product categories such as object storage, container platforms, log services, and monitoring and alerting. Enterprises can use these building blocks to assemble the layered architecture above without building infrastructure from scratch.
5.3 Human-AI Collaboration Workflow
The boundary of automation should be defined in advance. A recommended loop is “agent drafts, human reviews, agent revises”:
- In the drafting stage, the agent completes more than 80% of the structured content.
- Humans focus on reviewing data definitions, compliance wording, and key conclusions.
- Revision feedback is returned to the agent as a structured checklist for targeted edits, avoiding open-ended dialogue that can cause goal drift.
This preserves the agent’s advantages in speed and coverage while keeping high-risk judgments with humans.
5.4 Observability Metrics
| Metric category | Specific metrics | Purpose |
|---|---|---|
| Quality | First-pass success rate, source traceability rate, rework rounds | Measure delivery quality |
| Efficiency | End-to-end latency P50/P95, model generation time, tool and rendering time | Locate latency bottlenecks |
| Cost | Average token consumption per deliverable, number of calls | Evaluate unit delivery cost |
| Stability | Result consistency across repeated runs with the same input | Determine suitability for formal workflows |
Conclusion
The fact that an office agent can read 582 pages in eight minutes shows that long-document understanding is no longer the main weakness of office agents. The real bottleneck is how to turn “understanding” into “delivery” in a stable, controllable, and timely way.
For developers, the key is to convert subjective aesthetic judgments into verifiable engineering constraints: structured outlines, design tokens, deterministic validators, task budgets, and stop rules are all essential. For enterprise users, model selection should not rely only on single-dimension benchmark rankings. It should combine scenario-based regression testing with observable metrics.
In office scenarios, the evaluation standard remains a simple one: can the work be completed on time, with quality, and with traceability?
