Evaluating LLM-Generated Presentations: Quality Metrics and Engineering Practice
This article explores how to build a reliable pipeline for automatically converting long-form documents or outlines into presentation decks using large language models. It breaks down the core capabilities required—long-text processing, structured outline generation, automated layout, and information supplementation—then defines concrete quality dimensions such as content coverage, factual consistency, layout compliance, editability, and processing stability. A reference implementation sketch demonstrates a rule-based layout constraint system, and practical deployment recommendations are provided for teams looking to integrate LLM-based slide generation into their workflows.
1. Background and Problem Definition
Converting long documents into presentation slides is a frequent task in knowledge work. A report of tens of thousands of words typically requires extracting the chapter structure, compressing arguments, adding charts or tables, and ensuring consistent page layout. Doing this manually is time-consuming and prone to issues such as misaligned formatting, inconsistent fonts, and mismatched visuals. As a result, many teams are turning to large language models (LLMs) to assist or automate this workflow.
The core question addressed here is: given input material that may be lengthy, topically clear, or even just an outline, how can we build a reliable automated generation pipeline and objectively determine whether the output quality meets business requirements?
2. Core Capability Breakdown
A usable automatic slide generation system generally relies on four capability modules.
2.1 Long-Text Handling
Source documents often exceed the model’s context window. Common practices include chunked summarization, hierarchical distillation, and source localization. The document is first split by sections; each section is summarized into key points, and then the summaries are combined into a presentation outline. This preserves the original structure and avoids missing content that appears later in the document.
2.2 Structured Outline Generation
The outline serves as the intermediate representation connecting the source text to the slides. It is recommended to ask the model to output structured data—for example, JSON—where each slide contains a title, a bullet list, a suggested layout type, and references to relevant source paragraphs. Structured output allows the program to validate required fields and makes it easy for humans to edit later.
2.3 Layout Automation
The main challenge in layout is alignment, spacing, and fitting content within a slide. LLMs are not well-suited for pixel-level calculations. A more robust approach is to have the model select a layout from a predefined layout catalog, and then let a deterministic rendering engine compute absolute coordinates. Each layout should define constraints such as the number of text boxes, maximum character counts, and image aspect ratios. When content exceeds these constraints, the system should trigger pagination or content condensation instead of forcibly shrinking font sizes.
2.4 Information Enrichment and Image Support
When the input is only a topic or a brief outline, the system may need to retrieve external information to supplement facts and cite sources. For images and charts, options include asset libraries, chart-rendering libraries, or generating charts directly from structured data. Regardless of the method, all chart data must be traceable to its source, avoiding plausible-looking numbers without supporting evidence.
3. Quality Evaluation Dimensions
Evaluating such a system should not rely solely on overall visual impression; it should be broken down into measurable indicators.
- Content coverage: Are all key arguments from the original text included in the outline? This can be checked via manual sampling or keyword matching.
- Factual consistency: Do the data on the slides match the source text? Numeric content should be verified item by item.
- Layout compliance: Are heading hierarchy, font sizes, alignment, and white space consistent? Rule-based scripts can detect text overflow and overlapping elements.
- Editability: Does the exported file preserve native text and shape objects? This directly affects downstream collaboration and modification.
- Processing stability: Can the system handle inputs of varying length and structure? When it fails, it should produce clear error messages rather than partial or unstructured output.
4. Engineering Implementation Sketch
The pseudo-code below illustrates a typical pipeline. Function names are illustrative; actual implementation depends on the chosen model and rendering library.
def build_slides(document_text: str, layout_catalog: dict) -> list:# 1. Split by sections, keep each chunk within token limitschunks = split_by_sections(document_text, max_tokens=6000)summaries = [summarize(chunk) for chunk in chunks]# 2. Generate structured outlineoutline = generate_outline(summaries, output_format="json")# 3. Apply layouts per slide and enforce capacity constraintsslides = []for item in outline["slides"]:layout = layout_catalog[item["layout_type"]]if exceeds_capacity(item["bullets"], layout):item["bullets"] = condense(item["bullets"], layout["max_chars"])slides.append(render_slide(item, layout))# 4. Run rule-based quality checksrun_quality_checks(slides)return slides
Two points stand out in this flow. First, the outline is generated in a structured format, so the program can verify each field programmatically. Second, capacity checks are performed by deterministic rules, not by the model, making the output more predictable and reproducible.
5. Deployment Recommendations
- Fix the layout catalog first, then integrate the model. The fewer layout types, the easier it is to maintain consistent formatting.
- Set up human review checkpoints for data, conclusions, and citations. These are the most costly types of errors.
- Track the correspondence between each slide and source paragraphs. This simplifies error tracing and correction.
- When using external materials or images, verify copyright and usage licenses, and cite sources within the document.
- Build a small benchmark test set. Re-run evaluations each time the model or templates change to prevent quality regression.
6. Applicability Boundaries
Automatic slide generation is suitable for well-structured, information-heavy materials such as weekly reports, project updates, and training decks. For formal presentations that require strong visual storytelling, strict brand compliance, or nuanced argumentation, professional designers and editors are still necessary. The primary value of such systems is to shorten the time from document to first draft and reduce repetitive formatting work—not to replace human judgment on content quality.
