AI Image Generation Text Rendering: Principles, Evaluation, and Engineering Practices

    Text rendering is still one of the weakest links in generative image models, and Chinese-language content magnifies every difficulty involved. This article explains why writing text correctly is a fundamentally different task from drawing images well—covering latent-space compression that erases fine strokes, weak coupling between text and visual conditions, a shortage of high-quality image-text training data, and autoregressive decay in long text. It then defines the capabilities a production-ready, text-controllable image model needs, introduces a layered evaluation system built on Word Accuracy and Normalized Edit Distance (with a minimal runnable implementation), and walks through the engineering levers that make text rendering reliable: layered prompting, resolution staging, and local inpainting. It also covers cost and serving architecture at scale, plus a pre-launch checklist.

    1. Background: Why “rendering well” and “writing correctly” are two different problems

    The visual quality of generative image models has improved dramatically over the past few years, yet text rendering remains a persistent weak spot. The bottleneck is not compute—it is the fundamental nature of the two tasks. Natural images are the fitting of a distribution of continuous signals, which leaves plenty of room for error. Text, by contrast, is a symbolic system with high information density and strong structural constraints: shift a single horizontal or diagonal stroke of a character by a few pixels, and the meaning changes from one character to another.

    At the implementation level, the difficulty concentrates in four areas.

    1. Latent-space compression erases detail. Mainstream diffusion models first compress pixels into a low-dimensional latent space with an autoencoder, then denoise within that space. Chinese characters pack many strokes into a small number of effective pixels; once the compression ratio climbs, stroke intersections and corners get smoothed away, and the decoder can only “guess” a roughly similar glyph. This is why many models handle English reasonably well but degrade quickly on Chinese—Latin letters are highly redundant in structure, while Chinese characters leave almost no margin for error.

    2. Text and visual conditions are poorly coupled. Image-text alignment encodings are good at answering “what is in the scene,” but they do not necessarily tell the model “exactly where each character goes.” When the copy enters the model only as a single sentence, the model must simultaneously decide content, font size, font weight, letter spacing, line spacing, alignment, and line breaks. That is a severely underdetermined problem, which is why glyph shape drifts between samplings.

    3. High-quality image-text training data is scarce. A large share of internet images are weak pairs of “an image plus an unrelated caption.” Samples with clean layouts, legible font sizes, and clear copy hierarchy make up only a small fraction, so it is hard for models to learn proper typesetting rules from them.

    4. Long text suffers from autoregressive-style decay. Whether the model uses sequence modeling or region-based generation, as the copy gets longer the characters written correctly at the start get “pulled off track” by what follows—producing repeated words, missing characters, scrambled word order, and even the entire latter half degenerating into decorative strokes.

    Chinese-language scenarios amplify all four problems at once: many strokes per character, clusters of visually similar characters, mixed traditional/simplified forms and punctuation, vertical text alongside calligraphic typefaces, and mixed Chinese, Latin letters, numbers, and units. Any single mishandling surfaces directly as “typos.”

    2. Capability breakdown: what a competent text-controllable image model needs

    Turning “can write” into acceptance criteria makes model selection and tuning far easier.

    • Glyph-level reconstruction fidelity: The autoencoder needs higher latent-space resolution or glyph-aware training objectives so that small font sizes and dense strokes remain reconstructable.
    • Character-level condition injection: The copy to be rendered should be injected as an explicit condition (character- or word-level features, region-level positional constraints) rather than carried only by a single natural-language sentence.
    • Layout and typography understanding: The model should grasp hierarchy—large main title, smaller subtitle, body text as a block—and respect composition rules such as whitespace, alignment, and avoiding overlap with the main subject.
    • Long-text consistency: As length grows, the model should keep glyph shapes and content consistent from front to back and control repetition and missing characters.
    • Multilingual and Chinese-English mixing: The model should maintain balanced typesetting across different writing systems within the same frame.
    • Controllable local editing: Redrawing only the text region while preserving all other pixels makes “change the copy, not the image” a low-cost operation.

    3. Evaluation: how to objectively judge whether the text is correct

    Subjective impressions are unreliable, so you need quantifiable metrics. The two most widely used in the industry are:

    • Word Accuracy: Measures how well the generated result matches the target copy at the sequence level. It is typically computed with a sequence-comparison algorithm, ranges from 0 to 1, and a value closer to 1 means the content and order align more closely.
    • Normalized Edit Distance (NED): Takes the edit distance, normalizes it by the length of the target text, and subtracts it from 1 to remove the incomparability between long and short copy. It likewise falls in the 0–1 range.

    The two are complementary: Word Accuracy leans toward “overall similarity,” while NED leans toward “how many characters are wrong.” A ten-character line with two errors still scores well on NED, indicating the model “basically knows how to write” but lacks stability; if NED also drops sharply, the glyph reconstruction capability itself is usually flawed.

    The evaluation flow is generally: batch generate → OCR-recognize the text in the images → compare against the target copy with normalization → aggregate the metrics. Below is a minimal implementation you can run directly (standard library only):

    1. import re
    2. from difflib import SequenceMatcher
    3. def levenshtein(a: str, b: str) -> int:
    4. """Classic dynamic-programming implementation; returns the edit distance."""
    5. if not a:
    6. return len(b)
    7. prev = list(range(len(b) + 1))
    8. for i, ca in enumerate(a, 1):
    9. cur = [i]
    10. for j, cb in enumerate(b, 1):
    11. cur.append(min(
    12. prev[j] + 1, # deletion
    13. cur[j - 1] + 1, # insertion
    14. prev[j - 1] + (ca != cb), # substitution
    15. ))
    16. prev = cur
    17. return prev[-1]
    18. def normalize(text: str) -> str:
    19. text = text.strip().lower()
    20. text = re.sub(r"\s+", "", text) # remove newlines and extra whitespace
    21. text = text.translate(str.maketrans(",。!?;:、", ", .!?;:,"))
    22. return text
    23. def score(target: str, pred: str) -> dict:
    24. t, p = normalize(target), normalize(pred)
    25. ned = 1 - levenshtein(t, p) / max(len(t), 1)
    26. acc = SequenceMatcher(None, t, p).ratio()
    27. return {"ned": round(ned, 4), "word_accuracy": round(acc, 4)}
    28. if __name__ == "__main__":
    29. # Example: a single visually-similar character error
    30. print(score("泰国旅游最全攻略", "泰国旅游最全攻畧"))

    Design the evaluation set in layers, or a single score gives you little guidance. Useful layers include: short titles (4–8 characters), long copy (30+ characters), mixed Chinese-English, small-font body text, artistic or outlined text, text over complex backgrounds, and vertical or multi-column layouts. Scoring each layer separately lets you pinpoint whether the problem lies in glyph reconstruction, long-text consistency, or layout understanding.

    Watch out for these common pitfalls:

    1. OCR error contaminating the metrics. Recognizers themselves make mistakes, especially on outlined, gradient, or low-contrast text. Manually spot-check a batch of samples first to estimate the recognition-accuracy baseline.
    2. Inconsistent normalization. If full-width/half-width, traditional/simplified, punctuation, case, and whitespace are handled inconsistently, the metrics show jumps you cannot explain. The normalize function above exists precisely to prevent this.
    3. Ignoring error position. The same two errors have completely different business impact depending on whether they land in the title or in a corner watermark; consider weighting the title region separately.
    4. Drawing conclusions from a single sample. Generative models have high variance. Sample 3–5 times for the same prompt, take the mean, and record the standard deviation.
    5. Training-set leakage. If the evaluation copy overlaps heavily with the training corpus, scores look inflated; use self-authored copy for cross-validation.

    4. Engineering practice: four levers for using text rendering reliably

    4.1 Layered prompts: separate the copy from the scene description

    A vague description lets the model invent its own copy, and the result is inevitably uncontrollable. The recommended approach is to declare things in layers and give the text to be rendered explicitly:

    1. Generate a vertical social media cover, 3:4.
    2. Scene: a summer coastal highway, a vintage convertible, warm tones, film-like texture, empty space in the lower-right corner.
    3. Copy (write exactly as given; do not add, delete, or alter any characters):
    4. Main title: The Complete Summer Road Trip Guide
    5. Subtitle: 7 Days 6 Nights · Coastal Highway
    6. Layout: main title in the upper third, bold sans-serif, about 70% of the width; subtitle directly below, at 40% of the main title's size.
    7. Style: crisp text edges, no outlines or shadows.

    Three things matter here: give the copy in quotes or as a separate field; specify the size and position relationships explicitly; and forbid the model from rewriting the copy (most models support “write exactly as given” style constraints, though effectiveness varies by model and must be validated on your own evaluation set).

    4.2 Resolution and aspect ratio: compose first, finalize later

    Native support for multiple aspect ratios and reasonably high output resolutions (for example, any ratio in the 1024×1024 to 2048×2048 range) is highly valuable in real business: vertical covers, square product cards, and horizontal banners can all be produced from the same copy without post-cropping that cuts the text. In practice, “generate candidates quickly at low resolution to lock the composition → upscale or re-render at high step counts once a version is chosen” both saves compute and improves the usable output rate.

    4.3 Local inpainting: minimize the cost of changing copy

    In real business, copy changes far more often than the imagery does. The mature approach: first generate a usable image, lock the image regions, then apply local inpainting only to the text region. This preserves the composition and color palette you already tuned and avoids the style drift that full-image resampling brings. In engineering terms, keep the original image, the mask, and a copy version number so changes are easy to trace.

    4.4 Typical scenarios and acceptance focus

    • Social media covers: title prominence, whether it conflicts with the main subject, and legibility at thumbnail size.
    • Infographics / educational diagrams: information hierarchy, accurate numbers and units, module spacing.
    • E-commerce hero images and promotional materials: prices and spec numbers must be absolutely accurate and require manual double-checking.
    • Teaching hand-copied posters and courseware illustrations: standard Chinese glyph forms, avoiding typos in educational settings.
    • Packaging mockups and internal proposals: copy correctness and compliance with brand guidelines.

    Compliance note: AI-generated images released publicly should carry prominent disclosure labels as required; text in images involving commercial fonts needs licensing confirmation; do not produce materials containing real people’s portraits or misleading claims; and marketing copy must comply with advertising rules on absolute and superlative language.

    5. Cost and architecture: from “tune once” to “run all day”

    The per-image inference cost is determined mainly by model size, sampling steps, output resolution, and concurrency efficiency. When the workload scales from individual trials to tens of thousands of calls per day, the optimization space concentrates in four areas:

    1. Resolution tiering: Batch-generate candidates at low resolution first, and re-render only the finalists at high resolution.
    2. Caching and deduplication: Results for the same copy plus the same prompt can be reused, and cover-type materials typically have a high reuse rate.
    3. Batching and quantization: Improve single-GPU throughput by combining inference-side quantization with distilled small models to handle simple scenarios.
    4. Asynchronous tasks: Generation is a heavy task that takes seconds to tens of seconds, so it must be structured as “submit a task → poll or receive results via callback” to avoid synchronous interfaces being overwhelmed by long runtimes.

    The serving architecture can be built entirely from general-purpose cloud product categories, with no dependency on any vendor’s proprietary interfaces:

    1. Client
    2. → API gateway (authentication, rate limiting, quota)
    3. → Task queue (peak shaving, failed retries, priority tiers)
    4. → Inference cluster (elastic scaling on a container platform, scaled by queue length)
    5. → Object storage (originals, masks, and outputs bucketed separately, with lifecycle policies to clean up temporary files)
    6. → Content safety review (blocking political, adult, sensitive-term, and copyright risks)
    7. → Logging + monitoring/alerting (success rate, P95 latency, per-image time, cost aggregated by tenant)

    In public cloud environments such as Baidu AI Cloud, this structure can be implemented directly using general capability categories like containers, message queues, object storage, logging, and monitoring. What you actually need to invest in is three pieces of business logic: prompt-template management, the evaluation set and regression pipeline, and per-tenant cost accounting and quota governance. Get these three solid, and the marginal differences between models get smoothed out by engineering capability.

    6. Training on domestic compute: significance and engineering priorities

    Early multimodal model training relied heavily on accelerators from a single source. As domestically produced AI chips have matured in operator coverage, framework adaptation, and communication libraries, “completing the entire training pipeline on domestic chips” has become a viable path. This is not a simple hardware swap; the engineering problems are quite concrete:

    • Operator coverage and fusion: The performance and numerical stability of high-frequency operators—attention, normalization, convolution, and the like—directly determine whether training converges to the same accuracy.
    • Mixed-precision strategy: Choose an appropriate low-precision path while preserving convergence, and keep sensitive layers at higher precision.
    • Distributed parallelism: The combination of data, tensor, and pipeline parallelism and its match to the communication topology affect scaling efficiency.
    • Compilation and graph optimization: Use the compiler stack for operator fusion and memory reuse to reduce memory-access bottlenecks.
    • Long-run stability: Large-scale clusters inevitably experience hardware failures, so you need high-frequency checkpointing, automatic fault isolation, and resumable training; otherwise effective compute gets consumed by fault recovery.
    • Inference-side adaptation: Getting training to run is only the starting point—quantization, graph compilation, and batch scheduling determine whether the online per-image cost can actually come down.

    For developers, the impact is bidirectional. On one hand, localizing the training and inference stack makes compute supply more elastic. On the other, subtle differences in operator implementations and numerical behavior across backends can make the same prompt produce slightly different results in different environments. That is why the evaluation set must be re-run on the target inference environment rather than carrying over conclusions from the training environment.

    7. Deployment checklist

    • Have you built a layered evaluation set (short titles / long copy / Chinese-English mixing / small fonts / artistic text) with a threshold for each layer?
    • Have you unified the normalization rules for OCR and text, and measured the recognition baseline first?
    • Do you take multiple samples and average them for every prompt change, rather than judging from a single run?
    • Does the prompt declare the copy separately and specify font size, position, and proportion?
    • Have you implemented local inpainting so that changing the copy does not regenerate the whole image?
    • Are you using resolution tiering and result caching to control cost?
    • Have the tasks been made asynchronous, with retries, idempotency, and quotas configured?
    • Did you complete content-safety review and AI-generated labeling before launch?
    • Did you run the metrics on the target inference environment (not just the training environment)?

    The competition in text rendering may look like a contest of model metrics, but it is really a systematic contest of three things: the evaluation system, prompt engineering, and a serving architecture. Turning these three into reproducible, regression-testable, and auditable engineering assets is the real prerequisite for generative imagery to enter production workflows.