Accurate Text Rendering in Text-to-Image Models: Techniques, Evaluation, and Training on Domestic AI Chips
This article examines why text-to-image models struggle to render text correctly and how to make generated text accurate and readable. It covers the root causes of text-rendering failures, technical approaches such as injecting layout and glyph priors, two-stage generation, data engineering, and OCR-based alignment; how to evaluate text accuracy with concrete metrics and benchmarks; the engineering work required to run full-pipeline training on domestic AI chips; and practical guidance for enterprises deploying text rendering in production, including structured prompts, quality fallback, and common pitfalls.
Why “AI text drawing” has long been a hard problem
Generating a dreamlike landscape is not difficult for a generative model; getting it to spell a line of text correctly inside the image is. Poster headlines, product specifications, terms of a promotion, the small captions in an infographic—these seemingly “simplest” elements are exactly where text-to-image models tend to break down.
The core problem of visual text rendering
Rendering text requires a model to have two capabilities at once. The first is semantic understanding: knowing what should be said and what the character sequence is. The second is glyph control: reconstructing stroke structure, font weight, letter spacing, and layout relationships at the pixel level. The former is close to a language-modeling task; the latter is close to high-fidelity image reconstruction. Because the optimization objective of most image generation models is biased toward the latter, the language-side prior gets compressed, producing results that “look like text but aren’t actually text.”
Three typical failure modes
- Glyph corruption: missing or extra strokes, parts stuck together, strokes growing randomly—individual characters may still be recognizable, but the whole is unreadable.
- Character hallucination: characters that don’t exist in the source text, especially whole-word replacement or sentence drift in the middle of long text.
- Layout loss of control: the number of characters doesn’t match the preset region, long text overflows its bounds, line spacing is chaotic, and mixed Chinese-English typesetting is misaligned.
It’s worth emphasizing that Chinese is harder than English. The Chinese character set is large, glyph structures are complex, and stroke density is high. A model has to express more information entropy within the same pixel area, leaving far less room for error than Latin letters.
Root of the difficulty
The root cause is insufficient decoupling between the glyph space and the semantic space. If a model only receives a natural-language description, it has to “guess” the specific font, size, color, and position, and that uncertainty gets amplified into noise during decoding. This is why recent mainstream approaches all point in the same direction: externalize the uncertain layout decisions so that the model performs conditional generation given a specified layout and character sequence.
Technical approaches: from pixel fitting to joint glyph/language prior constraints
Injecting layout and glyph priors
In engineering practice, “text” is usually decomposed into explicit control signals and then fed to the generation backbone:
- Text region annotation: determine the bounding box, hierarchy, and rotation angle for each block of text;
- Character-level sequence conditioning: encode the target string and its language features and inject them, rather than relying only on a single global vector from the text encoder;
- Glyph rendering prior: first lay out a binary or grayscale glyph image with the specified font, then use it as a spatial condition map concatenated channel-wise with the noise latents;
- Regional attention constraint: introduce a region mask in the cross-attention layers to prevent text semantics from “leaking” into adjacent background regions.
# Illustrative flow for building structured conditionslayout = {"blocks": [{"text": "Summer New Product Launch", "bbox": [80, 120, 944, 260],"font": "bold_hei", "color": "#FFFFFF", "role": "title"},{"text": "Up to 50% off storewide", "bbox": [80, 300, 620, 380],"font": "regular_song", "color": "#FFE9A8", "role": "subtitle"},]}condition = build_condition(layout=layout, # layoutglyph_map=render_glyphs(layout), # glyph prior maptext_tokens=tokenize_text(layout), # character-level sequenceimage_size=(2048, 2048), # target canvas)image = pipeline(prompt=prompt, condition=condition)
The benefit of this approach is that it is explainable and controllable: if the font is wrong, you just change the font configuration—no model retraining needed.
Two-stage or hierarchical generation
Another approach is to solve the problem in layers. The first stage generates the background and the main visual subject while keeping text regions as low-information placeholders; the second stage renders the text regions at high resolution separately, then does edge blending and color harmonization. This strategy is more stable for very small font sizes and dense multi-line layouts, at the cost of a more complex pipeline and higher end-to-end inference latency.
Data: what actually sets the ceiling
A model’s capability ceiling is often determined by its data. Reusable data-engineering practices include:
- Image-text alignment cleaning: use OCR to relabel the real characters on training images, filter out label noise, and ensure “there is text in the image and the labels are correct”;
- Long-text sample mix: deliberately increase the proportion of long-paragraph, multi-line, mixed Chinese-English samples; otherwise the model only learns to write title-level short phrases;
- Font and layout diversity: cover different font weights, serif/sans-serif, artistic deformation, sticker and watermark occlusion, and other real-world distributions;
- Hard-example mining: feed failed glyphs and layouts from the evaluation set back into the training set to form an iterative loop.
Post-training alignment: turning “readability” into an optimization target
Pure pixel loss does not penalize “writing the text wrong.” That’s why an OCR-aware reward is commonly introduced: after generation, compute a consistency score between the OCR result and the original character string, and use it as a reward signal for preference optimization or rejection sampling. This way the model clearly knows during training that glyph correctness is just as important as image quality.
Evaluation: how to objectively measure whether the text is written correctly
Breaking down the core metrics
- Word/character accuracy: the proportion of recognized results that match the target text, measuring whether “the whole reads correctly”;
- Normalized edit distance (NED): measures the degree of difference in terms of insertion/deletion/substitution cost; it is more sensitive to partial stroke errors and is a necessary complement to accuracy;
- Text detection recall: whether text actually appears in the target region, and whether it is truncated or occluded;
- Layout consistency: whether the number of lines, hierarchy, and region occupancy match expectations;
- Image-quality metrics: sharpness, color harmony, and presence of artifacts, used to ensure that “readable but ugly” is not mistaken for success.
A single metric is not enough to draw a conclusion. In practice, look at the “recognition layer,” “layout layer,” and “quality layer” results together, and evaluate short-text and long-text scenarios separately.
The practical meaning of two kinds of benchmarks
One kind is a complex visual text generation benchmark, which tests readability when text coexists with scene elements—for example, text with buildings, text with products, text partially occluded. The other is a long-text rendering benchmark, which specifically stress-tests extreme cases with long paragraphs and large character counts. The former is close to real commercial posters; the latter exposes a model’s scalability flaws. Only by combining them can you cover the long tail of business needs.
Scenario-based evaluation sets are indispensable
Public benchmarks often diverge from the business distribution. It’s advisable to build your own evaluation set, stratified by business dimension: social media covers (short headline + person), e-commerce main images (spec table + price), educational illustrations (explanatory text), and advertising key visuals (artistic fonts). Keep a fixed proportion of manual spot checks at each layer, and attribute errors by type for the spot-check results.
Engineering essentials for full-pipeline training on domestic AI chips
In recent years, more and more multimodal models have chosen to complete the full pipeline from pre-training to post-training on domestic AI chips, and behind this lies a whole body of systems-engineering adaptation work.
Operator coverage and compilation optimization
- Operator completeness: image generation models involve a large number of normalization, dynamic-shape convolution, attention-variant, and interpolation operators; a single missing small operator can cause the whole network to fall back, with performance dropping sharply. Build an operator-coverage checklist and prioritize filling in the hot operators.
- Graph compilation and fusion: reduce memory-access overhead through computation-graph optimization, operator fusion, and memory reuse—usually the most direct way to speed up inference.
- Communication optimization: in multi-card training, the collective-communication overhead from parallel partitioning often becomes the bottleneck; you need to tune parallelism and overlap strategies for the chip’s interconnect topology.
Precision strategy and numerical stability
Mixed precision is the mainstream choice, but diffusion models are sensitive to numerical ranges across time steps. In practice, watch for: whether the loss-scaling strategy suits the domestic chip’s precision implementation; whether the output differences of different backends’ operators on the same input fall within an acceptable range; and whether precision drift occurs over long training runs. It’s advisable to keep one low-precision fast-training pipeline for experiment screening and another high-precision pipeline for final output.
Training stability and operability
Long-stable training has to handle checkpoint resumption, gradient-anomaly detection, sample-level replay, and so on. The domestic-chip ecosystem has coupling between framework versions, drivers, and communication libraries, so the version matrix should be brought into CI to lock down reproducible combinations of “chip model × framework version × driver version,” avoiding mismatched results when reproducing experiments.
The practical value of autonomy and self-reliance
Beyond supply-chain security, a fully self-reliant stack brings predictable cost structure. When both training and inference run on the same hardware stack, capacity planning, unit compute-cost accounting, and peak/off-peak scheduling all become easier to decide over the long term—which is especially critical for enterprises that need to continuously iterate their image-generation capabilities.
From model to business: key points for enterprise deployment
Scenario-matching principles
Not every need requires text rendering. Purely visual assets (atmospheric backgrounds, scene illustrations) can use regular generation capabilities; once a brand name, price, specification, or regulatory wording appears, you must enable the layout-controlled text-rendering pipeline with validation.
Size, ratio, and safe zones
Native multi-ratio output has become the norm, but in engineering you still need to watch for: social platforms have widely varying recommended sizes, so it’s worth maintaining a platform size-mapping table; text regions should have safe margins to avoid being cropped or occluded by the platform; small fonts easily blur after scaling, so do a readability regression at the target size after output.
Structured prompts: locking uncertainty in a cage
Vague descriptions induce the model to improvise; structured templates can significantly improve the first-pass success rate:
[TASK] Generate an e-commerce promotional main image, square, light background[SUBJECT] Summer running shoes, side view, placed in the lower right[TEXT - Main headline] "Summer New Product Launch" largest font size, centered, upper area[TEXT - Subheadline] "Light, breathable, comfortable all day" below the main headline, font size about 40% of the main headline[FONT STYLE] Sans-serif, clear strokes, no artistic deformation[COLOR] Main headline dark gray, subheadline brand blue[FORBIDDEN] No characters other than the specified text may appear in the image[QUALITY] HD, no noise, sharp text edges
The key is to make explicit “what to write,” “where to write it,” “how large,” and “what not to write.” Every degree of freedom the model decides on its own turns into failure rate.
Servicing and quality fallback
Generation quality has natural variance, so a production system must assume “a certain proportion will fail”:
def generate_with_check(prompt, expect_text, max_retry=3):for i in range(max_retry):img = image_service.generate(prompt=prompt,size="1024x1024",layout=build_layout(expect_text), # explicit layout)result = ocr_service.recognize(img) # general-purpose OCR capabilityscore = char_accuracy(result.text, expect_text)if score >= 0.95:store.upload(img) # archive to object storagemetrics.count("text_render_pass", tags={"retry": i})return imgmetrics.count("text_render_retry")return fallback_template(expect_text) # fallback: fill text into a template
Recommended supporting engineering capabilities include: an asynchronous task queue to absorb peak requests; storing generated results in object storage and accelerating delivery via a content delivery network; monitoring and alerting on success rate, retry rate, and single-call latency; and per-business-line call-cost accounting and quota management. On major public cloud platforms, these capabilities can be obtained as combinations of general-purpose cloud products—including model serving, object storage, message queues, logging services, and monitoring and alerting—so enterprises don’t need to build the underlying infrastructure themselves.
Three common misconceptions
- Looking only at sample images instead of batch pass rate: curated cases can’t reflect long-tail failure rates, so you must run batch regression.
- Ignoring copyright and compliance: font licensing, the commercial-use boundaries of generated content, and advertising-law wording all need up-front checks in the pipeline rather than after-the-fact fixes.
- Treating OCR as the sole judge: OCR can misrecognize too, so cross-validate it with manual spot checks, and set separate thresholds for artistically deformed fonts.
Conclusion
The progress of visual text rendering is essentially the result of three things maturing at the same time: on the model side, turning layout and glyphs from “implicit guessing” into “explicit conditions”; on the data and alignment side, turning “readability” into an optimizable, measurable target; and on the engineering side, giving training and inference a self-reliant, stable, and cost-controllable compute foundation. For enterprises, what really needs to be built is not trust in a particular model, but a production pipeline of structured input + automatic validation + failure fallback—models will keep iterating, and it’s the stability of the pipeline that the business can rely on in the long run.
