Why AI Image Generation Keeps Getting Text Wrong: Text Rendering Principles and Engineering Solutions
AI image generators often render text as garbled, misspelled, or invented characters, and Chinese is harder than English because of its larger character set, complex components, and dense sets of look-alikes. This article explains why diffusion models struggle with text, how to measure rendering quality with character-level accuracy, exact match rate, and normalized edit distance (with runnable Python), and how to choose among three optimization paths: data, model, and post-processing. It also provides a failure-mode troubleshooting checklist, guidance on estimating compute and cost, and a clear rule for deciding which use cases can rely on generated text and which need deterministic glyph synthesis.
1. Why “looking right” isn’t the same as “written correctly”
Generating a cat or a landscape, the model only needs to capture semantics and style. Generating text is harder: the model has to get three things right at once — correct glyph shapes, correct character order, and a sentence that reads correctly. If any one of the three breaks down, the image fails no matter how striking it looks. That’s why so many models produce beautiful images but botch the text: missing strokes, extra strokes, characters bleeding into one another, or outright invented glyphs that don’t exist.
Chinese raises the difficulty another notch. English words are linear combinations of a finite set of letters, whereas Chinese characters are highly structured, block-shaped glyphs — stroke-heavy, built from complex component combinations, and packed with confusable look-alikes (未/末, 己/已/巳, 日/曰). A single displaced stroke can turn one character into another, so the model needs more than a “semantically approximate” result; it needs glyph-level modeling that approaches pixel precision.
2. Why diffusion models are inherently bad at writing text
Most mainstream image generation models are built on a diffusion architecture: the model progressively denoises in noise space to reconstruct an image that satisfies the given condition. This global denoising excels at continuous textures and soft light and shadow, but it clashes with the nature of text in three ways:
- Discrete vs. continuous: Characters are discrete symbols with a single correct form. Interpolating in a continuous feature space tends to blur, break, or merge strokes at their edges.
- Sequence consistency: Text has to be written in the right order. The model must implicitly learn layout rules like “left to right, top to bottom,” or characters end up scrambled, duplicated, or displaced.
- Long-range dependency decay: The further along the sequence you go, the more the constraints from earlier text get diluted, and glyph and layout quality in the later portion degrade. This is the real reason long-text rendering is so much harder than short text — not simply that there are “more characters.”
3. How do you quantify text rendering quality?
You can’t rely on eyeballing it; you need reproducible metrics. The most common approach in the industry is an “OCR recognition + text comparison” pipeline.
3.1 Character-level accuracy (word/char accuracy)
Start by defining how edit operations are counted. Say the target text has N characters. After the model’s output is run through OCR and aligned with the target, count the substitutions (S), deletions (D), and insertions (I), then:
Accuracy = (N - S - D - I) / N
When the output matches the target exactly, the value is 1. When there are too many insertions, it can go negative, so in practice it’s usually clamped to [0, 1]. Watch the differences in definition:
- Character-level reports “detail correctness.” It’s sensitive to a single wrong or missing stroke and suits evaluating the overall quality of long text.
- Exact Match Rate = number of fully correct samples / total samples. It reports the “directly shippable ratio,” which is better suited to posters, product images, and other cases where rework isn’t an option.
Read both metrics together. A character-level score of 0.95 with an exact match rate of 0.3 means every sample is slightly off and none of them is usable as-is.
3.2 Normalized Edit Distance (NED)
Edit distance is the minimum number of insertions, deletions, and substitutions needed to turn string A into string B. Once normalized, it converges to the range [0, 1], where values closer to 1 mean closer to the target text. It turns local errors like a missing character or an extra stroke into a comparable continuous score, complementing accuracy nicely.
def levenshtein(a: str, b: str) -> int:"""Rolling-array implementation, memory O(min(m, n))"""if len(a) < len(b):a, b = b, aprev = list(range(len(b) + 1))for i, ca in enumerate(a, 1):cur = [i]for j, cb in enumerate(b, 1):cost = 0 if ca == cb else 1cur.append(min(prev[j] + 1, # deletioncur[j - 1] + 1, # insertionprev[j - 1] + cost)) # substitutionprev = curreturn prev[-1]def normalized_edit_distance(pred: str, target: str) -> float:"""NED = 1 - dist / max(len(pred), len(target))"""if not pred and not target:return 1.0 # both empty treated as a perfect matchif not pred or not target:return 0.0 # one side empty, treated as no matchreturn 1.0 - levenshtein(pred, target) / max(len(pred), len(target))print(normalized_edit_distance("春江花月夜", "春江花月夜")) # 1.0print(normalized_edit_distance("春江花月", "春江花月夜")) # 0.8 (missing one character)
There are several conventions for the normalization denominator, and you need to align on one within your team, otherwise the metrics can’t be compared across teams: dividing by max(len(pred), len(target)) is the most common; a symmetric variant divides by len(pred) + len(target) (which yields ≈0.889 for the example above) and penalizes errors more gently. Be sure to state which convention you used in the evaluation report.
3.3 Long-text-specific evaluation
Build separate sets for short sentences, long sentences, and paragraphs, and report accuracy and exact match rate broken down by Chinese, English, and mixed Chinese-English. Publicly available references include Marquee-Bench, AnyText-benchmark, and CC-OCR among other Chinese-related benchmarks; for specific task definitions and leaderboard conventions, refer to their papers and official repositories. The value of this kind of evaluation is that it exposes the “beautiful on short sentences, broken on long sentences” weakness.
4. How to choose among the three optimization paths
| Path | Main benefit | Relative cost | Suitable scenario | Main risk |
|---|---|---|---|---|
| Data side (high-quality image-text alignment data) | Sets the capability ceiling; the fuller the glyph coverage, the more stable | High (long collection, annotation, and cleaning cycle) | Self-development/fine-tuning, vertical fonts, or industry-specific phrasing | Dirty samples bake in wrong glyphs |
| Model side (enhanced encoder, high resolution with multi-scale fusion, region awareness) | Directly improves dense layouts and small-font clarity | High (training compute and tuning cost) | Long text, complex layouts, mixed Chinese-English | Overfits a single font family; generalization drops |
| Post-processing side (sharpening, OCR verification, deterministic glyph synthesis) | Fast results, strong controllability | Low (small engineering effort) | Commercial materials with a fixed character count that must be exact | Affects naturalness; must be coordinated with the layout style |
Data-side essentials: keep glyphs clear and legible, align image and text strictly, and vary the font styles (serif, sans-serif, handwriting, decorative); for Chinese, cover enough common characters and enough confusable-character groups so the model has sufficient discrimination; during cleaning, drop blurry, occluded, and mislabeled samples.
Model-side essentials: strengthen the text encoder’s representation of character sequences to keep semantics and order stable; combine high-resolution training with multi-scale feature fusion to reduce glyph distortion at small sizes and in dense layouts; add region-awareness mechanisms that provide a spatial prior for “where text appears and how much space it takes,” reducing text merging into the background.
Post-processing-side essentials: sharpening and contrast enhancement on the generated region improve both OCR recognition and human legibility; for scenarios with very high glyph-consistency requirements, use a hybrid approach where “the model produces the background and layout, and text is overlaid by deterministic rendering” — but this path requires extra layout and color work and reduces visual freedom, so whether it’s worth it depends on the cost of rework.
5. Failure-mode troubleshooting checklist
| Symptom | Common cause | Priority fix |
|---|---|---|
| Missing or broken strokes | Insufficient resolution / text region too small | Raise output resolution; increase the text region’s share |
| Extra strokes, character merging | Insufficient character spacing; distracting background texture | Constrain layout whitespace; use region awareness or local inpainting |
| Confusable-character mix-ups | Insufficient glyph distinctiveness | Add confusable-character data; route key characters through synthesis |
| Scrambled or duplicated character order | Weak sequence constraint in the text encoder | Shorten the copy; generate sentence by sentence, then stitch |
| Quality collapse in the later portion | Long-range dependency decay | Generate in per-sentence or per-line segments to shorten each sequence |
| High OCR score but poor to the human eye | Stiff font; doesn’t fit the image | Add font and color constraints; run manual spot-checks |
6. How to estimate compute and cost
Chinese text rendering has two easily overlooked cost amplifiers. First, the larger Chinese character set pushes up the compute and memory usage tied to the text encoder and vocabulary. Second, high resolution and long copy significantly lengthen the sequence, so attention cost grows accordingly. As a result, “long Chinese text” is usually far more expensive than “a short English sentence,” and you can’t extrapolate roughly from image counts.
On unit price: pricing varies widely across vendors, resolutions, and versions, and whether post-processing like OCR verification and glyph synthesis is included will noticeably change the total. Don’t base a budget on any secondhand “a few cents per image” claim. Instead, do three things during selection: check the chosen service’s current official pricing page, run a round of load testing with real copy and resolution, and fold post-processing and manual-review costs into the total.
Cost reduction is usually a combination of tactics: cache results for repeated copy; produce a low-resolution draft first and only render high definition after the copy is confirmed; and use a message queue to smooth and stagger batch jobs so you don’t keep resources reserved for peak load.
A typical pipeline’s division of responsibilities:
Business request → API gateway (auth / rate limiting) → Model inference (GPU/NPU cluster)→ Result object storage → Optional post-processing (OCR verification / glyph synthesis) → Return to frontend
Object storage handles persistence of large result volumes, message queues absorb batch peaks, and logging plus monitoring and alerting keep things stable and make cost observable. These are all general cloud capabilities — every major cloud provider offers equivalents — so compare options by compliance requirements, regional availability, and your existing tech stack.
7. Typical scenarios and a feasibility call
- Marketing materials and posters: the copy can’t be wrong — go with the synthesis approach, or spot-check generated output manually.
- Educational and office materials (handwritten posters, courseware, infographics): mixed Chinese-English and structured layouts are the main hurdles.
- E-commerce hero images and product-promo images: short selling-point phrases are fine to generate, but synthesize key specs and prices.
- Lightweight social content (meme stickers, greeting cards): the most forgiving — generated output can be used directly.
The rule of thumb is simple: when rework costs more than synthesis, choose synthesis.
8. FAQ
Q: Is Chinese really harder than English? Yes. It has a larger character set, more complex component structure, and more confusable characters, so it demands higher modeling precision at the same accuracy.
Q: Does a high OCR score mean good quality? Not necessarily. OCR only accounts for “is it recognizable.” Whether the font is harmonious, whether the whitespace is reasonable, and whether the subject is occluded still need manual review or visual metrics.
Q: Can generated text be used commercially as-is? That needs a separate review. The font’s own license and the compliance of trademarks and advertising wording in the copy are not something model output can guarantee — run the normal legal and copyright review before commercial use.
Q: When should you not count on generative rendering? When the copy is long, must be exact character for character, or requires a strict brand font, the generative approach’s failure rate and rework cost won’t add up.
9. Summary
Text rendering is the watershed of image generation because it tests three abilities at once — semantic understanding, precise glyph modeling, and sequence layout — and Chinese raises all three difficulties simultaneously. The path to shipping is clear: first build an objective yardstick from character-level accuracy, exact match rate, and NED; then combine-select from the data, model, and post-processing paths by scenario; use the troubleshooting checklist to pin down specific problems; and finally, measure compute and post-processing costs before setting a budget. Understanding this thread matters more than chasing a single “total score” when deciding whether your product can actually go live.
