Chinese Text Rendering and Arbitrary-Resolution Image Generation: Architecture, Evaluation, and Deployment
This article examines two practical bottlenecks in text-to-image generation: incorrect Chinese text rendering and limited output sizes. It introduces a mixed architecture that separates semantic understanding from pixel decoding, then discusses glyph-conditioned injection, synthetic training data, text-region weighting, and character-level evaluation for Chinese text accuracy. It also covers positional encoding, training sampling, and inference constraints for arbitrary resolutions, along with general engineering principles for training on new AI accelerators. Finally, it provides deployment guidance for poster and cover scenarios, including text hierarchy testing, canvas-size validation, hybrid generation, compliance review, and regression testing.
1. Background: The Next Challenge in Image Generation
Over the past few years, text-to-image models have made clear progress in image quality, style control, and semantic alignment. In real business scenarios, however, generated results often run into two problems. First, text in the image is wrong: glyphs have missing strokes, or an entire phrase turns into pseudo-text that looks like writing but is not readable. Second, output sizes are locked to a small set of preset aspect ratios, so the model cannot directly meet the layout requirements of posters, banners, social media covers, and other formats.
These two problems appear independent, but they are both related to how a model represents “semantics” and “space.” The following sections discuss each problem and a practical combination approach.
2. Mixed Architecture: Let “Understanding” and “Rendering” Play Separate Roles
A pure diffusion model progressively denoises in latent space. It is good at texture and detail, but it has weaker constraints on long instructions, multi-object relationships, and text content. An autoregressive language model is good at sequential reasoning and parsing complex instructions, but it is not suitable for directly and efficiently outputting high-resolution pixels.
A common approach is to separate the two:
- Understanding module: Built around an autoregressive language backbone, it parses prompts, plans the global composition, determines object relationships and the text content to be rendered, and outputs a set of high-level semantic representations.
- Decoding module: Built around a diffusion decoder, it generates pixel details conditioned on the semantic representation and injects text-rendering information locally.
The advantage of this division of labor is that errors can be localized. Problems in the understanding module are concentrated at the semantic level and can be corrected through instruction data and fine-tuning. Problems in the decoding module are concentrated at the detail level and can be improved with more image data, better noise scheduling, and stronger conditioning.
Prompt ──> [Autoregressive semantic module] ──semantic representation──> [Diffusion decoder] ──> Image│ ▲└──── text content and layout constraints ───────┘
3. Chinese Text Rendering: From “Looks Like Text” to “Correctly Written”
The difficulty of text rendering lies in the large character set, complex structures, and many similar characters. English has only a few dozen letters, while common Chinese characters number in the thousands, and many glyphs differ by only one stroke. If training is done only at the image level, the model easily learns the “visual style of text” rather than “whether the glyph is correct.”
There are three common engineering approaches:
- Glyph-conditioned injection: Convert the text to be rendered into glyph images or character-level encodings and feed them to the decoder as additional conditions. The model therefore does not have to infer glyphs only from the language representation.
- Synthetic data training: Use font-rendering programs to generate batches of images with precise annotations, covering different fonts, font sizes, backgrounds, and layout directions, so the model sees enough glyph variation.
- Text-region weighting: Increase the loss weight for text regions during training, or introduce a verification signal based on text recognition to penalize incorrect characters, missing characters, and garbled text.
During evaluation, “text accuracy” and “visual quality” should be measured separately. For text accuracy, text recognition can first extract text from the generated image, which is then compared character by character with the target text to calculate the character-level error rate. Visual quality can use conventional image-quality metrics together with human evaluation. The weight of the two types of metrics depends on the scenario: poster scenarios emphasize text accuracy, while pure illustration scenarios place more value on composition and style.
def char_error_rate(pred: str, target: str) -> float:"""Calculate character-level error rate based on edit distance (Levenshtein)."""m, n = len(pred), len(target)dp = list(range(n + 1))for i in range(1, m + 1):prev, dp[0] = dp[0], ifor j in range(1, n + 1):cur = dp[j]cost = 0 if pred[i - 1] == target[j - 1] else 1dp[j] = min(dp[j] + 1, dp[j - 1] + 1, prev + cost)prev = curreturn dp[n] / max(n, 1)print(char_error_rate("智能海报生成", "智能海报生成")) # 0.0print(char_error_rate("智能海抱生成", "智能海报生成")) # 0.2
4. Arbitrary Resolution: From Preset Ratios to Continuous Sizes
Most models use fixed aspect-ratio buckets, such as 1:1, 4:3, and 16:9. This makes batching convenient but limits output flexibility. Supporting arbitrary ratios requires handling three stages at the same time.
- Tokenization and positional encoding: Images are split into patches or discrete tokens. Positional encoding must represent arbitrary grid widths and heights. Common approaches include interpolatable two-dimensional positional encoding or dynamically computing position indices based on the actual width and height.
- Training sampling: Training data is grouped by aspect ratio, and samples within the same batch have the same size. This ensures computational efficiency while covering multiple ratios.
- Inference constraints: The output size must be divisible by the model’s downsampling factor; otherwise, misalignment or artifacts may appear at the boundaries. The total pixel area and side-length range should also be limited to avoid exceeding the memory budget.
The following function aligns any target size to a grid multiple and limits the side-length range. The multiple in the example is for demonstration only; the actual value is determined by the model’s downsampling structure.
def snap_to_grid(width: int, height: int, multiple: int = 16,min_side: int = 512, max_side: int = 2048):"""Align the target size to a grid multiple and limit the side-length range."""def snap(v: int) -> int:return max(min_side, min(max_side, round(v / multiple) * multiple))return snap(width), snap(height)print(snap_to_grid(1240, 1600)) # (1248, 1600)
After alignment, you should also check whether the aspect-ratio error is within an acceptable range. If the business requires strict pixel dimensions, you can perform proportional cropping or padding after generation, leaving the error to post-processing.
5. Training on Domestic AI Compute: General Engineering Principles
The efficiency of large-model training is determined jointly by computation, communication, and the data pipeline. Regardless of which acceleration hardware is used, the following principles are general:
- Data pipeline parallelism: Overlap data reading, decoding, and augmentation with model computation so accelerators do not wait for data.
- Communication-compute overlap: Perform gradient synchronization asynchronously during backpropagation and use multiple execution streams to hide communication latency.
- Operator fusion: Merge small operators such as normalization, activation, and optimizer updates into fused operators to reduce memory access and scheduling overhead.
- Combining graph mode and dynamic graph: Keep flexibility during development and debugging, and apply compilation optimization to stable computation subgraphs to reduce dispatch overhead.
- Mixed precision and state management: Use low-precision computation and shard or offload optimizer states such as moving averages to control memory usage.
When migrating to new hardware, proceed in order. First, align operator correctness one by one, that is, compare output errors for the same input. Then, align the end-to-end training loss curve. Only after that should you tune performance. Skipping the first two steps and tuning directly often leads to an abnormal loss curve that is difficult to diagnose.
6. Deployment Recommendations: Poster and Cover Scenarios
When evaluating whether to introduce this type of model, proceed as follows.
- Split text hierarchy: Titles, subtitles, and small body text have different accuracy requirements and should be tested separately.
- Build a size checklist: Count the canvas specifications actually used by the business and verify output quality for each non-standard size.
- Use hybrid generation: For key text, use a “model-generated background plus programmatic text overlay” approach. The text layer is drawn precisely by a program, balancing controllability and visual quality.
- Complete compliance review: Check generated content for copyright, trademark, and sensitive-information issues, and retain a human review step. If open-source models are used, check the commercial terms in their licenses.
- Run regression tests: When upgrading model versions, use a fixed sample set to compare text accuracy and visual quality, so that metric degradation after an update does not go unnoticed.
7. Summary
Image generation is moving from “looking realistic” to “writing correctly,” which requires effort in architecture, data, and evaluation at the same time. A mixed architecture separates semantic understanding from pixel rendering, making problems easier to localize. Glyph-conditioned injection and synthetic data can improve Chinese text accuracy. How positional encoding and size alignment are handled determines whether the model can support arbitrary ratios. On the training side, data pipelines, communication overlap, and operator fusion are general means of improving efficiency. In deployment, text precision and size requirements in the business scenario should be the standard, and a quantifiable evaluation and regression mechanism should be established.
