Text-to-Image on Domestic AI Chips: Chinese Text Rendering and Arbitrary Resolution in Practice
This article examines how to train a text-to-image model on domestically developed AI chips that combines strong Chinese text rendering with arbitrary-resolution output. It explains why accurate text rendering, especially for Chinese, remains the hardest part of image generation, walks through a hybrid architecture that pairs an autoregressive model for semantic understanding with a diffusion decoder for pixel-level detail, and covers the Tokenizer and positional-encoding changes required for non-standard aspect ratios. It also outlines full-pipeline training on domestic accelerators, reproducible preconditions, three layers of engineering optimization, a multi-dimensional evaluation framework, deployment considerations, and a practical engineering checklist. All parameter counts are illustrative examples rather than universal standards.
In recent years, two main threads have advanced in parallel across multimodal generation. The first is a shift in model capability, from “being able to generate images” to “understanding semantics and rendering text correctly.” The second is a shift in training compute, from relying on a single hardware source toward self-controlled, domestically developed silicon. Where these two threads meet, a question emerges that carries both engineering and ecosystem value: how do you train a text-to-image model on domestic AI chips that renders Chinese text with high fidelity and supports arbitrary-resolution output?
This article does not target any specific product. Instead, it breaks down the architecture choices, training engineering, evaluation methods, and deployment considerations behind this class of model from a general technical perspective. All parameter counts mentioned here are illustrative examples and do not represent universal standards.
1. Why text rendering is the real hard part of text-to-image
Text-to-image models are relatively mature at generating landscapes, portraits, and illustrations, but producing accurate text within an image, especially Chinese text, remains a widely recognized difficulty. It helps to view the problem through three lenses: the problem, its impact, and the countermeasure.
1.1 Large character set: problem and countermeasure
Problem: Chinese has a large set of commonly used characters, and once you add punctuation, English letters, digits, and special symbols, the glyph coverage the model must handle is far larger than the Latin alphabet.
Impact: The model tends to “blank on characters,” dropping strokes, inventing wrong characters, or confusing visually similar glyphs, which undermines the usability of real materials such as posters, menus, and instruction cards.
Countermeasure: Supply high-quality glyph training data that covers high-frequency characters, low-frequency characters, error-prone characters, and complex combinations. It also helps to introduce a glyph encoder that explicitly injects text content, stroke structure, and layout constraints into the generation process.
1.2 Complex glyph structure: problem and countermeasure
Problem: Chinese characters are two-dimensional block structures with dense strokes, radicals, and spatial relationships, making pixel-level modeling difficult.
Impact: Once a result becomes blurry, misaligned, or distorted, it can degrade into unrecognizable gibberish. The problem grows more pronounced after scaling, compression, or low-resolution rendering.
Countermeasure: Increase the training weight on text regions, use high-resolution text samples, and reinforce glyph detail during decoding. Complex backgrounds, low-contrast scenes, and occluded regions need their own training strategies.
1.3 Poor long-text consistency: problem and countermeasure
Problem: When an image must render a whole passage, such as poster copy, product selling points, or a menu list, the model has to keep font style consistent, alignment correct, and semantics coherent across a long sequence.
Impact: Getting individual characters right does not guarantee a readable passage. Long text often shows abnormal line spacing, drifting letter spacing, misaligned left-right structures, and quality that degrades in the latter half.
Countermeasure: Strengthen long-range modeling, train on multi-line text samples, and add layout constraints such as line alignment, column alignment, boundary constraints, and style-consistency constraints.
Because of this, judging whether a text-to-image model is “usable” cannot rest on visual appeal alone. It also depends on how the model performs on complex text-in-scene generation, long-text rendering, mixed Chinese-English layouts, and text over complex backgrounds. These capabilities often determine whether real business needs such as poster design, e-commerce creatives, and social media covers can actually ship.
2. Architecture: an autoregressive model to “understand,” a diffusion decoder to “write correctly”
To solve semantic understanding and precise text rendering at the same time, the industry is gradually converging on a “hybrid architecture”: an autoregressive model handles semantic understanding and global composition, while a diffusion decoder handles pixel-level detail and text rendering. Each does its own job.
Think of it as a pipeline:
Autoregressive part: the semantic brain
An autoregressive backbone understands the semantics of the text prompt and plans “what to draw, where to draw it, and how to compose the overall scene.” Its strength is strong modeling of language and global structure, translating an abstract description into structured generation intent.
For example, some sample configurations use an autoregressive backbone of roughly 9 billion parameters. This figure is only an illustrative reference for convenience, not a universal standard. Without a model card, experiment logs, or a public source, it should not be treated as a fixed metric.Diffusion decoder part: the pixel brush
The decoding stage can use a diffusion Transformer to “paint” the intermediate representation from the autoregressive model into a high-resolution image. Diffusion models have natural strengths in local detail, texture, and sharpness, compensating for the autoregressive model’s weakness in pixel-level precision.
For example, some sample configurations use a diffusion Transformer of roughly 7 billion parameters. Again, this number is illustrative; actual parameter counts should follow the specific training configuration.Glyph encoder: reinforcing “writing the right character”
The glyph encoder is dedicated to strengthening text rendering, injecting text content, glyph structure, and layout position into decoding so the model does more than “draw the shape of the characters”; it tries to “write them correctly.”
This division of labor amounts to “cognitive generation”: the language side figures out what to make, and the visual side draws it.
The following is conceptual pseudocode that illustrates the flow only and does not represent any specific implementation:
# Conceptual sketch: autoregressive + diffusion decoder hybrid generation flowdef generate_image(prompt, canvas_size, glyph_input):# 1. Autoregressive backbone: understand semantics + plan global compositionlayout = autoregressive_model.plan(prompt) # structured generation intent# 2. Glyph encoder: inject text content and glyph featuresglyph_feature = glyph_encoder.encode(glyph_input) # strengthen text rendering# 3. Diffusion decoder: paint at pixel level, output image at the target sizeimage = diffusion_decoder.decode(layout, # semantic and layout conditionsglyph_feature, # text / glyph conditionssize=canvas_size # supports non-standard resolutions)return image
In real engineering, pseudocode cannot go straight to production; you still need to add input validation and dependency notes.
Input validation and dependency notes
- Prompt constraints: The prompt must be non-empty and should have a maximum length. Filter out sensitive, non-compliant, or high-risk content. If the prompt contains text to be rendered, extract it into a structured field rather than concatenating it into a long sentence.
- Canvas size range: You must define the minimum and maximum resolution, the allowed aspect ratios, and the size step or quantization granularity. For example, you might restrict the range from 1024×1024 to 2048×2048, but the specific range should be determined by VRAM, throughput, and business needs.
- Glyph input format: Text content, font style, font size, color, target region, and alignment should be passed in a structured form. If you use a detection model or a layout module, define the coordinate system, scaling method, and fallback strategy on failure.
- VRAM estimation: VRAM covers not only model weights but also activations, gradients, optimizer states, communication buffers, KV cache, or feature caches. Before going live, estimate peak VRAM through small-scale stress testing rather than judging by parameter count alone.
- Error handling: When a size exceeds limits, glyph input is missing, the prompt is empty, VRAM is insufficient, or decoding times out, return a clear error code and provide a fallback, for example lowering the resolution, generating in tiles, skipping the text layer, or prompting the user to adjust parameters.
This “semantic side + pixel side” decoupling delivers two direct benefits: text rendering quality is easier to improve, and there is room for swapping modules, optimizing them independently, and extending resolution support later.
3. Arbitrary resolution: why non-standard sizes matter
Traditional text-to-image models usually support only a few preset ratios, such as 4:3, 16:9, and 1:1. But the asset sizes in real business scenarios are often “non-standard”: an e-commerce product detail hero image might be 1240×1600, a vertical feed cover might be 1080×1440, and a poster might be any combination of 2:3, 3:4, or 9:16.
Supporting arbitrary ratios and sizes comes down to the Tokenizer and positional-encoding strategy. To make the model work across a continuous space of sizes, rather than forcing the canvas to be cropped or padded to a fixed ratio, you typically need to do the following.
3.1 Define implementation boundaries
First, define the size range the business supports instead of claiming “arbitrary size” with no boundaries. At minimum, define:
- Minimum and maximum resolution;
- Allowed aspect-ratio range;
- Minimum and maximum total pixel count;
- Continuous sizes or discrete steps;
- Fallback strategy when limits are exceeded.
3.2 Improve the Tokenizer strategy
Image tiling and serialization should adapt to different aspect ratios. Common approaches include adaptive patch partitioning, dynamic-resolution encoding, and preserving aspect-ratio information instead of forcing scaling. The goal is to avoid stretching, padding, semantic misalignment, or clipped text under non-standard ratios.
3.3 Optimize positional encoding and attention design
Under different resolutions, both global and local consistency can be affected. You can use 2D positional encoding, pluggable positional encoding, or frequency interpolation, and cover multiple aspect ratios during training to reduce the model’s poor adaptation to non-standard canvases.
3.4 Cover size diversity during training
If the model sees only a few fixed ratios during training, it easily degrades on non-standard sizes at inference time. Training should cover a wide enough size distribution, including common business ratios, extreme ratios, and continuous size sampling.
3.5 Evaluate the VRAM and throughput impact
Higher resolution significantly increases VRAM and compute, especially for attention. Engineering teams should assess peak VRAM, per-step time, and throughput changes in advance. When necessary, use sequence parallelism, tiled decoding, gradient checkpointing, and low-precision inference.
3.6 Fallback when limits are exceeded
Arbitrary resolution does not mean infinitely large. When a user’s input size exceeds the model’s capability, provide a stable fallback, such as:
- Automatically scaling down proportionally into the supported range;
- Generating in tiles and stitching them together;
- Reducing texture complexity or quality level;
- Returning a clear error and prompting the user to adjust the size.
For developers, the value of arbitrary-resolution capability is this: for posters, social media covers, and product hero images, you can generate directly at the target placement size, avoiding the composition damage and clipped text that come from secondary cropping.
4. Full-pipeline training on domestic AI chips: engineering optimization is key
Completing full-pipeline training of a multimodal large model on domestic AI chips, rather than only doing inference or migrating part of the pipeline, is the most core and most ecosystem-meaningful part of this kind of work. The difficulty often lies not in the algorithms themselves but in getting the entire training chain to run end to end and tuning it, including data preprocessing, large-scale pretraining, supervised fine-tuning, and reinforcement learning.
You typically need to build or adapt, on top of a general deep learning framework, a training suite that covers the end-to-end pipeline. To make the approach more reproducible, it is worth putting together a preconditions checklist before training.
Reproducibility preconditions checklist
| Item | What to record |
|---|---|
| Chip model | Chip name, architecture, VRAM capacity, interconnect |
| Framework versions | Deep learning framework, driver, compiler, operator library versions |
| Cluster scale | Node count, cards per node, total cards, network topology |
| VRAM configuration | Per-card VRAM, peak VRAM, VRAM optimization strategies |
| Parallelism strategy | Combinations of data parallelism, tensor parallelism, pipeline parallelism, sequence parallelism, etc. |
| Data format | Image format, text annotation format, storage format, encoding |
| Resolution distribution | Training size coverage, aspect-ratio distribution, resolution sampling strategy |
| Key log metrics | Loss, gradient norm, throughput, step time, VRAM usage, communication ratio, OOM count |
Without these preconditions, performance results are hard to reproduce, and it is difficult to tell whether an optimization is genuinely effective.
Three layers of engineering optimization
Scheduling layer: relieving the dispatch bottleneck
In large-model training, the efficiency of dispatching instructions from the host to the device often becomes a bottleneck. Multi-level pipelined dispatch of the dynamic graph splits the dispatch process into multiple pipelined stages, letting instruction dispatch overlap more fully with device compute and reducing device idle time.Communication and compute: overlapping multiple streams
In distributed training, communication and compute often wait on each other. Multi-stream parallel scheduling overlaps operations such as gradient synchronization and tensor-parallel communication with computation, improving device utilization and shortening per-step time.Operator layer: hardware-affine fused operators
Training performance depends heavily on whether operators fit the hardware’s characteristics. You can use fused operators affine to the target chip’s architecture, for example fusing optimizer updates, exponential moving average, and normalization, to reduce memory-access overhead and improve throughput.
How to verify whether an optimization works
Optimization should not rely on intuition; verify it with controlled experiments. At minimum, record:
- Per-step time before and after optimization;
- Samples per second (throughput);
- Peak VRAM;
- Communication time ratio;
- Device utilization;
- Long-run training stability;
- Whether the loss curve matches the baseline;
- Whether precision drift or abnormal NaN values appear.
Algorithms set the ceiling on capability; engineering optimization determines whether that capability can actually be trained on domestic compute, and trained in a way that runs and runs fast.
5. Results and evaluation: how to measure this kind of model
You cannot evaluate a text-to-image model by whether a single image looks good. For models with strong text-rendering ability, you especially need a multi-dimensional evaluation. Below is a suggested evaluation table.
| Evaluation dimension | What to observe | Suggested pass criterion | Common errors |
|---|---|---|---|
| Complex text-in-scene generation | Readability of multi-line text, mixed Chinese-English, and text over complex backgrounds | Human spot-check acceptance rate at or above the business threshold; higher for critical business | Missing strokes, wrong characters, stuck-together glyphs |
| Long-text rendering | Font consistency, line spacing, column alignment, layout completeness | Text blocks remain legible overall, with no large-scale misalignment or truncation | Abnormal line spacing, quality drop in the latter half |
| Image quality | Sharpness, texture realism, artifacts, structural soundness | No obvious structural breakdown; main subject stays clear | Blurring, distortion, repeated textures |
| Semantic alignment | Whether the generated content matches the prompt | Subject, style, and text content align with the prompt | Missing subject, wrong text content |
| Resolution adaptation | Whether composition and text stay complete at non-standard sizes | Usable at the target size, without severe secondary cropping | Stretching, padding, clipped text |
| Engineering performance | Throughput, VRAM, latency, stability | Meets online SLA and resource budget | OOM, timeouts, high tail latency |
It is worth stressing that this article makes no unverifiable ranking or “leading” claims. If you cite a public evaluation, state the evaluator, the evaluation set, the time, the model version, the prompt settings, and the reproduction conditions. A statement like “performs well on a specific public benchmark” only holds once its source and basis are clear; when it cannot be verified, delete it or replace it with a neutral description.
Also, leaderboards are only a reference. In real business, you should run targeted validation against your own datasets, especially business high-frequency words, brand terms, product names, campaign copy, and specific font styles.
6. Deployment and engineering support
For this kind of model to reach application, beyond the model itself, you need a full set of supporting engineering capabilities. Whether to use cloud services is not the focus of this article, but if cloud deployment is involved, you can choose appropriate general-purpose capabilities based on business scale, latency requirements, and compliance requirements.
Selection considerations
- Object storage: Suitable for storing training data, model weights, and generated assets. When selecting, consider data scale, read/write throughput, hot/cold tiering, lifecycle management, access control, and compliance auditing.
- Container platform: Suitable for elastic scheduling of training jobs and inference services. Consider GPU/NPU scheduling, quota management, image distribution, elastic scaling, and fault recovery.
- Message queue: Suitable for decoupling asynchronous tasks such as batch image generation, queued rendering, and failed-job retries. Consider peak throughput, message reliability, ordering requirements, and backlog monitoring.
- Logging and monitoring/alerting: Used to observe training loss, throughput, VRAM usage, inference latency, error rates, and resource utilization. Consider log retrieval cost, metric dimensions, alerting strategy, and long-term retention needs.
If content generation is involved, add content moderation for generated output, licensed-asset management, sensitive-word filtering, and a result-labeling mechanism. Evaluate specific capabilities against your own business scale and compliance requirements, and do not treat any single vendor’s capability as the only option.
7. FAQ
7.1 What are the common Chinese text-rendering errors?
Common problems include missing strokes, extra strokes, misaligned left-right structures, unbalanced top-bottom structures, confusion between similar glyphs, layout drift across a passage, and unreadable text over complex backgrounds. Improvements include adding glyph data, strengthening the glyph encoder, adding layout constraints, increasing the training weight on text regions, and building a dedicated evaluation set.
7.2 How do you optimize VRAM for arbitrary resolution?
Approaches include limiting the maximum total pixel count; using tiled decoding; enabling gradient checkpointing; enabling mixed precision; reducing batch size; using sequence parallelism or feature partitioning; and automatically scaling down oversized inputs. Before going live, determine peak VRAM through stress testing rather than estimating model weights alone.
7.3 What acceptance metrics matter for domestic-chip training?
At minimum: whether training converges stably; whether the loss curve matches the baseline; whether throughput hits the target; whether peak VRAM is under control; whether long runs produce OOM or NaN; whether key operators show precision deviation; whether logs are complete; and whether per-step time can be quantitatively compared before and after optimization.
7.4 How do you judge whether domestic-chip migration is truly complete?
“It runs” is not the same as “done.” True completion means end-to-end training with reproducibility, observability, and verifiable performance. Run a controlled comparison in a reference environment with the same data and configuration, and confirm that both training results and engineering metrics meet expectations.
8. Engineering checklist
| Stage | Check item | Acceptance point |
|---|---|---|
| Data preparation | Deduplication, resolution distribution, text annotation, sensitive-content filtering | Data covers business scenarios, with no severe contamination |
| Training validation | Chip model, framework versions, parallelism strategy, log metrics | Preconditions complete; loss and throughput traceable |
| Inference deployment | Size validation, timeout retries, fallback strategy, content moderation | Non-standard sizes usable; clear returns on errors |
| Content compliance | Licensed assets, generated-content moderation, sensitive-prompt filtering | Risk under control, with logging and audit capability |
Conclusion
The significance of “completing full-pipeline training of a multimodal model on domestic AI chips” lies not only in getting one model to run, but in validating the feasibility of a path: self-controlled compute plus a mature generation architecture plus deep engineering optimization. For developers, understanding the hybrid design of autoregressive and diffusion decoders, the Tokenizer changes behind arbitrary resolution, the dedicated modeling for Chinese text rendering, and the optimization and acceptance methods for domestic-chip training holds more long-term value than memorizing any specific model.
As the domestic-compute ecosystem keeps maturing, this ability to “understand semantics, write the right characters, and generate on demand” will appear more and more in real production environments.
