Training Image Generation Models on Domestic AI Accelerators: A Full-Pipeline Technical Guide
Training image generation models on domestic AI accelerators requires the entire pipeline—data preprocessing, large-scale pretraining, supervised fine-tuning (SFT), and feedback-based reinforcement learning (RL)—to run stably and reach results comparable to mainstream solutions. This article breaks down the key technical decisions behind a hybrid architecture that pairs an autoregressive backbone with a diffusion decoder: how to inject explicit glyph conditions for accurate Chinese text rendering, how to redesign the image tokenizer and position encoding for native arbitrary-aspect-ratio generation, and how to keep large-scale training fast and stable through framework adaptation, pipelined operator dispatch, multi-stream communication-compute overlap, and hardware-affinity fused operators. It also presents a multi-dimensional evaluation framework for text rendering and generation quality, plus practical guidance for teams moving a model from demo to production on domestic compute.
1. Why Move Image Generation Training onto Domestic Compute
Multimodal generative models are becoming core infrastructure for content production, and whether training compute is autonomous and controllable directly determines the ceiling of technical progress. Over the past few years, training large-scale image generation models has relied heavily on a small number of accelerators. Whenever that supply changes pace, model iteration cycles get disrupted. As a result, establishing a complete “data-to-model” pipeline on domestic AI acceleration platforms is no longer just a supply-chain security concern—it is a critical test of whether the domestic software and hardware stack can carry frontier training workloads.
There is a premise here that is easy to overlook. To genuinely verify that a domestic training path is usable, running a small-scale demo is not enough. The entire pipeline—data preprocessing, large-scale pretraining, supervised fine-tuning (SFT), and feedback-based reinforcement learning (RL)—must run stably on domestic hardware and reach quality comparable to mainstream approaches. A missing operator or a framework compatibility issue in any single stage will cause training to halt or accuracy to regress.
Using a class of “cognition-oriented” image generation models as an example, the technical essentials can be broken down across four layers: architecture, text rendering, resolution, and training engineering. Cognition-oriented generation centers on combining a language model’s understanding and reasoning with image generation—the goal is not simply to “draw well,” but to “read the instruction, understand the semantics, and render the text correctly.”
2. Hybrid Architecture: Autoregressive Backbone + Diffusion Decoder
A pure diffusion model denoises an image directly from noise and has natural weaknesses in global semantic consistency and complex instruction following. Pure autoregressive per-pixel or per-token generation, on the other hand, struggles to balance detail quality. A mainstream approach in recent years is to combine the two into a hybrid architecture in which each module plays to its strengths:
- Autoregressive backbone: responsible for semantic understanding, instruction parsing, and global composition planning. It is essentially a language-model-style backbone that figures out what to draw, how to organize the scene, and what text to include, then emits a structured intermediate representation.
- Diffusion decoder: responsible for high-quality pixel restoration and detail rendering. It is typically implemented as a DiT (Diffusion Transformer), takes the conditioning information produced by the backbone, and reconstructs the semantics into a clear, coherent image.
This division of labor can be summed up in one line: the backbone “reads,” and the decoder “draws well and writes correctly.” Compared with a single end-to-end model, this decoupling lets each module be optimized independently—the backbone can reuse mature training and alignment practices from language models, while the decoder can focus on visual quality and text rendering—yet the overall architecture still supports unified training on domestic compute.
In engineering terms, the autoregressive backbone is usually the larger of the two (it carries language understanding), while the diffusion decoder is comparatively compact (it focuses on restoration quality), and the two connect through a conditional-injection interface. The exact sizes depend on the compute budget and quality goals; there is no single correct answer. What matters most is matching the backbone’s understanding power to the decoder’s restoration power.
3. Chinese Text Rendering: From “Tip-of-the-Pen Amnesia” to “Getting It Right”
Image generation models have long suffered from a pain point: text inside the picture often “doesn’t look right” or is outright fabricated. English already struggles with this, and Chinese is even harder because of its dense strokes, complex structure, and many visually similar characters—making it prone to missing strokes, deformation, and “tip-of-the-pen amnesia.” This is also an important yardstick for whether an image model is truly “usable in production.”
The usual solution is to introduce a dedicated text/glyph conditional module inside the decoder (often called a Glyph Encoder–style structure) that injects glyph information as an explicit additional condition, rather than hoping the model will “infer” glyph shapes from text tokens on its own. The key points are:
- Explicit glyph modeling: convert the text content into glyph-level visual conditions (such as a rendering skeleton or glyph description) and feed them to the decoder alongside the semantic conditions, so the model “writes stroke by stroke” instead of “drawing by feel.”
- Text token and visual token coordination: the autoregressive backbone first decides “which characters to write and where,” and the decoder then guarantees “the characters are written correctly and clearly.”
- Long-text consistency: for posters and covers with large blocks of text, the layout, font size, and letter spacing of multi-line text must stay coherent, avoiding cases that are locally correct but globally disordered.
Precisely because text rendering demands all three abilities—understanding, planning, and restoration—it is the key scenario for judging whether a hybrid architecture succeeds: only when the backbone reads correctly and the decoder writes correctly can Chinese posters and infographics come out production-ready on the first pass.
4. Arbitrary-Aspect-Ratio Resolution: The Tokenizer Strategy Is Key
Most image generation models are limited to a few preset aspect ratios, such as 1:1, 4:3, and 16:9. But real business scenarios—posters, social media covers, banners, vertical long images—often need non-standard resolutions, for example a vertical poster at 1240×1600. If a model can only output fixed ratios, users have to crop or stretch the result afterward, which directly hurts the quality of the final output.
To support arbitrary ratios, the core work is reworking the tokenizer (image tokenizer) and the position-encoding strategy so that they are decoupled from resolution:
- Ratio-agnostic image tokenization: the way an image is split into patches should not be tied to a fixed width and height; it should adapt to different aspect ratios so that any size maps to a reasonable token sequence.
- 2D position-encoding extrapolation: adopt an extrapolatable position-encoding scheme so the model can still generate stably at sizes it never saw during training.
- Resolution diversity in training: mix multiple ratios and sizes in the data to avoid overfitting to a single aspect ratio.
A simplified processing logic can be understood as follows (pseudocode, illustrative of the idea only):
def build_image_tokens(image, target_patch=16):# Key: the patch grid adapts to width and height, not a fixed square gridgrid_h = ceil(image.height / target_patch)grid_w = ceil(image.width / target_patch)patches = adaptive_patchify(image, grid_h, grid_w)# Extrapolatable position encoding supports resolutions not seen in trainingpos = rope_2d_positions(grid_h, grid_w)tokens = embed(patches) + posreturn tokens, (grid_h, grid_w)
With these changes, a model can natively support arbitrary-aspect-ratio generation over a fairly wide range (for example, from 1024×1024 up to 2048×2048), which is especially practical for posters and covers that need custom sizes.
5. Training Engineering Optimization: Making Domestic Compute Truly Saturate
Completing full-pipeline training of a large model on domestic hardware is rarely about “whether it can run”—the real challenge is “whether it can run fast and stable.” A successful deployment usually requires effort at the same time across framework adaptation, parallelism strategy, and operators. The following is a general-purpose set of optimization ideas (not a feature unique to any single product):
- Framework and training-suite adaptation: build an integrated training suite that covers “data preprocessing → pretraining → SFT → RL” on top of mainstream deep learning frameworks (including domestic in-house frameworks), reducing the cost of moving between frameworks.
- Relieving the dispatch bottleneck: use dynamic-graph, multi-level pipelined operator dispatch to spread scheduling overhead across multiple levels, so a single dispatch point does not become a throughput bottleneck.
- Communication-compute mutual concealment: through multi-stream parallelism (separating the compute stream from the communication stream), let parameter synchronization and gradient reduction overlap with forward and backward computation to raise overall cluster utilization.
- Ascend-affinity / hardware-affinity fused operators: fuse high-frequency operators such as the optimizer and normalization (for example, merging the optimizer update with its state maintenance, and fusing normalization like RMSNorm) to cut memory-access and kernel-launch overhead. These fused operators must be co-tuned with the specific hardware instruction set, and they are a key detail in whether a domestic platform’s performance can match the mainstream.
It is worth stressing that these optimizations do not exist in isolation—they are coupled. Operator fusion reduces single-card compute time, multi-stream parallelism conceals cross-card communication, and pipelined dispatch keeps scheduling from blocking. Only when all three stack together can large-scale pretraining reach usable efficiency on a domestic cluster.
From an infrastructure standpoint, such training jobs also typically rely on general-purpose capabilities from a cloud platform to guarantee stability—for example, object storage (holding massive training data and checkpoints), containerized scheduling (managing large-scale distributed tasks), monitoring and alerting (detecting dropped cards and accuracy anomalies in real time), and logging services (tracking the training trajectory). Baidu AI Cloud and similar platforms offer these kinds of general-purpose capability categories, which can serve as a foundation reference for hosting the training engineering.
6. How to Evaluate Text Rendering and Generation Quality
To objectively judge whether an image generation model is usable, looking at a handful of sample images is far from enough; multi-dimensional evaluation is usually needed:
- Complex visual text generation evaluation: focus on the accuracy, readability, and layout reasonableness of text in the image, especially for high-difficulty scenarios like Chinese.
- Long-text rendering evaluation: assess the completeness and consistency of large, multi-line text under long-text conditions.
- General image quality and aesthetics: including both subjective and objective metrics such as composition, color, detail, and texture.
- Instruction-following ability: verify whether the model can accurately understand and execute complex, multi-constraint descriptions.
- Robustness and stability: consistency across different random seeds, different resolutions, and different prompts.
A model that leads in dedicated text-rendering evaluation while maintaining strong general generation quality is closer to a “ready for direct production” level; and support for arbitrary resolution is a hidden bonus in real business that often decides “whether it can be used as-is.”
7. Summary and Practical Recommendations
Moving the entire training pipeline of an image generation model onto domestic compute is a comprehensive challenge that spans architecture design, algorithmic capability, systems engineering, and hardware adaptation. In summary:
- Architecture: the hybrid approach of autoregressive backbone plus diffusion decoder balances semantic understanding with detail restoration and is the mainstream path for cognition-oriented generation.
- Capability: Chinese text rendering depends on explicit glyph-condition injection rather than hoping the model learns it on its own.
- Usability: native arbitrary-aspect-ratio generation significantly lowers the cost of putting the model into business.
- Engineering: only by systematically doing framework adaptation, pipelined dispatch, multi-stream concealment, and operator fusion well can domestic compute genuinely carry frontier training.
For teams planning to train multimodal generation models on a domestic platform, the recommendation is to get an end-to-end minimal loop working first, then scale up step by step, and to establish an evaluation system for text rendering and resolution robustness as early as possible—because those two capabilities are often the dividing line between a model that stays a “demo” and one that reaches “production.”
