Efficient Training Practices for Image Generation on Domestic AI Chips
As image generation transitions from being visually appealing to practically usable, text rendering remains a significant bottleneck due to issues like missing strokes, character deformation, and misaligned long sentences. This article presents a reusable framework for end-to-end training on domestic AI chips, covering problem definition, model architecture, data and evaluation, training strategies, and engineering deployment. It details how to achieve accurate text rendering, stable long-text generation, and high-resolution outputs with arbitrary aspect ratios while maintaining acceptable training throughput on local hardware.
As image generation evolves from “looking good” to “being usable,” text rendering remains a persistent shortcoming. Issues such as missing strokes, deformed characters, and misaligned long sentences make it difficult to deploy these models in scenarios like posters, cover arts, and infographics. Meanwhile, an increasing number of teams aim to complete the full training pipeline—from pre-training to alignment—on domestic AI chips. This reduces reliance on external computing power and facilitates integration with existing object storage, container platforms, and logging and monitoring systems. This article provides a reusable framework covering problem definition, models and data, training strategies, evaluation, and engineering deployment.
1. Problem Definition: Controllable Text Generation and Computing Constraints
Text in image generation is fundamentally an alignment between “visual symbol sequences” and “semantic intent.” The model must understand the semantic and layout constraints in the prompt while stably rendering characters, font sizes, line spacing, and alignment in the pixel space. Because of the dense strokes and diverse structures of Chinese characters, the requirements for resolution, alignment precision, and post-processing fault tolerance are even higher. Training on domestic AI chips adds another layer of constraints: operator coverage and precision paths, communication topologies, memory and bandwidth ratios, and the maturity of compilation and graph optimization toolchains all directly impact whether training with “high resolution + strong text constraints” can run stably. Therefore, the goal is not to optimize a single metric, but to simultaneously satisfy the following on local hardware: text accuracy, long-text stability without degradation, arbitrary aspect ratio generation, and acceptable training throughput.
2. Model Architecture: Making Text Rendering an Explicitly Learnable Capability
Relying solely on general diffusion or autoregressive image heads often treats text as texture, lacking character-level constraints. In practice, the text pathway can be made explicit: use a text encoder to provide character-level or word-level conditions, and inject them into the visual backbone via cross-attention or gating mechanisms. For scenarios requiring precise typography, layout tokens (position, font size, alignment) can be introduced as additional conditions. On the decoding side, a multi-stage structure of “low-resolution semantics first, then high-resolution character refinement” can be adopted, allowing the model to stabilize content and composition before supplementing stroke details. For the loss function, in addition to reconstruction and perceptual losses, a glyph consistency term can be added: apply differentiable or approximate OCR to the generated image, and calculate the edit distance or character classification loss against the target string, directly incorporating “writing correctly” into the optimization. For long-text scenarios, the position encoding and relative position bias must cover sufficient lengths, and higher weights should be given to text regions in the attention mechanism to prevent the latter half of the text from blurring.
3. Data and Evaluation: Closing the Loop on Text Quality with Quantitative Metrics
Data quality determines the ceiling. A “prompt–layout–image–OCR verification” quadruple should be constructed: text in images must strictly match annotations, covering various font sizes, fonts, background contrasts, and multilingual mixed layouts. Rendering engines can be used to synthesize a large number of controllable samples, followed by domain adaptation using real posters and covers. Negative samples are equally important: samples with missing strokes, wrong characters, or scrambled sequences should be used for contrastive learning to reduce the model’s tendency to hallucinate meaningless characters. Evaluation should not rely solely on subjective perception; a fixed test set should be used to track character accuracy, normalized edit distance, long-text completeness rate, and multi-line alignment error. Complex visual text and long-text rendering can be treated as two independent dimensions: the former emphasizes complex backgrounds and artistic fonts, while the latter focuses on length and line-break stability. Every training iteration should align with these metrics to prevent the scenario where “the image becomes more beautiful, but the text becomes worse.”
4. Efficient Training Paths on Domestic AI Chips
The full training pipeline typically includes large-scale pre-training, text-specific fine-tuning, high-resolution continued training, and alignment. On domestic AI chips, it is first necessary to ensure that key operators (attention, convolution, normalization, and approximate OCR in the loss function) are available at the target precision, prioritizing mixed precision and recomputation to save memory. For distributed training, batch sizes can be split using data parallelism, while text conditions and the visual backbone can be divided using pipeline or tensor parallelism. In terms of communication, reduce the frequency of cross-node synchronization and use gradient accumulation to achieve an effective batch size. Resolution curriculum is critical: first stabilize semantics and text skeletons at 512 or 768, then scale up to 1024 and 2048 to avoid throughput crashes from starting directly at high resolutions. Compilation and graph optimization should solidify common shapes (standard aspect ratios) to reduce dynamic shape overhead. During training, metrics such as loss, character accuracy, throughput, and memory usage should be integrated into monitoring, alerting, and logging services, enabling quick rollbacks if NaN or cliff drops in text metrics occur. Checkpoints and datasets should be stored in object storage, and the container platform should unify images and scheduling to ensure reproducibility.
5. Inference and Productization: Arbitrary Aspect Ratios and Cost Control
Once training is complete, inference must support resolutions from 1024 to 2048 and arbitrary aspect ratios in between, preventing secondary cropping from destroying text. In implementation, variable resolution position encoding or patch-wise decoding can be used, with increased sampling steps or guidance strength for text regions. Post-processing can involve a lightweight OCR verification to locally redraw obvious typos, but the primary quality should still come from the model itself. For cost control, domestic AI chips can leverage quantization, distillation, caching text conditions, and batching requests with the same layout to keep single-image generation within acceptable latency and cost limits. When providing a unified API, parameters such as size, seed, negative prompt, and text emphasis weight should be exposed to facilitate template-based calls for covers, posters, and infographics.
6. Application Scenarios and Engineering Suggestions
Social media covers require large titles, strong contrast, and the fusion of characters and landscapes: prompts should clearly state the main title’s word count, position, and color scheme, and the model should use layout conditions to lock positions. Commercial posters emphasize brand colors and white space, which can be managed using reference images plus text layer conditions. Infographics involve multi-paragraph explanatory text plus diagrams, making them suitable for long-text specific models. Regardless of the scenario, prompt engineering should be structured: subject, raw text (ideally given character by character), layout, style, and taboos. Teams are advised to build a “generation–OCR–manual sampling” pipeline, feeding high-frequency typos back into the training set. For security and compliance, sensitive words and false advertising texts should be intercepted, and generated images should have watermarks or metadata declarations.
7. Conclusion
Building image generation into a multimodal capability that “can write correctly” on domestic AI chips relies not on slogans, but on explicit text pathways, quantifiable text losses, curriculum-based resolution training, and an engineering closed loop integrated with storage, containers, and monitoring. By following this path, local hardware can simultaneously achieve text accuracy, long-text stability, and high-resolution arbitrary aspect ratio outputs, moving covers, posters, and infographics from demos to scalable production. Future work can refine layout control further (letter spacing, vertical text, text wrap) and integrate it with Retrieval-Augmented Generation (RAG) to reduce the model’s tendency to fabricate characters.
