End-to-End Training of an Open-Source Image Generation Model on Domestic Compute Platforms
This article describes an engineering practice for training a multimodal image generation model end to end on domestic AI compute platforms. It covers the decoupling of semantic understanding and pixel rendering, data preprocessing and tokenization, large-scale pretraining, supervised fine-tuning, reinforcement learning, evaluation loops, pipeline and parallel optimization, arbitrary-aspect-ratio generation, Chinese text rendering, and open-source release and deployment considerations. The goal is to present a reproducible, auditable, and extensible technical path rather than to promote or evaluate any specific third-party product.
End-to-End Training of an Open-Source Image Generation Model on Domestic Compute Platforms
1. Background: Image Generation Models Enter an Engineering Training Phase
Competition in multimodal image generation is shifting from single-point demonstrations to full engineering pipelines. Whether a model can consistently produce commercial posters, educational illustrations, and social media covers depends not only on network architecture, but also on whether data preprocessing, large-scale pretraining, supervised fine-tuning, reinforcement learning, inference deployment, and evaluation loops can run end to end. Completing the full training process on domestic AI compute platforms involves framework adaptation, operator optimization, parallel strategies, and communication-computation overlap. The engineering difficulty is no lower than that of model architecture design itself.
The core signal from the original practice is that an image generation model can complete the full training pipeline—from data preprocessing to large-scale pretraining—on domestic compute platforms and be released as open source. The value lies not only in “training a model,” but also in validating a reproducible, auditable, and extensible engineering path. For developers, this means they can use public code and model weights to study hybrid architectures, text rendering, arbitrary-aspect-ratio generation, and other key capabilities. For enterprise users, this provides a more controllable technical reference for scenarios such as custom-size posters and direct Chinese text generation.
This article discusses general technical approaches and practices. It does not promote or evaluate any specific third-party product. The question it addresses is: how can the full training pipeline of a multimodal image generation model be implemented solidly on domestic compute platforms?
2. Goals and Design Principles: Decoupling Semantic Understanding from Pixel Rendering
Image generation models commonly face three pain points. First, semantic understanding may be insufficient: outputs may “look plausible” but have unreasonable layouts. Second, text rendering can be unstable, especially for long Chinese text, which may suffer from missing strokes, incorrect characters, or malformed glyphs. Third, resolution is often constrained: traditional models are mostly trained on preset aspect ratios, so non-standard sizes require cropping, stretching, or outpainting, which affects the usability of posters and covers.
An effective design principle for these problems is to decouple cognition from rendering: let an autoregressive module handle semantic understanding, global composition, and layout planning, while a diffusion decoder handles pixel-level detail generation and text rendering. The two are connected through an intermediate representation, forming a pipeline that first understands the content and then renders it correctly. The autoregressive part can use a scale of roughly one billion parameters and handle joint modeling of text and image tokens. The diffusion decoder can adopt a diffusion Transformer structure and introduce a glyph encoder to provide text-shape priors. The direct benefit is that the semantic module does not need to handle all pixel details, and the decoder does not need to understand complex instructions from scratch. Each component focuses on its own role, reducing training difficulty and improving controllability.
This approach is consistent with the industry direction of “cognitive generation”: combining the reasoning capability of language models with image generation capability so that the model can not only draw, but also understand what to draw, where to place it, and how the text should be written. From an engineering perspective, this hybrid architecture also makes staged training and problem isolation easier: semantic errors and rendering errors can be investigated separately, avoiding the “one error causes all errors” problem of a single end-to-end model.
3. Full Training Pipeline: From Data Preprocessing to Reinforcement Learning
Full-process training is not simply calling a training script. It covers a complete chain that includes data cleaning, resolution bucketing, tokenization, pretraining, supervised fine-tuning, reinforcement learning, and evaluation. A practical training pipeline usually includes the following stages:
- Data preprocessing: Clean image-text pairs and filter low-quality, duplicated, or violating data. Detect and annotate text regions to build glyph and layout priors. Perform resolution bucketing by aspect ratio to avoid damaging composition through uniform resizing.
- Image tokenization: Encode images into discrete or continuous tokens while preserving aspect-ratio information. For non-standard resolutions, use dynamic grid splitting and position-encoding interpolation so that inputs with arbitrary aspect ratios can be understood by the model.
- Large-scale pretraining: Train the semantic and layout module with an autoregressive objective, and train the decoder with a diffusion objective. The two stages can alternate or be jointly optimized, but loss weights and gradient scales must be controlled.
- Supervised fine-tuning: Introduce high-quality poster, cover, and illustration data to strengthen text rendering and layout control. Add dedicated samples for long Chinese text, covering multiple fonts, font sizes, and background complexity levels.
- Reinforcement learning: Build reward signals from metrics such as readability, text accuracy, layout reasonableness, and aesthetic preference, and perform preference optimization. Attention must be paid to reward-model bias and training stability.
- Evaluation loop: Establish evaluation sets for complex visual text generation, long-text rendering, and non-standard-resolution usability, with fixed sampling and manual review procedures.
An illustrative training pseudocode is as follows:
def train_step(batch):text_tokens, image_tokens, glyph_feats = batchsemantic = ar_module(text_tokens, image_tokens)ar_loss = ar_loss_fn(semantic, image_tokens)latents = diffusion_decoder(semantic, glyph_feats)diff_loss = diffusion_loss_fn(latents, image_tokens)loss = ar_loss + diff_weight * diff_lossreturn loss
The pseudocode above only expresses structural relationships. Actual engineering work also needs to handle mixed precision, gradient accumulation, checkpoint resume, data loading bottlenecks, and multi-machine multi-GPU communication.
4. Engineering Optimization: Pipelines, Parallelism, and Fused Operators
When training multimodal models on domestic compute platforms, engineering optimization usually focuses on three areas: dispatch efficiency, communication-computation overlap, and operator adaptation.
First, dynamic-graph multi-level pipeline dispatch. A large number of small operators and dynamic control flow in training scripts can create dispatch bottlenecks and leave compute resources idle. Organizing the process into multi-level pipelines—splitting data loading, forward computation, backward computation, and parameter updates into overlapping stages—can reduce waiting time.
Second, multi-stream parallelism and communication-computation masking. In multi-machine multi-GPU training, gradient synchronization and parameter communication are major overheads. Multi-stream parallelism can schedule communication tasks and computation tasks on different streams, allowing communication to be partially hidden during computation and improving overall throughput. The key is to divide the computation graph reasonably and control stream dependencies and synchronization points, avoiding extra scheduling overhead caused by excessive parallelism.
Third, fused operators and affinity optimization. For high-frequency operations such as optimizer state updates, exponential moving averages, and normalization, use fused operators to reduce memory access and launch overhead. For attention, convolution, normalization, and other modules, perform operator affinity adaptation to ensure stable performance on the target hardware. Engineering teams should establish operator regression tests to avoid numerical precision or dynamic-shape support issues after fusion.
These optimizations do not pursue extreme performance at a single point. They pursue end-to-end stability: the data pipeline should not block, compute streams should not sit idle, communication should not become a bottleneck, and operators should not frequently fall back. Only when the full pipeline runs through does model training become reproducible.
5. Arbitrary Aspect Ratios: From Preset Ratios to Non-Standard Sizes
Traditional image generation models are often trained on preset ratios such as 4:3 and 16:9. At inference time, non-standard sizes often require cropping, stretching, or outpainting, which can damage composition and text layout. The key to supporting arbitrary aspect ratios is to improve the tokenizer strategy: dynamically split image tokens according to the target aspect ratio while preserving aspect-ratio information; support continuous interpolation or dynamic grids in position encoding; and use multi-ratio buckets during training so that the model sees enough width-height combinations.
In terms of capability boundaries, it is possible to natively support arbitrary-aspect-ratio image generation from 1024×1024 to 2048×2048, such as non-standard resolutions like 1240×1600. For scenarios that require custom sizes, such as posters, social media covers, and e-commerce banners, this capability can reduce post-processing crops and improve first-pass usability. For deployment, it is recommended to define minimum and maximum side lengths, restrict extreme aspect ratios, perform local repainting for overly long text regions, and provide aspect-ratio validation and fallback strategies on the inference side.
6. Chinese Text Rendering: Glyph Priors and Layout Control
Chinese text rendering is one of the hardest challenges in image generation. Chinese has a large character set and complex stroke structures. Long text also involves font size, line spacing, alignment, line breaks, and background blending. In a hybrid architecture, the glyph encoder provides glyph priors, the diffusion decoder handles pixel-level rendering, and the autoregressive module handles text position and layout planning. Only through this coordination can issues such as missing or malformed characters be mitigated.
For evaluation, public evaluation sets can be built around complex visual text generation and long-text rendering, focusing on text accuracy, readability, layout reasonableness, and aesthetics. If an open-source model performs strongly on these evaluations, it indicates that its text-rendering pipeline has strong reference value. However, direct Chinese poster generation does not mean complete freedom from manual proofreading: scenarios with multiple fonts, artistic text, very small font sizes, and complex background overlays still require human review. Enterprises should establish a loop of generation, detection, local repainting, and human review.
7. Open Source and Deployment Recommendations
The significance of open-source release lies in reproducibility and secondary development. Code repositories, model repositories, and online demonstration entry points should provide clear license information, dependency descriptions, inference examples, and evaluation scripts. Technical reports should disclose training data scale, architecture details, optimization strategies, and known limitations. For developers, it is recommended to first validate the data pipeline and tokenizer on a single machine with small-scale data, and then gradually scale to multi-machine training. For enterprise users, it is recommended to first build a small evaluation set around high-frequency scenarios such as posters and covers, and then assess the usability of custom resolutions and Chinese text rendering.
Deployment requires attention to four types of risk. First, long text and artistic fonts still require manual proofreading. Second, extreme aspect ratios and very large sizes may lead to quality degradation. Third, full-process training is costly, so compute budgeting and checkpoint resume must be planned. Fourth, open-source licenses, data compliance, and content safety review cannot be ignored. Only when the engineering pipeline, evaluation loop, and compliance mechanisms are all in place can an image generation model trained end to end on domestic compute platforms truly enter production environments.
