An Open-Source Image Generation Model Trained End to End on Domestic AI Chips

    This article examines an open-source image generation model trained through the full pipeline—from data preprocessing and pretraining to supervised fine-tuning and reinforcement learning—on domestic AI accelerator hardware. It explores the model’s hybrid autoregressive-and-diffusion architecture, Chinese text rendering capabilities, arbitrary-aspect-ratio generation, and engineering optimizations for efficient and stable training. The discussion also outlines practical considerations for developers evaluating the model for poster, cover, infographic, and other layout-intensive applications.

    1. Why Full-Pipeline Training Is More Difficult Than Inference

    Image generation models typically require a long training pipeline that includes data cleaning and preprocessing, text-image pairing, pretraining, supervised fine-tuning, preference optimization or reinforcement learning, evaluation, and model export.

    Migrating this entire pipeline to domestic AI accelerators involves far more than verifying whether individual operators can run. The main challenges include end-to-end throughput, communication and computation overlap, memory management, training stability, and the depth of integration between the framework and operator libraries. Inference can often achieve reasonable performance through operator replacement or graph optimization. Training, however, involves backpropagation, optimizer states, mixed precision, data pipelines, and distributed communication. A bottleneck in any one of these areas can significantly reduce overall throughput.

    For this reason, completing the full training process on domestic chips is important because it validates an engineering system spanning data, models, frameworks, and operators. It is not simply proof that a particular model can run on a specific type of hardware.

    For developers, this type of implementation offers reusable engineering ideas, including how to organize data pipelines, schedule multi-stage execution, hide communication overhead through multi-stream parallelism, select hardware-friendly fused operators, and maintain training stability during supervised fine-tuning and reinforcement learning.

    2. A Hybrid Architecture: Autoregressive Understanding and Diffusion-Based Rendering

    The model uses a hybrid architecture consisting of an autoregressive backbone and a diffusion decoder.

    The autoregressive component handles semantic understanding, text-condition parsing, and global composition. It is better suited to processing long text prompts, object relationships, and layout intent. The diffusion decoder is responsible for pixel-level detail generation, converting semantic representations into high-quality images. The two components are connected through conditional injection and feature alignment, creating a division of labor in which the model first understands the instruction and then renders the image.

    The key advantage of this design is the clear separation of responsibilities. The autoregressive module does not directly optimize for pixel-level fidelity. Instead, it produces stable semantic and layout representations. The diffusion decoder does not need to reinterpret the entire prompt; it focuses on detailed rendering under semantic constraints.

    This division is particularly useful for Chinese posters, where the layout may contain multiple visual elements and complex text hierarchies. A diffusion-only approach can become unstable when handling long text and local glyph structures. With autoregressive semantic modeling, the system can first determine the relationships between the title, subtitle, body text, and visual subject before passing the representation to the decoder for refinement.

    The model also introduces a glyph encoding module to strengthen character-structure information. Text generation is not only about whether an output resembles a character. It also involves stroke structure, character spacing, line spacing, and layout consistency. Glyph encoding injects prior knowledge of character shapes into the generation process, helping reduce incorrect characters, missing characters, and merged strokes.

    From an engineering perspective, this requires high-quality training data containing text images and accurate text annotations. Evaluation should cover short titles, long paragraphs, mixed Chinese and English text, and text rendered over complex backgrounds.

    3. Chinese Text Rendering: From Generation to Readability and Layout Quality

    Chinese text rendering is more difficult than English text rendering. Chinese characters have complex glyph structures and dense strokes, while visually similar characters may differ only in subtle details. Poster generation also requires consistent font styles, hierarchy, and overall layout.

    Improving direct Chinese text generation usually requires work at three levels.

    3.1 Data

    Training data should cover different fonts, font sizes, colors, backgrounds, perspectives, and layout densities. Text annotations must also be accurate. For long-form text, paragraph-level samples are necessary; otherwise, the model may learn to generate only short headlines.

    3.2 Conditioning

    In addition to text embeddings, the model can use glyph encoding, positional constraints, and layout-region prompts to specify both what should be written and where it should appear.

    For poster generation, title areas, body-text areas, and decorative elements can be modeled separately. This helps reduce interference between text and background textures.

    3.3 Training and Evaluation

    Pretraining establishes general image-text alignment. Supervised fine-tuning strengthens text-rendering capabilities, while post-training focuses on incorrect characters, missing characters, and disorganized layouts through preference optimization.

    Evaluation should not focus only on overall visual appeal. It should also measure text readability, character accuracy, long-text consistency, and layout quality. Stable performance on public evaluations involving complex visual text generation and long-text rendering is an important prerequisite for using the model in poster production workflows.

    4. Arbitrary Aspect Ratios: Why Non-Standard Resolutions Matter

    Traditional image generation models are often trained and executed at fixed aspect ratios, such as square, widescreen, or portrait formats. In real applications, posters, e-commerce banners, and social media covers frequently use non-standard dimensions. A composition may be close to portrait orientation without matching a predefined ratio.

    If a model supports only preset aspect ratios, developers may need to crop, stitch, or extend the output. These post-processing steps can distort layout elements and remove parts of text regions.

    The model improves its tokenizer strategy to support native generation at arbitrary aspect ratios within a range from 1024×1024 to 2048×2048, including non-standard resolutions such as 1240×1600.

    The engineering benefit is reduced reliance on post-processing. The model can organize the composition directly according to the target canvas. This generally requires addressing issues such as positional-encoding extrapolation, resolution bucketing, the resolution distribution in training data, and memory consumption.

    Developers should note that support for non-standard resolutions does not mean unlimited extrapolation to arbitrary image sizes. Extremely unusual aspect ratios outside the training distribution may still cause quality degradation. Higher-resolution generation also increases memory usage and latency, so batch processing, tiled or chunked decoding, and quantization strategies may be needed for optimization.

    5. Engineering Optimizations for Full-Pipeline Training on Domestic Chips

    The central challenge of full-pipeline training is connecting data preprocessing, pretraining, supervised fine-tuning, and reinforcement learning into a stable workflow. Common optimizations include the following.

    5.1 Multi-Stage Scheduling in Dynamic Graphs

    Multi-stage pipelines can organize data loading, forward computation, backpropagation, and parameter updates to reduce scheduling and dispatch bottlenecks. This helps keep the accelerator hardware busy.

    For developers, the main considerations are the number of pipeline stages and the design of buffer queues. Poor configuration can allow data preprocessing to become the bottleneck.

    5.2 Multi-Stream Parallelism and Communication-Compute Overlap

    In distributed training, communication overhead can reduce the amount of effective compute. Multi-stream execution allows communication and computation to run on different streams and overlap with each other, improving overall throughput.

    In practice, developers must carefully manage stream synchronization, memory reuse, and the risk of deadlocks.

    5.3 Hardware-Friendly Fused Operators

    Common operations such as normalization, optimizer updates, and element-wise computation can be fused to reduce kernel launches and memory-access overhead.

    Numerical stability must remain the primary constraint. Each optimization should be validated through small-scale comparison experiments to ensure that it does not negatively affect convergence.

    5.4 End-to-End Monitoring and Fault Tolerance

    Long-running training requires monitoring of loss, gradient norms, throughput, memory usage, and communication metrics. Checkpoints should support fast recovery, while data pipelines should provide retry and deduplication mechanisms.

    During supervised fine-tuning and reinforcement learning, teams should also monitor abnormal reward behavior, output degradation, and training crashes.

    6. Open-Source Value and Deployment Recommendations

    The value of an open-source model is not limited to its weight files. Reproducible training methods and reference approaches for inference and deployment are equally important.

    When evaluating the model, developers should consider the following questions:

    • Is the target application primarily focused on Chinese text and layout generation?
    • Is native generation at non-standard resolutions required?
    • Can the available local hardware support high-resolution inference?
    • Is additional fine-tuning needed to match a brand’s visual guidelines?

    For deployment, it is advisable to first validate semantic correctness and layout quality at a lower resolution, then gradually increase the resolution. Text regions should also receive additional post-processing validation. Evaluation datasets should include metrics for character error rate, readability, and layout consistency.

    Overall, this type of model is significant because it validates domestic-chip full-pipeline training, hybrid image generation, Chinese text rendering, and non-standard-resolution support within a single engineering framework.

    For the industry, it provides a reference path from general-purpose image generation toward controllable layout generation. For developers, understanding the division of responsibilities between the model components and the associated engineering optimizations is more valuable in the long term than focusing only on the appearance of individual generated images.