Architecture and Practice of a Text-to-Image Model Trained End to End on Chinese-Made Chips

    This article explains the architecture and engineering behind a multimodal text-to-image model whose full training pipeline runs on Chinese-made chips. It covers the hybrid autoregressive and diffusion design, text rendering, flexible image dimensions, and training optimizations such as multi-stream parallelism and fused operators. It also outlines how to evaluate such models and what to consider before deploying them.

    Text-to-image generation is moving beyond producing attractive images. Commercial posters, knowledge cards, and social media covers also require models to follow complex instructions, produce correct text, and control layout. If a model can complete the full workflow, from data processing to training, on domestic Chinese chips, the training framework, operator support, parallelism strategy, and engineering toolchain have reached a substantial level of independence.

    This article uses a multimodal image generation model trained entirely on Chinese-made chips as an example to analyze its technical approach and practical value. The model size, architecture, and performance described here are based on publicly released information. Any real deployment should be validated against the available hardware, data scale, and business requirements.

    1. Why End-to-End Training on Chinese-Made Chips Matters

    Image generation models have traditionally relied on large-scale general-purpose accelerator cards. Model teams must solve algorithmic problems and also handle hardware adaptation, operator support, communication efficiency, memory management, and training stability. Completing the full training workflow on Chinese-made chips is therefore not a simple matter of porting model code to new hardware. It is a full-stack systems engineering effort.

    A complete training workflow includes at least the following stages:

    1. Data preprocessing: cleaning images, adjusting resolution, annotating text, pairing images with captions, and formatting training samples.
    2. Pretraining: teaching the model the correspondence between images and text, along with foundational skills such as objects, scenes, composition, and visual style.
    3. Supervised fine-tuning: using high-quality instruction data to improve how the model interprets user descriptions, layout requirements, and text content.
    4. Preference or reinforcement optimization: using human preferences, quality scores, or task feedback to further improve generated results.
    5. Evaluation and deployment adaptation: comprehensively verifying text accuracy, composition quality, resolution support, and inference efficiency.

    This work tests how complete China’s domestic software and hardware ecosystem is for real large model tasks. Multimodal models typically involve dense matrix computation, attention mechanisms, normalization, communication synchronization, and data pipelines. A bottleneck in any one of these areas can reduce overall training efficiency.

    2. Core Architecture: Coordinating Autoregressive Understanding and Diffusion Generation

    The model uses a hybrid architecture that combines an autoregressive model with a diffusion decoder. The overall approach can be summarized as follows:

    • The autoregressive module interprets the request and plans the scene.
    • The diffusion module generates pixels and restores fine detail.
    • The text encoding module improves the accuracy of Chinese and other complex text.

    Rather than asking a single model to handle every task, this design divides the work according to each component’s strengths.

    2.1 Autoregressive Module: Semantic Understanding and Global Composition

    Autoregressive models are well suited to long text, contextual relationships, and multi-step reasoning. Given a complex image generation instruction, the module can first extract the following information:

    • What the main subject of the image is;
    • How the subjects relate to each other in space;
    • What the background, lighting, and style should be;
    • Where the title, body text, and tags should be placed;
    • Which elements should be emphasized and which should be de-emphasized.

    For example, a user may request a vertical science poster with a main title, a subtitle, three key points, and a footer note. The model must first understand the hierarchy among these text elements, then form the overall layout. If the model relied only on a traditional image generation process, it might produce an image that looks like a poster, but it would not reliably handle the text content or layout.

    Public information indicates that the autoregressive component has roughly 10 billion parameters and handles semantic understanding and global planning. It does not output final pixels. Instead, it outputs a high-level representation of the image’s content, structure, and semantic relationships.

    2.2 Diffusion Decoder: Visual Detail and Image Reconstruction

    The diffusion decoder is better suited to continuous visual space. It converts the high-level semantic representation into a concrete image through step-by-step denoising, and it reconstructs textures, edges, lighting, color, and local detail.

    This module uses a large-scale diffusion transformer with billions of parameters. Compared with simply scaling up a diffusion model, the key advantage of the hybrid design is that a language model first works out the requirements, and then a visual decoder renders the image.

    The flow can be represented as follows:

    1. User prompt
    2. ↓
    3. Text understanding and semantic planning
    4. ↓
    5. Layout, object relationships, text hierarchy
    6. ↓
    7. Diffusion-based visual decoding
    8. ↓
    9. High-resolution image

    This architecture is especially useful for scenes with multiple subjects, multiple text blocks, and multiple constraints, because it separates language reasoning from visual generation.

    2.3 Text Encoding Module: Improving Text Rendering

    Text rendering has long been a weak point for image generation models. Traditional models often produce text-like textures but struggle to render Chinese characters with correct structure, stroke order, and long text accurately. The underlying problem is that text in an image is both a visual texture and a language symbol with strict discrete structure.

    To address this, the model adds a dedicated text encoding mechanism that passes text content and font characteristics more fully to the visual generation module. The goal is not simply to show text-like styling in the image. The aim is to ensure:

    • Character structure is well formed;
    • Title content matches the input;
    • Multiple lines of text follow the correct order;
    • Font size, spacing, and hierarchy remain stable;
    • Long Chinese text remains readable against complex backgrounds.

    For tasks such as commercial posters, event promotion images, course covers, and knowledge cards, the text encoding module is often more important than simply increasing the number of sampling steps.

    3. Non-Standard Resolutions: Moving from Fixed Templates to Flexible Canvases

    Traditional image generation models usually support a few fixed aspect ratios, such as square, landscape, and portrait. Fixed ratios simplify training and inference. Real business canvas sizes, however, are often irregular:

    • Posters may need a very tall portrait format;
    • Social media covers may use wide formats;
    • E-commerce product images may need margins adjusted for the product information;
    • Information cards may use custom width and height;
    • Printed materials must match specific page dimensions.

    The model improves its image tokenization strategy so it can support more flexible width and height combinations. According to public descriptions, it supports image dimensions on the order of 1,000 to 2,000 pixels and allows non-standard ratios, such as vertical canvases close to 5:6.

    Non-standard resolution support means more than offering more sizes. It provides three specific benefits:

    First, it reduces post-generation cropping. Generating at a fixed ratio and then cropping can cut off text, subjects, or other key visual elements.

    Second, it lowers the cost of reformatting. Users can specify the target canvas directly, and the model can account for margins and subject placement during generation.

    Third, it fits production systems better. Companies can generate content dynamically for different channel layout requirements, instead of training or preparing a separate template for each size.

    However, supporting arbitrary ratios does not mean every size will maintain the same quality. In practice, memory usage, generation latency, text clarity, and stability at the edges of the image still need to be checked.

    4. Engineering Optimizations for Training on Chinese-Made Chips

    For large model training on Chinese-made chips, the main challenge is usually overall execution efficiency, not individual operators. The optimization directions mentioned in public materials have general reference value.

    4.1 Multi-Level Pipelining to Reduce Task Dispatch Overhead

    Large model training involves many repeated computations. If each computation stage frequently waits for scheduling, the hardware cannot stay highly utilized. Multi-level pipelining in dynamic graph mode prepares upcoming tasks in advance and reduces idle time between computation tasks.

    The basic idea is:

    1. Data preparation → Forward pass → Backward pass → Gradient communication → Parameter update
    2. ↘ Prepare the next batch in parallel ↗

    When data loading, computation, and communication overlap, overall training throughput improves.

    4.2 Multi-Stream Parallelism for Overlapping Communication and Computation

    In distributed training, computation and communication often wait on each other. For example, after some devices finish computing gradients, they must synchronize parameters. If compute units stall completely during communication, training efficiency drops noticeably.

    Multi-stream parallelism lets some computation tasks run at the same time as communication tasks. While preserving the correct dependencies, it hides as much communication time as possible. For large models, this typically requires the training framework, compiler, and hardware runtime to work together.

    4.3 Fused Operators to Reduce Intermediate Overhead

    Optimizers, normalization, and attention-related operations appear frequently during training. Fusing several consecutive operations reduces reads and writes of intermediate tensors and the number of memory accesses.

    Public information mentions optimization directions including parameter updates, exponential moving averages, normalization, and communication-related fused operators. The shared goals are to reduce memory access, lower the number of scheduling operations, and improve the effective utilization of each computation.

    Note that more fused operators do not always mean better results. Developers must also verify numerical stability, mixed-precision error, and compatibility across different batch sizes. Otherwise, training may fail to converge or produce unstable results.

    5. How to Evaluate the Real Capabilities of Such Models

    Image generation models should not be judged only on a few showcase images. A more reliable evaluation framework should cover the following dimensions.

    5.1 Semantic Compliance

    Check whether the model fully follows the subject, action, scene, style, and quantity requirements in the prompt. For complex descriptions, verify that multiple conditions are satisfied at the same time.

    5.2 Text Accuracy

    Focus on the correctness of Chinese characters, the completeness of long text, punctuation, line break logic, and font readability. For poster tasks, text accuracy is often more critical than visual style.

    5.3 Composition and Layout

    Check whether the main subject is obscured, whether the title is in a reasonable position, whether the visual hierarchy is clear, whether margins are balanced, and whether the composition remains stable at different sizes.

    5.4 Resolution Generalization

    Do not test only standard squares. Cover landscape, portrait, and non-standard ratios, and compare detail, text, and subject consistency across different canvas sizes.

    5.5 Engineering Efficiency

    Organizations should also monitor training throughput, memory usage, inference latency, failure rate, batch generation stability, and reliability over long runs. A model with strong output quality still has limited production value if it cannot run reliably in the target environment.

    6. Implications for Enterprise Applications

    For content production teams, the most direct value of this type of model is reducing the work involved in regenerating and reformatting content. Designers often generate a background first, then use image editing tools to add text and elements. If a model can directly produce a reasonably accurate Chinese poster, the time from concept to first draft shortens.

    For development teams, integration should focus on the following capabilities:

    • Split user requests into four types of parameters: content, layout, style, and dimensions;
    • Provide structured input for titles, body text, and tags;
    • Add text recognition and quality checks to generated results;
    • Provide automatic retry or local repair for images that fail checks;
    • Log prompts, resolutions, random seeds, and review results to support issue tracking;
    • Establish a review process for materials involving copyright, privacy, and sensitive content.

    A reasonable production workflow could look like this:

    1. User request
    2. ↓
    3. Structured parsing and safety check
    4. ↓
    5. Image plan and text layout generation
    6. ↓
    7. Image generation
    8. ↓
    9. Text recognition, clarity, and layout validation
    10. ↓
    11. Human review or automatic retry
    12. ↓
    13. Export for different channel sizes

    7. Conclusion

    The value of a text-to-image model trained end to end on Chinese-made chips lies not only in generating sample images. It also reflects the maturing coordination among algorithm architecture, training systems, and hardware ecosystem. The autoregressive module handles semantic understanding, the diffusion decoder handles visual reconstruction, and the dedicated text encoding mechanism improves the handling of Chinese poster, complex cover, and multi-element layout tasks.

    Support for non-standard resolutions brings the model closer to real production scenarios. Optimizations such as multi-level pipelining, multi-stream parallelism, and fused operators provide engineering examples for running large-scale multimodal training on domestic compute.

    For developers, the most useful takeaway is not the marketing label of any single model, but the general method behind it: connecting language understanding, visual generation, text rendering, and underlying compute into one closed loop. As training frameworks, chip compilers, and evaluation systems continue to mature, image generation models may evolve from content creation tools into intelligent design infrastructure that can be embedded in enterprise production workflows.