End-to-End Text-to-Image Training on a Domestic Chinese AI Stack

    Text-to-image systems are often trained on general-purpose GPUs and only later moved to other accelerators. This article explains why that split raises migration cost, and how data preparation, training, and inference can instead run on one domestic Chinese AI software and hardware stack while remaining trainable, reproducible, and evaluable.

    End-to-End Text-to-Image Training on a Domestic Chinese AI Stack

    Multimodal generation, especially text-to-image, has long followed a quiet path dependency: data processing, pretraining, and fine-tuning are usually done in a general-purpose GPU ecosystem, and inference is later “ported” to domestically developed Chinese accelerators. That path can produce results quickly, but it ties operators, memory access, collective communication, and precision strategy to the habits of one hardware platform. Later migration is costly and less stable. A more useful engineering result is a closed loop—from data preparation and model training through inference deployment—on the same domestic Chinese AI software and hardware stack, with generation quality at a level that is usable by mainstream practice.

    The sections below cover positioning, architecture, training strategy, hardware adaptation, and practical evaluation. The goal is to treat text-to-image as a trainable, reproducible, and measurable system capability, not a one-off gallery of sample images.

    1. Positioning: More Than Another Text-to-Image Endpoint

    Functionally, a text-to-image model usually accepts Chinese and English prompts, produces realistic images, illustrations, and design sketches, and works with a language-understanding module: interpret the instruction first, then generate under that condition. Focusing only on attractive samples misses the system-level value:

    • Consistency across the pipeline. Cleaning, tokenization, visual encoding, training parallelism, checkpoints, and the inference service share the same operators and precision conventions. That reduces the failure mode in which training runs but inference does not match.
    • Conditional generation aligned with language modeling. Treating image generation as conditional generation, and sharing modeling ideas with autoregressive or diffusion-style language generation, makes unified multimodal scheduling and alignment easier.
    • Hardware-aware design from the start. Operators are not rewritten after training. Layer splits, attention implementations, and communication topology are chosen in advance for the compute and bandwidth profile of the domestic NPU.

    For developers and enterprise users, that means a more controllable supply chain, more predictable cluster utilization, and later iterations that do not require another port followed by months of retuning.

    2. Model Structure: Unified Generative Modeling and Multimodal Alignment

    Text-to-image does not have to copy one public recipe. A more stable approach is to extend the generative framework of large language models toward vision.

    1. A unified view of conditional generation
    Treat the image as a high-dimensional sample conditioned on text and optional control signals. The text side supplies semantic conditions; the vision side generates step by step in latent space or pixel space. Keeping the modeling paradigm consistent with the language side makes it easier to share optimizer settings, learning-rate schedules, and stability techniques such as gradient clipping, loss scaling, and mixed precision.

    2. Efficient multimodal alignment
    Training instability often comes from a mismatch between text semantics and visual representations. In practice this requires:

    • A stable text encoder and a stable mapping into visual latents, so the conditioning signal is neither too weak nor too strong.
    • A schedule for the weights of alignment loss and generation loss: secure semantics first, then refine texture.
    • Tokenization and semantic coverage for Chinese prompts that are on par with English, so Chinese instructions do not drift into an English visual style.

    3. Operators and parallelism suited to domestic accelerators
    NPUs differ from general-purpose GPUs in matrix-multiply shapes, vector width, HBM bandwidth, and collective communication. Those differences should shape the architecture:

    • Fuse attention with convolution or linear layers to reduce intermediate activations written out to memory.
    • Split pipeline and tensor parallelism according to the chip topology, so cross-node traffic stays lower.
    • Use memory-friendly layouts that are contiguous and aligned, and avoid frequent transposes.

    These choices decide whether a large cluster can train stably for a long time, rather than merely producing images at small scale.

    3. Training Strategy: Diffusion Ideas and Implementation-Level Optimization

    Image generation still widely borrows from diffusion, or from flow matching: denoise step by step from noise, with text as the condition. The slogan matters less than the implementation:

    • Operator composition. Fuse timestep embeddings, conditional cross-attention, normalization, and activations where possible, to cut kernel-launch overhead.
    • Memory access. Balance activation recomputation and selective checkpointing against compute. Choose latent resolution and batch size from peak bandwidth, not by copying GPU rules of thumb.
    • Parallelism. Data parallelism provides throughput; model parallelism keeps the model on a device. Hide pipeline bubbles with micro-batches and scheduling.
    • Numerical stability. Under mixed precision, clip and reparameterize the diffusion noise schedule and variance terms so long runs do not collapse late in training.

    The data side must be controllable end to end as well: cleaning image-text pairs, detecting watermarks and duplicates, bucketing by resolution, and balancing Chinese and English. If the data pipeline runs on a different hardware stack and is only later imported into the training cluster, encodings often diverge and random seeds become hard to reproduce. One point of closing the loop on a domestic stack is that data augmentation and training operators share the same runtime.

    4. Hardware and Systems: A Cluster Built for Long Training Jobs

    Domestic servers aimed at large-model training typically emphasize large-scale accelerator interconnects, compute units suited to Transformers and multimodal models, and power, cooling, and fault isolation that can sustain long jobs. Engineering attention should go to:

    • Interconnect and topology. Whether collective communication matches the model split, and whether bandwidth becomes the bottleneck in attention layers.
    • Stability. Device loss, link retraining, checkpoint frequency, and asynchronous saves, so a local fault does not destroy a multi-week job.
    • Inference alignment. Keep training operators and the inference service as close to the same implementation as possible, to limit semantic drift between training in FP16 and inference in INT8.

    At the system level, object storage can hold datasets and checkpoints, a container platform can run elastic jobs, and logs and monitoring can track loss curves and communication latency. These are general capabilities and are not tied to any one vendor’s proprietary interface names.

    5. Practical Evaluation: How to Judge Image Quality More Carefully

    Sample images can show an upper bound. They cannot replace evaluation. It helps to split the work by task type.

    Diagrams and educational picture books
    A prompt such as “a diagram of the solar system, in the style of a children’s science picture book” tests whether relative positions of bodies are plausible, labels are readable, and the style is consistent. Geometry and text rendering are more exposed here than in a pure landscape.

    Film stills and realistic portraits
    A prompt such as “a still in the style of a science-fiction film” tests lighting, materials, composition, and character consistency. Hands, perspective, and text artifacts matter more than the first visual impact.

    Coverage across styles
    Build small sample sets separately for realism, illustration, and design sketches. Compare them with the same seeds and resolutions, instead of publishing only curated highlights.

    Useful evaluation dimensions include text adherence, structural plausibility, style consistency, readability of Chinese text, and stability across repeated generations. Enterprise use should also cover latency, concurrency, compliance filtering, and watermark policy. The entry point may be an open inference platform or a private deployment. What matters is a clear prompt convention—subject, style, camera, and constraints—not dependence on one branded page.

    6. Practical Advice and Common Pitfalls

    • Align before polishing. If the condition is too weak, the image can look good while missing the prompt. If it is too strong, outputs become rigid and less diverse.
    • Resolution bucketing. When several side lengths are mixed in one run, a poor batching or padding policy wastes compute or introduces artifacts.
    • Unequal Chinese and English prompts. A large gap for the same meaning is often a data-mix or tokenization problem, not proof that “the model cannot do Chinese.”
    • Migration illusions. A learning rate and batch size tuned on hardware A, copied unchanged onto hardware B, often blow up the loss before the model is at fault. Recalibrate against peak bandwidth and on-chip cache.
    • Do not invent metrics. Publicly state only reproducible settings and qualitative observations. Without measurements, do not quote peaks or speedups; a cautious phrase such as “usable at a mainstream level” is enough.

    7. Summary

    The value of training a text-to-image model end to end on domestic Chinese chips is not another generation endpoint. It is placing data, training, and inference on the same assumptions about operators and parallelism, so the system can scale, be operated, and be iterated. Architecturally, keep conditional generation and multimodal alignment. In implementation, fuse operators and optimize memory access and communication for the NPU. In evaluation, separate diagrams, realism, and illustration, and judge adherence and stability. For enterprise users, this approach can reduce supply-chain and migration risk. For developers, it is a repeatable path of hardware-aware modeling. Later work can add finer control signals such as layout, depth, and edges, more stable text rendering, and instruction following unified with the language side—rather than stopping at how a single sample looks.