End-to-End Training of Multimodal Image Generation on China-Developed AI Chips
Text-to-image models are now judged by more than visual appeal. This article explains how an open-source image generation effort trained entirely on China-developed AI chips and software covers data processing, training, evaluation, and inference, and how to assess instruction following, Chinese text rendering, stability, and production use.
Text-to-image models have improved sharply in recent years. The industry focus has moved beyond whether a model can produce an attractive image. Teams now ask whether it can follow complex instructions, render text accurately, handle Chinese-language scenarios, and be trained and deployed reliably on different computing stacks.
An open-source image generation model advanced by teams in China has drawn attention recently. Its significance is not simply that another text-to-image tool exists. The training pipeline covers data processing, model training, evaluation and optimization, and inference adaptation, and the effort attempts to run that full path on domestically developed Chinese AI chips and the related software stack. For organizations that want to reduce dependence on a single hardware and software supply chain and build their own compute base, the practice is a useful reference.
1. Model role: from a text-to-image tool to multimodal generation
Functionally, this class of model is used for text-to-image generation. A user provides a natural-language prompt, and the model produces an image from the subject, action, setting, composition, style, and lighting described in that prompt.
Typical uses include:
- Generating images from Chinese or English text
- Creating styles such as photorealistic photos, concept design, children’s illustration, and poster sketches
- Producing science diagrams, flowcharts, and teaching illustrations
- Drafting product visuals, spatial design concepts, and advertising ideas
- Combining the generator with other language models in a workflow of text understanding, image generation, and revision
Compared with earlier models, newer systems are judged on a wider set of criteria. Sharpness is not enough. The model also has to capture relationships in the prompt. “A white puppy standing next to a red bicycle” and “a red puppy standing next to a white bicycle” involve multiple entities and attribute bindings. The model must separate color, subject, and spatial relation rather than simply stitching visual elements together.
The value of the model can be read on three levels:
- Visual quality: clarity, color, detail, and overall aesthetics
- Semantic consistency: whether the result matches the written description
- Engineering usability: runtime stability, inference efficiency, deployment cost, and hardware and software fit
2. Hands-on results: simple, well-structured prompts are usually more stable
In practical trials, short prompts with a clear structure tend to produce more stable results. For example:
Create a children’s science illustration of the solar system that includes the Sun, the eight planets, orbits, and Chinese labels, with bright colors and a clear layout.
This prompt states the topic, audience, composition, and style, so the model can more easily build a coherent scene. For people, landscapes, indoor spaces, and common objects, current models can often produce images that are pleasant to look at.
To raise the success rate, split the prompt into parts:
- Subject: what object should appear
- Scene: where that object is
- Action or relation: how objects interact
- Visual style: photorealistic, illustration, sketch, low-poly, or poster
- Composition: close-up, wide shot, overhead view, symmetry, or negative space
- Quality: rich detail, soft light, consistent color, and similar constraints
Instead of writing only “draw a city at night,” use something more specific:
Create a modern city-night illustration. The foreground is a rail transit platform, the middle ground shows pedestrians and street shops, and the background shows high-rise buildings and lights. Use a blue-purple palette and a horizontal layout suitable as an illustration in a science book.
More concrete wording leaves less room for free invention and makes later debugging easier.
3. How the model behaves in harder scenes
3.1 Multi-subject relations remain difficult
When a scene contains several characters, several objects, and explicit spatial relations, the model may get the count wrong, mix up positions, or bind attributes to the wrong object.
A request such as “on the left, a student wearing glasses; on the right, a teacher holding a book; both standing in front of a classroom blackboard” may swap the people, drop the book, or distort the glasses. That is not only an image-quality issue. Parsing of entity relations in longer text is still unstable.
Useful mitigations include fewer parallel demands in one sentence, layered descriptions, and placing the most important subject near the start of the prompt. For a complex poster, generate the background and main subjects first, then adjust text and local elements in a later edit.
3.2 Chinese text rendering needs a separate check
Text remains one of the hardest tasks for image models. Even when the subject, color, and layout look right, titles, labels, or captions on a poster may contain wrong characters, deformed strokes, stuck glyphs, or incomplete wording.
For Chinese posters, course covers, product promotions, or science graphics, do not treat the model output as the final asset. A safer flow is:
- Generate a background with little or no text
- Add titles in an image editor or layout tool
- Manually proofread labels on diagrams
- Check resolution, margins, and readability on the final image
If the task is to generate a chart, verify the layout first and the text second. For education, healthcare, finance, public services, and other high-accuracy uses, all text must be reviewed by a person.
3.3 Named characters and highly specific styles are uncertain
When a prompt names a well-known film or TV character, a particular artist’s style, or a very specific visual symbol, training data, compliance policy, and semantic generalization can all affect the result. The output may not match the request.
In practice, generic style language is more reliable: “thick-paint illustration,” “vintage print,” “soft watercolor texture,” or “high-contrast cinematic lighting.” That reduces copyright and compliance risk and tends to produce more stable visuals than leaning on one named subject.
4. Why full-stack training on China-developed chips matters
Running a complete training job on domestically developed Chinese AI chips is not a simple hardware swap. It also depends on compilers, operator libraries, distributed communication, memory management, data loading, and monitoring.
A typical training flow can be summarized as:
Raw data collection↓Cleaning, deduplication, and quality filtering↓Image-text pairing and annotation↓Distributed data loading↓Pretraining and staged fine-tuning↓Automatic evaluation and human review↓Inference-graph optimization and deployment adaptation
A compatibility gap in any stage can lower training efficiency. If an operator lacks an efficient implementation on the target chip, or communication between devices is weak, utilization drops. Training also has to handle mixed precision, gradient synchronization, checkpointing, and failure recovery.
“Full-stack domestic” training therefore matters in three ways:
- It shows that a China-developed software and hardware stack can carry large-model training
- It provides a base for later scale-out, industry deployment, and supply-chain resilience
- It pushes algorithms and infrastructure to be optimized together, rather than only porting an existing recipe onto another chip
Completing a training run does not mean every scenario will reach the same performance. Real efficiency still depends on chip model, cluster size, parallelism strategy, operator optimization, model size, and input resolution. When selecting a stack, look for reproducible benchmarks and a complete deployment path, not only the label “trained on domestic chips.”
5. Building a more objective evaluation
A few strong sample images do not show what a model can actually do. A better approach is a test set that covers several dimensions.
5.1 Basic generation
Use common prompts for people, animals, buildings, landscapes, and indoor scenes. Check completeness, clarity, and style consistency.
5.2 Instruction following
Test color, count, position, action, and subject relations. For example, ask for three objects and specify the size, color, and relative position of each, then see whether the model follows those constraints.
5.3 Chinese understanding
Use Chinese prompts that include idioms, long sentences, domain terms, and multiple constraints. Judge how well the model handles complex meaning.
5.4 Text generation
Test titles, short labels, numbers, and simple tables. Record wrong-character rate, missing-character rate, and readability. Score text separately so strong overall visuals do not hide weak lettering.
5.5 Stability and repeatability
Generate the same prompt several times and see whether subject, composition, and style swing widely. In production, consistency usually matters more than an occasional standout image.
5.6 Engineering metrics
Also track memory or equivalent memory use, time per image, concurrency, retry-on-failure behavior, log completeness, and deployment difficulty. For an internal service, add access control, data isolation, and content-safety review.
6. Practical ways to put it into use
This kind of model fits creative drafts and content assistance better than a full replacement for professional design. A workable pattern is: model draft, human selection, local revision, unified layout, then quality review.
Inside an organization, the model can be wrapped as a shared service. Object storage can keep outputs, a message queue can handle asynchronous jobs, a container platform can scale capacity, and logging plus monitoring can show runtime health. These are ordinary cloud building blocks. The point is a production flow that can be traced, rolled back, and audited.
Research and engineering teams should also keep prompt templates, a library of failure cases, and a written human-evaluation standard. After each model upgrade, rerun the same test set so changes are not judged only by impression.
7. Conclusion
An image generation model trained end to end on China-developed AI chips answers two questions at once. Can the model deliver visual quality close to mainstream systems? And can a domestically developed Chinese AI software and hardware stack support the full path from data through training to deployment?
In practice, simple scenes, familiar styles, and clear prompts often work well. Complex subject relations, precise Chinese text, and demanding commercial design still need human revision. For developers, the useful question is not whether one image looks striking. It is whether stability, controllability, deployment efficiency, and ongoing iteration meet the job.
Treat the model as a base capability for creative generation, design assistance, and multimodal applications—not as a tool that turns one sentence into a finished deliverable. That framing makes its technical value easier to judge and leads to a more reliable workflow in real projects.
