Training Image Generation Models on Domestic Chips: Text Rendering and Arbitrary-Resolution Practices

    Image generation models still struggle with accurate text rendering and custom output sizes. This article describes a hybrid autoregressive-plus-diffusion architecture that addresses Chinese text rendering and arbitrary aspect ratios, along with engineering practices for training and evaluation on domestic accelerators.

    Image generation models have improved markedly in visual quality, yet two classes of failures remain common.

    The first is text rendering. Titles, slogans, prices and menus must appear with complete strokes, correct glyphs and sensible layout. Diffusion models generate images by iterative denoising and do not explicitly constrain discrete character structure, so misspellings, omissions and stroke merging are frequent. The problem is more severe for Chinese because of the large character set and complex glyphs.

    The second is output size. Many training sets and inference configurations cover only a handful of preset ratios such as 1:1, 4:3 and 16:9. Posters, social-media covers and app splash screens typically need custom dimensions. Naïve scaling or cropping destroys composition and can truncate text.

    This article uses a hybrid architecture aimed at Chinese text rendering and arbitrary-ratio generation as a running example, discusses its design, and outlines engineering points for training and deployment.

    Hybrid Architecture: Understand First, Then Draw

    Splitting generation into two stages has become a common recent pattern.

    1. Semantic understanding and global planning. An autoregressive model reads the prompt and emits a structured plan of subjects, layout, color palette and the text that must be rendered. Sequential generation lets the model exploit language-model reasoning for complex instructions such as “place the title in the upper-left, a QR code in the lower-right, overall cool tones.”
    2. Detail rendering. A diffusion decoder denoises a high-resolution image conditioned on the plan.

    The division of labor is therefore: the first stage interprets the request; the second stage paints the picture accurately.

    Compared with a pure diffusion pipeline the hybrid design differs as follows:

    Dimension Pure diffusion Autoregressive semantics + diffusion decoder
    Semantic understanding One-shot mapping by a text encoder Step-by-step reasoning that can emit an intermediate plan
    Text accuracy Relies only on text embeddings; glyphs easily fail Can inject an explicit glyph encoder
    Composition control Weak, largely stochastic More stable global plan
    Inference latency Dominated by denoising steps Extra cost of autoregressive generation

    The hybrid pipeline is longer; quality versus latency must be traded off at selection time.

    Why a Separate Glyph Encoder Is Needed

    The diffusion decoder is normally conditioned on semantic vectors. Those vectors indicate that “some text belongs here” but do not precisely encode each character’s structure. Asking the decoder to recover glyphs from semantics alone forces it to learn both language meaning and stroke details simultaneously, which is unstable.

    A more reliable approach extracts glyph- or character-level features for the text that must be rendered and injects them as extra conditions. The semantic module then decides “what to write and where,” while the glyph module decides “what each character looks like.”

    The following sketch shows the data flow among the three modules (illustrative only):

    1. def generate_image(prompt: str, size: tuple[int, int]):
    2. # 1. Semantic module: parse prompt, emit layout and text list
    3. plan = semantic_planner(prompt)
    4. # 2. Glyph module: produce glyph conditions for each text item
    5. glyph_cond = [
    6. glyph_encoder(item.text, item.font_style)
    7. for item in plan.texts
    8. ]
    9. # 3. Diffusion decoder: fuse semantics, glyphs and target size
    10. latent = diffusion_decoder(plan.embedding, glyph_cond, size)
    11. # 4. Decode to pixels
    12. return vae_decode(latent)

    In production the text list should be passed separately from the scene description (e.g., a dedicated field “title: Summer New Product Launch”) so that glyph matching and later verification are easier.

    Supporting Arbitrary Resolutions

    Why Preset Ratios Appear

    Resolution limits usually originate in training. If every sample is cropped to a fixed size, both positional information and token-grid shape become bound to that size. At inference a different ratio pushes positional signals out of the training distribution, producing repeated structures, edge artifacts or unbalanced composition.

    Three Changes Required for Native Arbitrary Ratios

    1. Tokenizer: when the image is split into patches or latent tokens, width and height are divided independently by the stride, allowing non-square grids.
    2. Positional encoding: use 2-D encodings or interpolable encodings so that every grid shape receives a valid positional signal.
    3. Training data: sample many aspect ratios and randomize crop shapes so the model does not memorize a few fixed grids.

    Size Alignment

    Regardless of architecture, the output size must be an integer multiple of the tokenizer stride. The following helper snaps an arbitrary request into the allowed range:

    1. def align_resolution(
    2. width: int,
    3. height: int,
    4. stride: int = 16,
    5. lo: int = 1024,
    6. hi: int = 2048,
    7. ) -> tuple[int, int]:
    8. """Snap target size to a multiple of the tokenizer stride and clamp to range."""
    9. def snap(v: int) -> int:
    10. v = min(max(v, lo), hi)
    11. return max(lo, (v // stride) * stride)
    12. return snap(width), snap(height)
    13. print(align_resolution(1240, 1600)) # (1232, 1600)

    Here 1240 is snapped to 1232. Callers should log the actual output size so users are not surprised by a mismatch. For poster layouts, text coordinates computed in pixels must be recomputed after alignment, otherwise text may drift from the intended region.

    Training Pipeline on Domestic Accelerators

    A full training run typically comprises data preprocessing, large-scale pre-training, supervised fine-tuning (SFT) and reinforcement learning (RL). When all four stages run on the same domestic accelerator cluster three bottlenecks dominate.

    Dispatch Overhead

    When a model contains many small operators, host-side scheduling can exceed device compute time and the device idles. Typical remedies are graph compilation, operator fusion and multi-level pipelined dispatch that overlap host scheduling with device execution. Gains depend on architecture, batch size and cluster scale; profiling is required to locate the hot spots.

    Overlapping Communication and Compute

    Gradient synchronization and parameter-shard communication consume noticeable time. Placing communication on a separate stream that runs concurrently with forward and backward passes hides part of the latency. Dependencies must be respected so that gradients are fully synchronized before they are used.

    Operator Fusion and Numerical Consistency

    Fusing consecutive operations (normalization, activation, optimizer update) into a single kernel reduces memory traffic. Common fusion candidates include AdamW updates, EMA parameter updates and RMSNorm.

    EMA is:

    θ_ema = β · θ_ema + (1 − β) · θ

    where θ is the current parameter and β the decay. After fusion the implementation must be compared layer-by-layer against the original to confirm that errors stay within acceptable numeric tolerance.

    When porting to new hardware, operator numerics may differ slightly. A fixed-seed alignment test that records max absolute and relative error per layer, plus early monitoring of the loss curve, is recommended.

    Evaluating Text-Rendering Ability

    Text rendering can be scored on four axes.

    1. Character accuracy: OCR the generated image and compare token-by-token with the target string.
    2. Layout integrity: check that text is not cropped, occluded or overflowing its region.
    3. Long-text consistency: for paragraphs, inspect line spacing, wrapping and punctuation.
    4. Mixed-script capability: when Chinese, English, digits and symbols co-occur, check font and tracking harmony.

    Character-level comparison can use edit distance or sequence similarity. A standard-library sketch:

    1. import difflib
    2. def text_similarity(pred: str, target: str) -> float:
    3. return difflib.SequenceMatcher(None, pred, target).ratio()