Reference-Based Character Image Generation: A Practical Prompt Guide

1. From “finding a prompt” to “building a character generation workflow”

In many character-focused image generation scenarios, beginners often ask the same thing: can you just give me one prompt that makes the model generate a specific anime character? For example, they want a character with fixed hair color, fixed outfit, fixed signature item, and a consistent art style, while keeping the character’s identity stable.

This request may look simple, but it involves three separate problems:

1. Character identity recognition: The model must understand the character’s visual appearance, clothing structure, color relationships, signature items, and overall temperament.
2. Style consistency: The model should keep the original visual style—such as anime, chibi, illustration, or meme style—instead of suddenly switching to realism, 3D, or another style.
3. Action and scene controllability: Once the character’s identity is fixed, the model still needs to generate new actions, expressions, compositions, camera angles, and scenes.

A better approach is not to search for one “universal prompt,” but to build a character image generation workflow where the reference image is the core and the prompt is a supplement. The reference image tells the model “who this character is,” while the prompt tells the model “what the character should do this time.”

This approach reduces the uncertainty caused by pure text descriptions and improves controllability. It is especially useful for fan works, meme images, character stickers, promotional materials, avatar design, and similar scenarios.

2. Why pure text prompts struggle to generate a fixed character reliably

1. A character may have too many stable identifying features for text to cover fully

A character may have multiple recognizable features, such as:

• Hair color, hairstyle, and hair accessories;
• Eye color, face shape, and habitual expressions;
• Clothing type, collar, cuffs, skirt, cape, and other outfit details;
• Color proportions, line style, and decorative elements;
• Signature props, tail, earrings, weapons, or accessories;
• Character temperament, such as cute, lazy, tsundere, serious, or gentle.

If the model depends entirely on text, it must process many abstract terms at once. However, different models may interpret similar descriptions differently. For example, descriptions such as “long blue hair,” “maid-style outfit,” “whale tail,” or “cute art style” may produce different proportions, line styles, and clothing structures.

2. Text prompts often lose some constraints when many details are included

When a prompt contains too much information, the model may respond more strongly to parts that have higher semantic clarity or apparent priority. For example, if a user asks for a character to “sit on the ground,” “hold a bowl with both hands,” “show an innocent expression,” “keep the original outfit,” “use chibi style,” and “keep the image simple,” the model may weaken some conditions if the description is too dense. This can lead to problems such as:

• The pose changes, but the character’s outfit is modified;
• The expression matches, but the art style drifts;
• The composition is simpler, but the character’s signature item is lost;
• The action is correct, but the hands, props, or food structure break.

3. Character identity and image content need layered control

Character identity belongs to “subject stability,” while actions and scenes belong to “image variation.” If both are mixed into the same prompt, the model may struggle to decide what must be preserved and what is allowed to change.

It is therefore better to split the prompt into layers:

• Must be preserved: core character identity, color palette, signature items, and art style.
• Allowed to change: pose, expression, camera angle, background, props, and scene.
• Explicitly forbidden: realism, outfit changes, hair color changes, extra hands or fingers, and confusing composition.

This helps the model establish clearer control boundaries.

3. Core methodology: the image answers “who,” and the prompt answers “what to do”

The most effective basic principle in character image generation is:

When a reference image is available, do not repeatedly rebuild the character’s identity with text. Use the image as the primary reference and use the prompt to describe the desired change.

This principle contains two key points.

1. The reference image anchors identity

A reference image directly provides visual information. Compared with text, an image can convey more accurately:

• Character proportions;
• Facial structure;
• Outfit details;
• Color relationships;
• Line style;
• Signature items;
• Overall temperament.

When the model sees a reference image, it is more likely to maintain character consistency. This is especially important for repeated generation, batch generation, or creating a series of character stickers, where a reference image is more stable than a long text prompt.

2. The prompt controls variation

A reference image usually shows only one fixed pose or one fixed frame. If the user wants the character to perform a new action—such as sitting, running, hugging, eating, kicking, making a heart gesture, lying down, or turning around to look back—the prompt needs to clearly specify:

• The subject of the action;
• Body orientation;
• Limb positions;
• Expression state;
• Gaze direction;
• Composition;
• Camera distance;
• Background environment;
• Style requirements;
• Negative constraints.

In other words, the prompt should not repeatedly explain who the character is. It should focus on what should change in this generation.

4. Preparing reference images: the first step to character consistency

The quality of the reference image directly determines the upper limit of the generated result. If the reference image is blurry, shows only one angle, has a cluttered background, or does not fully show the character’s features, even a detailed prompt may struggle to maintain stability.

1. Prefer clear, front-facing character images

An ideal reference image should have the following characteristics:

• The character is clearly visible and not blocked;
• The face, outfit, and color palette are recognizable;
• The resolution is high and details are not overly blurred;
• The background is as simple as possible to reduce interference;
• The character’s proportions are clear, making it easier to judge the head-to-body ratio;
• Signature items are complete, such as hair accessories, tail, cape, weapons, or shoes.

If only low-quality images are available, they can first undergo basic repair, upscaling, denoising, or cropping before being used as reference inputs.

2. Multiple angles are more stable than a single image

If possible, using multiple reference images can significantly improve consistency. For example:

• Front view;
• Side view;
• Back view;
• Half-body view;
• Full-body view;
• Sticker expression sheet;
• Outfit detail image;
• Signature prop image.

Multi-angle references help the model understand the character’s three-dimensional structure and reduce structural errors when the character turns, appears from the side, is shown from behind, or adopts complex poses.

3. Character sheets are often better base assets than a single finished illustration

A single finished illustration may include a specific scene, pose, or lighting setup. If used directly as a reference, the model may also copy the background or pose. A better approach is to use structured character assets such as character sheets, three-view sheets, full-body designs, or outfit breakdown images.

These assets have several advantages:

• They contain more complete character information;
• They introduce less pose interference;
• They make it easier to extract identity features;
• They are easier to reuse for action changes;
• They are more suitable for building a character asset library.

4. Create a character feature checklist

When preparing reference images, it is also useful to create a character feature checklist. For example:

Character name:
Core palette: primary color, secondary color, accent color
Hair: length, color, accessories
Eyes: color, shape, expression tendency
Outfit: top, bottom, shoes, outerwear
Signature items: tail, weapon, accessory, badge
Art style: chibi, anime, flat, line art, thick paint
Temperament: cute, lazy, tsundere, calm
Forbidden: realism, outfit change, hair color change, extra limbs

This checklist can serve as the basis for a prompt template and can also be used for team collaboration, batch generation, and quality checks.

5. Image-to-image workflow: from reference image to target image

Image-to-image generation is not simply uploading an image and letting the model freely reinterpret it. The process needs to use prompts and parameters to clarify which parts should be preserved and which parts should change.

1. Upload the character reference image

First, input the character reference image into the image generation tool. The tool can be a general-purpose image generator, an intelligent programming assistance tool, a creative platform, or a local generation pipeline. The tool type is not the most important factor. What matters is whether it supports image reference, pose control, inpainting, or style transfer.

After uploading, observe whether the model already recognizes the character’s core features. If the model clearly changes the hair color, outfit, or signature items, the reference image may not have enough influence, or the prompt may not emphasize identity preservation strongly enough.

2. Set the image reference strength

Different tools expose reference control in different ways, but it can generally be understood as “reference strength” or “repainting amount.”

General principles:

• If the character identity should remain as unchanged as possible: use a higher reference strength and keep the prompt simple.
• If the pose and action should change significantly: the identity constraint can be reduced appropriately, but the prompt must clearly state which features must be preserved.
• If the goal is to create something completely new: the reference image can be used as a style reference rather than an identity reference. In this case, the result is closer to “borrowing the style.”
• If creating a meme or sticker: prioritize character recognizability first, then adjust the action and composition.

If the tool supports layered control, it may be possible to control the influence of the face, outfit, pose, and background separately.

3. Explicitly prevent copying the original pose

If the character needs to break away from the original standing, sitting, or facial pose, the prompt should include expressions such as:

Do not use the original pose.
Do not keep the original standing pose.
Do not copy the original background.
Do not keep the original composition.

These constraints are especially important when the action change is large. For example, if the reference image shows a standing character but the target image requires the character to sit on the ground holding a bowl, the model may produce an unnatural mix of standing, sitting, and lying poses if it defaults to following the original pose.

4. Use inpainting for local detail problems

If the overall composition is mostly correct but some local areas have problems, inpainting or local repair can be used. Common issues include:

• Incorrect hand structure;
• Broken cutlery shape;
• Abnormal food proportions;
• Asymmetric eyes;
• Messy clothing patterns;
• Wrong orientation of a signature item;
• Missing tail or hair accessory;
• Distorted text, logo, or badge.

The idea behind inpainting is to keep the surrounding area unchanged and regenerate only the problematic region. This reduces character drift that can happen when the entire image is regenerated.

6. Prompt structure: character identity, action, composition, style, and constraints

A high-quality prompt should not simply pile up adjectives. It should have a clear structure. A recommended template is:

Preserve the character identity from the reference image: hair color, hairstyle, eye color, outfit, color palette, signature items.
Art style: anime, chibi, cute, clean line art, simple background.
Action: The character sits sideways on the ground, body facing the left side of the frame, legs bent and tucked to the side.
Expression: Innocent yet self-assured.
Prop: Holding a large bowl with both hands; the bowl contains white rice.
Composition: Subject centered slightly to the left; simple layout suitable for a meme.
Forbidden: Do not copy the original standing pose, do not change the core color palette, do not use a realistic style, do not create extra limbs.

1. Character identity section

The character identity section locks in stability and should be placed near the beginning of the prompt. It can include:

• Whether to preserve the character from the reference image;
• Which visual features must be kept;
• Whether outfit changes are forbidden;
• Whether color palette changes are forbidden;
• Whether signature items should be retained.

Example:

Preserve the reference character: long blue hair, blue-and-white palette, cute style, signature accessory, original character proportions.

If a character has many fixed elements, it is not recommended to write too many modifiers at once. Prioritize the most important recognizable features.

2. Art style section

The art style section unifies the visual language. Common expressions include:

• Anime;
• Chibi;
• Cute;
• Flat illustration;
• Clean line art;
• Simple background;
• Meme style;
• Sticker style.

If the reference image already has a clear style, the style description can be simplified so that the model is not pulled away from it by too many text instructions.

3. Action section

The action section should clearly describe body relationships and spatial relationships. It should include:

• Character position;
• Body orientation;
• Limb state;
• Head direction;
• Gaze direction;
• Relationship with other objects.

For example:

The character sits sideways on the left side of the frame, body facing the front-left, legs bent and tucked to the side, holding a bowl with both hands.

This is more specific than simply saying “the character is sitting,” and it usually produces more stable results.

4. Expression section

The expression section should match the character’s temperament. Common expressions include:

• Innocent;
• Self-assured;
• Happy;
• Pouting;
• Surprised;
• Shy;
• Confident;
• Sleepy;
• Serious;
• Proud.

Expression descriptions should not be too long, or they may conflict with the action. For example, if the character must hold a bowl with both hands and also perform a complex gesture, the model may struggle to satisfy both conditions.

5. Prop section

Props are often a source of errors in character images. The description should clearly define:

• Prop type;
• Prop quantity;
• Prop material;
• Prop position;
• Relationship between the prop and the hands;
• Prop proportions.

For example:

Holding a large white bowl with both hands; the bowl contains white rice; the bowl is slightly larger than the character’s head.

If a prop has a complex structure, it is better to generate it in stages: first generate the character and action, then generate the prop separately, and finally composite or inpaint it.

6. Composition section

The composition section determines whether the image fits the intended use case. Common requirements include:

• Subject centered;
• Empty space for text;
• Close-up;
• Half-body;
• Full-body;
• Low angle;
• Top-down view;
• Eye-level view;
• Simple background;
• Suitable for adding text;
• Suitable for cropping into stickers.

For example:

The character is placed slightly to the left of the center, the background is clean, and a small amount of empty space is left at the top, suitable for a meme.

7. Negative constraint section

Negative constraints are used to exclude unwanted results. Common constraints include:

No realism.
No 3D.
Do not change the hair color.
Do not change the outfit color palette.
Do not use the original pose.
Do not copy the original background.
No extra fingers.
No incorrect limbs.
No complex background.
No garbled text.

Negative constraints should not be too long, or they may dilute the most important instructions. Prioritize the items that are most likely to go wrong.

7. Strategies for complex actions and multi-character compositions

1. Break down complex actions

If the target action is complex—for example, the character sits, holds a bowl, turns the head, blinks, and has food in the background—it is better to split the process into steps:

1. Generate the character’s basic identity first;
2. Generate the sitting pose and body orientation next;
3. Add the action of holding the bowl;
4. Adjust the expression and details last.

This step-by-step approach is usually more stable than entering all conditions at once.

2. Describe multi-character compositions separately

When multiple characters appear in the image, do not rely on one vague prompt. Describe each character separately:

• Character A’s position;
• Character B’s position;
• Character A’s action;
• Character B’s expression;
• Whether the characters interact;
• Relative size proportions;
• Front-and-back overlap relationships.

For example:

Character A stands sideways on the left side of the frame, hands on hips, with a proud expression.
Character B crouches on the right side, hands raised, with a surprised expression.
Both characters keep the same art style; the background is simple, and a proper distance is maintained between them.

Multi-character generation can easily cause identity confusion. It is best to provide an independent reference image for each character and clearly specify their spatial positions in the prompt.

3. Memes should prioritize recognizability

Meme images often depend on character identity and expressive action to create humor. When generating a meme, prioritize the following:

• The character is recognizable at a glance;
• The action is exaggerated but structurally clear;
• The expression has emotional intensity;
• The background is simple;
• Props do not block the main subject;
• The image leaves space for text.

If the character’s identity is weakened by the action, the meme effect will drop significantly. In that case, reduce action complexity and restore character recognizability first.

8. Common failure types and how to fix them

1. Character identity drift

Symptoms: hair color changes, outfit changes, signature items disappear, or the face no longer resembles the character.

Fixes:

• Increase the influence of the reference image;
• Reduce the number of descriptive elements;
• Add instructions to preserve core character features;
• Add negative constraints such as “do not change hair color or outfit”;
• Use multiple reference images;
• Start with a small variation, then gradually adjust the action.

2. Action does not match expectations

Symptoms: the prompt asks for sitting but the model generates standing; the prompt asks for a side view but the model generates a front view; the prompt asks for holding a bowl but the model generates waving.

Fixes:

• Use more specific body descriptions;
• Clearly state body orientation;
• Clearly state limb positions;
• Add “do not use the original pose”;
• Reduce the share of style description and increase the priority of action description;
• Use a pose reference or sketch as assistance.

3. Art style drift

Symptoms: the reference image is anime style, but the result becomes realistic, 3D, or thick-painted.

Fixes:

• Clearly state “keep the anime art style” in the prompt;
• Reduce descriptions of complex lighting;
• Add instructions such as “clean lines, simple, flat feeling”;
• Add negative constraints such as “no realism, no 3D, no complex shadows.”

4. Hands or props break down

Symptoms: abnormal finger counts, deformed bowls, chaotic food structures, or misplaced chopsticks.

Fixes:

• Simplify the prop structure;
• Avoid complex hand gestures;
• Use inpainting;
• Generate the prop and hands separately;
• Reduce the density of image details;
• Clearly specify the quantity and position of the prop.

5. Background distracts from the subject

Symptoms: the background is too complex, the character is overwhelmed, or irrelevant elements appear.

Fixes:

• Add “simple background”;
• Specify a solid-color or light-colored background;
• Add “no clutter”;
• Use inpainting to remove background interference;
• Generate the character first, then generate the background separately.

9. Character asset library and prompt template management

If the same character is generated frequently, it is better to build a character asset library rather than writing prompts from scratch every time.

1. What a character asset library should include

Each character can have a standard profile:

Character name:
Reference images: front, side, back, half-body, sticker sheet
Core features: hair color, eye color, outfit, signature items
Palette: primary, secondary, accent
Art style: anime, chibi, flat, meme
Common expressions: happy, innocent, proud, sleepy
Common actions: standing, sitting, holding an object, heart gesture
Negative constraints: realism, outfit change, hair color change
Prompt templates: base version, action version, meme version

2. Templatization improves reuse efficiency

Prompts can be split into replaceable modules:

[Character identity] + [Art style] + [Action] + [Expression] + [Prop] + [Composition] + [Negative constraints]

For example:

Preserve the character identity from the reference image; keep the blue palette and signature items.
Art style: chibi, cute, clean lines.
Action: sitting sideways on the left side of the frame, legs bent and tucked to the side.
Expression: innocent yet self-assured.
Prop: holding a large bowl with both hands.
Composition: subject centered slightly to the left; simple background.
Negative constraints: do not copy the original standing pose, do not use a realistic style, do not change the core palette.

This structure is easy to modify and can be reused quickly across different scenarios.

3. Use version management

Character generation often requires multiple iterations. It is recommended to keep the key information for each generation:

• Reference image version;
• Prompt version;
• Generation parameters;
• Output images;
• Problem records;
• Fix strategies;
• Final selected result.

This avoids repeating trial and error and reduces misunderstandings in team collaboration.

10. Content compliance and creative boundaries

Although character image generation is mainly a creative activity, compliance boundaries still matter.

1. Respect character and asset rights

If a character comes from another creator, an adapted work, or community fan creation, pay attention to the scope of authorization, attribution requirements, and usage restrictions. This is especially important for commercial use, public release, or further redistribution.

2. Avoid misleading generation

Generated images should not be used to fabricate real people, impersonate others, create false information, or mislead the public. Character fan works should clearly indicate their derivative nature so that readers do not mistake them for official materials.

3. Pay attention to boundaries involving people and imagery

For content involving real people, minors, sensitive imagery, or potentially controversial subjects, remain cautious. Character creation should be based on respect, legality, and appropriate expression.

4. Keep technical use transparent

When publishing generated content, you can follow platform requirements and note that the image was created with AI assistance. This helps with content governance and reduces the risk of misunderstanding.

11. Summary: do not memorize prompts; build a generation logic

The key to character image generation is not finding a mysterious, universal prompt that can reproduce every character with one click. The key is building a stable generation logic.

It can be summarized as follows:

1. When a reference image is available, prioritize image-to-image generation or image reference.
2. The reference image handles character identity, while the prompt handles action variation.
3. Do not repeatedly describe the character with long text; focus on describing the changes in the current image.
4. When the action changes significantly, clearly state “do not use the original pose.”
5. Write character identity, art style, action, expression, props, composition, and negative constraints in separate layers.
6. Generate complex images step by step, and use inpainting first when local problems appear.
7. For long-term creation, build a character asset library and prompt templates.
8. When publishing generated content publicly, pay attention to copyright, authorization, and compliance labeling.

Once this method is established, character image generation changes from “lucky spell casting” to a reproducible, iterable, and collaborative engineering workflow. Whether you are creating character stickers, fan illustrations, memes, or character visual assets, this approach is more stable, efficient, and sustainable.