The Pelican on a Bicycle: A Prompt-Based Health Check for Model Capabilities

1. The Phenomenon: How a Pelican Became the Community’s Thermometer

Whenever a new large model is released, a familiar scene plays out in technical communities: developers spontaneously type the same prompt—“Draw a pelican riding a bicycle”—and then gather around to inspect the model’s output. This practice, colloquially known as the “pelican on a bicycle” test, was initially driven organically by community developers and eventually evolved into regular generation contests: everyone uses the same prompt to generate images, then posts the results for side-by-side comparison.

Its popularity is no accident. Take the recent launch of a frontier multimodal model as an example—community feedback generally indicated that its performance was inconsistent. The same task would sometimes be executed beautifully, and other times produce elementary mistakes. This kind of “intelligence fluctuation” is hard to describe clearly in a sentence or two, but the pelican-on-a-bicycle test makes this vague impression intuitive: once the image is posted, the model’s state is immediately apparent.

2. Why “Pelican on a Bicycle” Specifically

A seemingly casual test task actually hits two criteria for a “good test”: simple enough, yet complex enough.

2.1 The Simple Side: Intuitive, At-a-Glance Judgment

Many complex model capabilities—long-context memory, tool-call orchestration, multi-step reasoning—are hard to judge in seconds from a single sentence. “Pelican on a bicycle” is different: anyone with usage experience can give an intuitive evaluation within seconds:

• Whether the pelican’s legs align with the pedals;
• Whether the structural relationship between wheels and frame is reasonable;
• Whether multiple generations are stylistically consistent and structurally stable;
• Whether there are obvious “tell-tale” failures, such as an extra leg, a floating pedal, or wheels disconnected from the body.

This kind of evaluation doesn’t depend on professional benchmark sets or scoring scripts. Everyone can participate, and the传播 cost is extremely low—this is precisely why it has become community culture.

2.2 The Complex Side: Big Capabilities Behind a Small Prompt

Conversely, this task actually places considerable demands on the model. Broken down, it involves at least four capability dimensions:

1. Semantic understanding and commonsense association: The model needs to grasp both the morphological features of the pelican as a specific bird (large beak, throat pouch, body shape) and the mechanical structure of a bicycle (frame, two wheels, pedals, handlebars), and establish a reasonable compositional relationship between them.
2. Spatial relationships and geometric constraints: Riding is a strongly constrained scenario—leg posture must match pedal position, and the body’s center of gravity should fall near the seat. This kind of geometric alignment between multiple objects is exactly where generative models most often fail.
3. Temporal and sequential consistency: In Agent scenarios, the model often needs to generate multiple images consecutively or execute step by step (first draw structure, then add details, then modify parts), with each step remaining consistent with prior results—testing the ability to continuously maintain state.
4. Structured output and code organization: When the task is completed in SVG, HTML, or front-end code form, it additionally tests code organization skills—whether layer order, coordinate calculations, and element naming are clear and reasonable.

In other words, the pelican test is a problem with a “narrow entrance but great depth”: the prompt is only one sentence, yet the capabilities the model must invoke cover almost the entire core chain of multimodal generation and Agent execution. For a model that has entered the Agent stage, it serves very well as a “felt-sense thermometer.”

3. Why “Dumbing Down” Deserves Serious Attention

“Dumbing down” is a colloquial community term referring to performance fluctuations exhibited by the same model at different times or in different sessions. From an engineering perspective, this phenomenon is not mysterious; its causes are typically related to the following factors:

• Server-side dynamic scheduling and sampling strategies: Inference services may adopt different sampling configurations or scheduling paths under different loads, causing the output distribution to shift;
• Silent iterations of underlying versions: Model performance may drift with version updates, while users may not perceive the changes in time;
• Error amplification in long-chain tasks: In Agent scenarios, small per-step failures are gradually amplified along the task chain, ultimately manifesting as “getting dumber overall.”

Regardless of the cause, for developers who depend on model outputs, stability matters as much as peak capability. A model that is “sometimes good, sometimes bad” is far less valuable in automated pipelines than one that is “consistently good.” This is why the community needs a low-cost, high-frequency, horizontally comparable monitoring method—and the pelican test fills exactly that role.

4. From Meme to Method: A Deployable Evaluation Design

The value of the pelican test lies not in itself, but in the fact that it demonstrates an approach: “using a fixed task to probe model state.” Developers can absolutely engineer it.

4.1 Fixed Prompt and Sampling Protocol

First, fix the input to eliminate variable interference:

Task: Please use SVG code to draw a pelican riding a bicycle.
Requirements:
1. Complete bicycle structure: frame, front wheel, rear wheel, pedals, handlebars, seat;
2. The pelican's legs stay in contact with the pedals, with a posture consistent with riding;
3. Use semantic element naming (e.g., wheel-front, pedal-left);
4. Output as a single renderable SVG file.

Sampling protocol recommendation: use the same prompt and the same parameter configuration, sample N times consecutively (e.g., 10–20 times), and observe the distribution of results rather than single-run performance—single-generation quality is random, but distribution is the measure of stability.

4.2 A Structured Checklist

For each output, record along fixed dimensions:

Dimension	Checkpoint	Verdict
Structural completeness	Are frame, two wheels, pedals, handlebars, and seat all present	Pass/Missing
Geometric alignment	Do legs contact pedals; is center of gravity reasonable	Pass/Misaligned
Semantic correctness	Are pelican features (large beak, throat pouch) recognizable	Pass/Confused
Sequential consistency	Is structural style stable across multiple generations	Stable/Drift
Code quality	Is SVG structure clear; any redundant nesting	Good/Poor

4.3 Illustrative Output and Judgment Logic

A “passing” SVG output should at minimum present clear layering and naming structurally, for example:

<svg viewBox="0 0 400 300">
  <g id="bicycle">
    <circle id="wheel-front" cx="290" cy="220" r="45"/>
    <circle id="wheel-rear"  cx="110" cy="220" r="45"/>
    <path id="frame" d="M110,220 L200,140 L290,220 M200,140 L185,225"/>
    <circle id="pedal-crank" cx="185" cy="225" r="14"/>
  </g>
  <g id="pelican">
    <ellipse id="body" cx="200" cy="150" rx="55" ry="35"/>
    <path id="beak" d="M245,140 L300,150 L245,160 Z"/>
    <path id="leg-to-pedal" d="M210,175 L195,215 L185,225"/>
  </g>
</svg>

The key check is not artistic quality but logic: whether the endpoint of the leg path (leg-to-pedal) falls near the pedal (pedal-crank), whether the centers of the two wheels are roughly on the same horizontal line, and whether naming is semantic. These are precisely indirect evidence of whether the model’s “thinking is clear.”

4.4 Running It in a Pipeline: Lightweight Regression Testing

If a team calls model services through an API (for example, general large model services provided on platforms such as Baidu AI Cloud), the above process can be turned into a scheduled task:

# Pseudocode: batch sampling and structural checks
PROMPT = "Please use SVG code to draw a pelican riding a bicycle..."
def run_probe(model_client, n=10):
    results = []
    for i in range(n):
        svg = model_client.generate(PROMPT)
        score = {
            "wheels_complete": count_elements(svg, "wheel") >= 2,
            "leg_near_pedal":  check_geometry(svg),
            "render_ok":       try_render(svg),
        }
        results.append(score)
        save_to_storage(svg, metadata={"round": i, "ts": now()})
    return summarize(results)

A complete engineering loop includes four steps: scheduled batch sampling with a fixed prompt; writing each output’s SVG, latency, and version information to object storage or a logging service; automatically parsing the SVG with scripts for rule-based checks (whether elements are complete, whether coordinate constraints are satisfied); and when metrics are abnormal, notifying through monitoring alerts for manual review. In this way, the “pelican thermometer” upgrades from a community meme to a lightweight model capability regression test pipeline—extremely low cost, yet able to capture the signal that “the model isn’t right today” at the earliest opportunity.

5. Boundaries and Rationality: A Thermometer Is Not a Medical Report

Finally, it must be emphasized that the pelican test is a “thermometer,” not a “medical report.”

• Limited coverage: It primarily probes visual generation and structured output capabilities and cannot reflect long-text reasoning, code engineering, tool calling, or other dimensions;
• Sample size issues: Survivorship bias is inherent in community propagation—failure screenshots are more likely to be shared, which can easily lead to overly pessimistic estimates of model state;
• Subjectivity of evaluation: Unless rule-based or automated scoring is introduced, “seeing it at a glance” remains influenced by personal experience.

The rational approach is to treat it as a high-frequency, low-cost leading signal: when pelican images show anomalies, then launch standard benchmark sets for deeper root-cause analysis; when pelican images look normal, do not assume the model is fine in every respect. Combining felt sense with benchmarks is the way to form a complete judgment of model state.

6. Conclusion

The popularity of “pelican on a bicycle” is essentially a large-scale, self-organizing, zero-cost social experiment on model stability. It reminds all practitioners: in an era of rapid model iteration, “how strong it is at its best” is important, but “how stable it is day to day” is equally critical. For developers, rather than passively watching, it is better to borrow this approach and design a dedicated “pelican task” for your own business scenarios—one prompt, one checklist, and one scheduled pipeline can bring changes in model state fully into view.