A Multi-Model LLM Evaluation of Xiaohongshu Store-Visit Posts: High Scores Require More Than Good Prose

    This article presents a multi-model evaluation of Xiaohongshu-style store-visit review posts as a structured instruction-following task rather than a pure writing exercise. It describes a shared restaurant prompt, six hard constraints, and a seven-dimension scoring framework covering compliance, consistency, information disclosure, evidence use, persona stability, platform fit, and value judgment. It also examines an offline evaluation pipeline, raw-output handling, and common failure modes such as omitted queue-time facts and leaked reasoning. The results show why average scores alone are insufficient: production readiness depends on factual accuracy, complete constraint execution, clean delivery, and stable platform fit, not just fluent prose. Finally, the article outlines how to turn a one-off test into a reusable evaluation system for long-term model comparison.

    1. Why Evaluate Both “Does It Sound Right?” and “Is It Correct?”

    Asking a large language model to generate a Xiaohongshu-style store-visit post may look like a creative task. Give the model restaurant information and ask it to write vividly, with sensory detail and a sense of presence. Once the task is used for actual content production, however, it becomes clear that the hard part is not only language.

    A qualified store-visit review post is usually subject to several constraints at once:

    • The title must fit the platform’s reading habits.
    • Key information must not be omitted.
    • Facts such as price, portion size, and waiting time must not be rewritten.
    • Shortcomings must not be disguised as nonexistent strengths.
    • The tone must feel like a real experience, not advertising copy.
    • The ending must provide a clear judgment.
    • Total length and emoji count must also meet requirements.

    For these reasons, this kind of task is better treated as a test of structured instruction-following capability than as a pure prose contest. A model may produce a very smooth article while omitting the queue time. It may also include all the source material but append a large amount of generation-process explanation to the body, sharply reducing platform fit.

    The core question in this test can be summarized as follows: when multiple constraints exist at the same time, can the model reliably remember and execute every requirement?

    2. Test Material and Task Design

    The test used the same source material for a neighborhood restaurant to avoid prompt differences affecting results. The restaurant was a long-established local business with the following main details:

    • It had been operating for many years, and a business license from the late 1990s remained on the wall.
    • Its signature item was firewood-cooked wontons, priced at CNY 18 for 12.
    • During peak periods, the queue typically exceeded 40 minutes.
    • The owner had a direct manner but could remember regular customers’ preferences.
    • The restaurant accepted only cash and WeChat Pay.
    • An adjacent influencer-famous restaurant used pre-made meals; half a portion cost CNY 29.

    The model was assigned the persona of a food reviewer with a picky palate but a fair attitude, and it was explicitly instructed not to write a soft promotional article. To test stability under complex constraints, the task included six hard requirements:

    1. The title must be no more than 20 Chinese characters and must include a specific number.
    2. The body must mention the owner’s temper and the queue length.
    3. Only years provided in the source material may be used; the model must not infer additional years.
    4. The comparison with the adjacent restaurant may state facts only; it must not use insulting or inflammatory evaluations.
    5. The ending must clearly answer whether the restaurant is worth visiting.
    6. The full text must be 400–500 Chinese characters, with no more than six emojis.

    These requirements appear simple, but they cover title generation, fact extraction, tone control, comparative expression, length control, and conclusion output. The model must first understand the task, then plan the article structure, and finally verify length and format. If any stage fails, the final result is downgraded.

    3. The Seven-Dimension Scoring System: Looking Beyond the Total Score

    To avoid the misjudgment that “a high total score means comprehensive excellence,” the evaluation divided results into seven dimensions, each scored from 0 to 1.

    3.1 Compliance

    Compliance checks whether all hard requirements are met, including title length, the number requirement, key facts, length range, emoji count, and the ending judgment.

    This dimension is well suited to rule-based prefiltering. For example, string matching can confirm whether the body includes “40 minutes,” price information, and the year, while character counts can verify title and body length. Semantic requirements, such as whether a clear value judgment is given, still require review by a human or judge model.

    Hard constraints should not be checked only after the article has been generated. A more reliable pipeline is:

    1. task parsing → constraint checklist → content generation → rule validation → semantic review → final output

    If the model does not explicitly build a constraint checklist, a common problem is an article that looks complete but misses one key requirement.

    3.2 Consistency

    Consistency concerns whether the model remains faithful to the source material, especially information about price, quantity, time, payment methods, and the relationship between stores.

    For example, “CNY 18 for 12 wontons” must not become “a large bowl for a little over ten yuan,” and “more than 40 minutes” must not be softened into “occasionally requires queuing.” These seemingly minor rewrites directly affect credibility. To make an article smoother, models often summarize or polish; in factual content, however, overgeneralization can cause information distortion.

    Consistency evaluation should also check for contradictions between sentences. If the text says the restaurant accepts only cash and WeChat Pay but later says it accepts credit cards, that is a clear error. If it states that the queue exceeds 40 minutes and later says there is basically no wait, that is also a clear error.

    3.3 Information Disclosure

    Store-visit review content differs from ordinary food writing because it needs to explain information that affects consumer decisions. Queue time, price, payment methods, and service attitude are all matters readers need to know when making a decision.

    This dimension does not require the article to mechanically list every item in the source material. Instead, it requires the model to proactively present key facts while keeping the article readable. Negative information in particular must not be hidden to protect the recommendation effect.

    Truly valuable content does not merely tell readers whether the food tastes good. It also explains:

    • How much time the visit may cost.
    • What the approximate budget is.
    • Whether the service experience is stable.
    • Whether visitors need to prepare cash.
    • Whether the restaurant suits people seeking a comfortable environment.

    3.4 Evidence Use

    The evidence dimension evaluates whether the model uses concrete details from the source material instead of stacking vague adjectives.

    Expressions such as “has a lively atmosphere,” “smells especially good,” and “the owner has personality” are summary statements with limited persuasive power when used alone. By contrast, “a business license from 1998 still hangs on the wall,” “the queue exceeds 40 minutes at peak times,” and “CNY 18 buys 12 wontons” are more likely to create a verifiable sense of place.

    The more specific the details, the more the article reads like real observation. The more generic the adjectives, the more it resembles template generation. During evaluation, the key is to observe whether the model turns numbers, years, payment methods, and store relationships into natural narrative rather than simply piling them together.

    3.5 Persona Stability

    The task does not ask for an ordinary introduction; it asks for the style of a reviewer who is picky about food but fair. This role means the model should be willing to point out problems without exaggerating attacks merely to create conflict.

    A stable persona usually has three qualities:

    • It makes clear judgments about taste, price, and waiting time.
    • It acknowledges the restaurant’s shortcomings.
    • It remains restrained when expressing criticism and does not present personal feelings as absolute facts.

    For example, “The owner can be blunt; if service matters to you, take that into account” is specific and restrained. “The owner is terrible; anyone who goes will regret it” clearly exceeds what the source material supports.

    3.6 Platform Fit

    Platform fit is not only about whether the text sounds like Xiaohongshu. It also includes whether the text is suitable for mobile reading and real publishing.

    Common positive features include:

    • A short, clear title.
    • An opening that quickly presents the core selling point or conflict.
    • Short paragraphs that are easy to read on a phone.
    • Key information that is easy to scan.
    • A natural tone with some sense of personal experience.
    • No unnecessary task explanation, format notes, or generation process.

    In this test, one major issue was that some models included internal reasoning content in the output. If phrases such as “I need to satisfy the following requirements” or “Next, I will analyze this from several aspects” appear in the body, they severely damage platform fit even if the content itself contains no factual errors.

    3.7 Value Judgment

    A store-visit review post cannot stop at listing information. It must ultimately answer whether the restaurant is worth visiting. But “worth it” cannot be separated from conditions.

    A more mature judgment gives suitable audiences and limiting conditions, for example:

    • It is worth it if you value price and old-shop flavor and can accept the wait.
    • It may not be suitable if you are unwilling to queue or care a great deal about service.
    • For budget-conscious diners seeking traditional flavors, the cost-performance ratio is relatively high.
    • For people who care more about atmosphere and convenient payment, the appeal is limited.

    This kind of conclusion is more useful than a simple “highly recommended,” and it better fits the fair reviewer persona.

    4. The Most Commonly Overlooked Stage in the Evaluation Pipeline

    This test used a unified gateway, submitted requests one at a time, persisted raw results, and then performed offline scoring. This design has three advantages.

    First, every complete output remains available for review. Model generation is stochastic. If only the final score is saved, it becomes impossible to determine whether an issue came from the model, the judge, or preprocessing.

    Second, all models can be scored against the same standard. Online immediate scoring is easily affected by request order, context state, and temporary service fluctuations. Offline scoring is better suited for horizontal comparison.

    Third, the effects of special output formats can be analyzed separately.

    Some models output hidden reasoning markers or process content before or after the body. There are two ways to handle this:

    • Remove the process content first, then score the pure body.
    • Keep the raw output and let the judge decide which content belongs to the body.

    Both approaches are reasonable, but their conclusions are not the same. The first is closer to “the final product users would see”; the second is closer to “the model’s raw delivery result.” If the task objective is direct publishing, contamination by process content is itself a problem and should not be treated as negligible noise.

    Therefore, an evaluation report must clearly state whether length was measured on the raw output or on the body after process content was removed, and whether the judge saw the complete result or the cleaned result. Otherwise, different experiments cannot be compared fairly.

    5. The Most Notable Findings

    The clearest conclusion from this test is that the average-score gap between the top and bottom models was not extremely large, but the reasons for point loss were fundamentally different.

    Some models lost points on expression: sentences were not natural enough, or paragraph transitions felt somewhat template-like. Others lost points on process control: they mixed reasoning processes, task decomposition, and self-check content directly into the body. The latter problem is more serious, because the issue is not “mediocre writing” but a non-compliant delivery format.

    Another notable phenomenon is that a single capability cannot represent overall performance. A model may be factually accurate and thorough in disclosure yet still unsuitable for direct publishing. Another may produce vivid copy but omit the queue time or incorrectly rewrite the price.

    Therefore, model selection should not rely only on the average score. It should also examine the distribution across dimensions. In actual production, platform fit and compliance are often more important than ornate prose. A somewhat plain but complete result is usually more usable than an engaging article with factual errors.

    6. Turning a One-Off Evaluation into a Reusable System

    For long-term model comparison, consider establishing the following mechanisms.

    Build a Fixed Test Set

    Do not rely on a single prompt. Prepare source material across categories such as food, travel, consumer electronics, and home goods, and define price, time, quantity, forbidden expressions, and conclusion requirements for each category.

    Separate Hard and Soft Metrics

    Word counts, numbers, years, and key facts can be checked automatically. Tone, naturalness, and platform feel require judge models or human review. The two should not be merged into one vague total score.

    Record the Complete Run Environment

    Record the test date, model version identifier, random parameters, request failures, retry counts, and judge version. Model services are updated continuously. Without environment records, later retests cannot explain changes in scores.

    Use Multi-Round Review

    In the first round, rule-based programs filter obvious errors. In the second round, a judge model scores the results. In the third round, disputed samples are checked by humans. This reduces the influence of a single judge’s preferences on the result.

    Add Stability Metrics

    Generate multiple outputs from the same model on the same prompt and observe score variance. This is more meaningful than a single highest score. A model truly suitable for production must not only write well occasionally; it must consistently meet requirements.

    7. Conclusion: From “Prose Ranking” to “Delivery Reliability”

    This store-visit writing test shows that differences between large models are not limited to vocabulary richness and sentence elegance. The more critical capabilities include identifying constraints, preserving facts, disclosing shortcomings, controlling length, maintaining the persona, and outputting a body that can be delivered directly.

    If models are used for content production, the recommended approach is a combination of automated rule checks, semantic judging, and human spot checks, rather than relying on a single total score. For batch publishing scenarios, the priorities are usually:

    1. No factual errors.
    2. No missing key constraints.
    3. No leakage of reasoning process.
    4. Stable platform fit.
    5. Only then compare language style and creativity.

    In other words, what is often overestimated is not a model’s prose quality, but the assumption that one smooth-looking article proves stable production capability. The ultimate goal of model evaluation is not to find the single best writing sample, but to identify tools that can consistently deliver acceptable results under complex constraints.