Seven-Dimension Review of Six Flagship LLMs on Xiaohongshu Note Title Generation

    This article reports a seven-dimension evaluation of six flagship large language models on one task: generating platform-style note titles for Xiaohongshu, a Chinese lifestyle-sharing platform, from a given topic. It covers the test setup, the seven scoring dimensions, the overall results, and the pattern of score gaps between models. It also explains why response time did not track output quality, and offers three practical recommendations for developers choosing models for content generation.

    1. Why We Ran This Evaluation

    In content generation work, “given a topic, produce several platform-style note titles” is one of the most common tasks, and one of the easiest ways to expose differences between models. It looks simple, but it tests several things at once: compliance awareness, style control, factual support, and judgment about commercial value. Looking only at whether the output reads smoothly or whether the response comes back quickly often leads to incomplete conclusions.

    To get more reliable results, this evaluation used fixed test conditions. Six widely available flagship large language models were tested on the same set of topics, covering beauty, home, digital products, and workplace content. All models used the same prompt template, the same output format requirements, and the same scoring criteria. The evaluation aimed to answer two questions. First, which models work out of the box for this kind of task? Second, which weak dimensions would actually affect business usability?

    2. Seven Evaluation Dimensions

    The scoring system has seven dimensions. Each one is scored independently, and the overall score is the mean of the seven.

    1. Compliance: Whether the title touches platform red lines, such as exaggerated claims, absolute wording, or false efficacy claims.
    2. Consistency: Whether the titles in one batch stay stable in style, sentence structure, and information content.
    3. Disclosure: For content involving promotion, partnerships, or product trials, whether the title appropriately discloses the creator’s identity or commercial interest.
    4. Evidence: Whether the numbers and effect descriptions in a title are supported by the body text or source material, rather than fabricated.
    5. Persona fit: Whether the tone and voice match the intended creator persona, such as an ordinary user sharing experience or a professional blogger.
    6. Platform fit: Whether the title follows the platform’s implicit community norms, including title length, emoji use, and topic tags.
    7. Value density: Whether the information and appeal match the title’s length, meaning whether every character does useful work.

    3. Overall Results

    Table 1. Scores by model across the seven dimensions and the overall average (0 to 1 scale).

    Model Compliance Consistency Disclosure Evidence Persona Platform Value Average
    Model A 0.98 0.96 0.98 0.95 0.85 0.93 0.91 0.94
    Model B 0.96 0.90 0.95 0.95 0.82 0.89 0.88 0.91
    Model C 0.98 0.91 0.97 0.84 0.86 0.89 0.90 0.91
    Model D 0.97 0.92 0.98 0.72 0.84 0.91 0.89 0.89
    Model E 0.98 0.68 0.98 0.84 0.82 0.80 0.86 0.85
    Model F 0.87 0.66 0.93 0.72 0.54 0.62 0.42 0.68

    4. Reading the Score Gaps: Three Tiers

    1. Top tier: close competition, choose by preference

    Model A leads the second-place model by 0.03 points, with a score of 0.94. This margin does not represent a real capability gap in a seven-dimension average. The top three models (0.94, 0.91, and 0.91) sit in a very narrow range. They are all usable, and the choice among them mostly depends on stylistic preference. Their differences lie in small stylistic tendencies rather than clear weaknesses. When choosing among them, engineering factors such as ecosystem compatibility and inference cost can take priority.

    2. Middle tier: usable, but each has a weak spot that needs a safety net

    Models D and E score between 0.85 and 0.89. Most of their dimensions are normal, but each has a clear low point. Model D scores only 0.72 on evidence. It tends to produce titles that look attractive but lack support, so production use requires a fact-checking step. Model E drops to 0.68 on consistency. Titles in the same batch vary in length and drift in style, so it needs stricter output format constraints.

    3. Gap and collapse zone

    The real dividing line falls between Model E and Model F, where the average drops from 0.85 to 0.68. Model F’s problem is not one weak dimension. Several dimensions fail together, and its value density score is only 0.42. This dimension measures the information density and appeal of a title. A score of 0.42 means its titles are often wordy, empty, and unfocused. One dimension that works like a delivery-completeness check pulls the average down sharply. This suggests that Model F has structural weaknesses on short-text tasks that require precise style control. We do not recommend using it directly in production.

    5. The Counterintuitive Finding: Fast Does Not Mean Good

    The evaluation also recorded response times. Their relationship to ranking was close to zero.

    • The fastest model, Model E (about 9 seconds), ranked fifth.
    • The slowest model, Model D (about 112 seconds), ranked fourth.

    This gives an important engineering takeaway: inference speed and output quality have no stable positive correlation on short text generation tasks. A fast model does not necessarily sacrifice quality, and a slow model does not necessarily deliver better results. Long response times may come from deep reasoning chains, and those chains add limited value to one-sentence titles. When choosing a model, treat response time as a cost factor, not a quality signal.

    6. Three Recommendations for Developers

    1. Do not decide based on the average score alone. Check the variance across dimensions first, and identify whether the weak dimensions affect your critical business path. For example, product-promotion scenarios are sensitive to evidence, while brand scenarios are sensitive to disclosure.
    2. Design different safeguards for each tier. Top-tier models can go directly to production. Middle-tier models need format validation or fact-checking. Models below the gap line should only be used in low-cost scenarios.
    3. Build your own evaluation set. General benchmark scores cannot replace testing on real business scenarios. Turn the seven-dimension framework into a reusable evaluation script and run regression tests regularly. A simple score aggregation example is shown below.
    1. dimensions = ["compliance", "consistency", "disclosure",
    2. "evidence", "persona", "platform", "value"]
    3. def overall(scores: dict) -> float:
    4. return sum(scores[d] for d in dimensions) / len(dimensions)
    5. def flag_weak(scores: dict, threshold=0.75):
    6. return {d: v for d, v in scores.items() if v < threshold}

    7. Conclusion

    The core conclusion of this seven-dimension review can be stated in one sentence: the key information for model selection lies in the shape of the score gaps, not in the size of the average scores. A tightly clustered top tier shows that the market has matured considerably, so the real decision should turn to cost and engineering fit. A collapse in a single dimension, such as the 0.42 value density score, often predicts that a model is unusable for a specific task more reliably than a lower average does. We hope this framework saves you trial-and-error costs.