Multi-Model Housewarming Blessing Test: Why Higher-Priced Models Also Lose Points
This article reports a hands-on test of ten text generation models on a housewarming blessing task. The task required two messages for different recipients, with limits on length, banned phrases, and topics. The article explains the seven scoring dimensions and the main failure types, and it shows why model rankings do not always follow price or positioning. It also outlines a repeatable evaluation process that combines automated checks, blind human review, and failure tagging.
1. This Is Not a Simple Blessing Writing Task
Many people treat housewarming blessings as an easy copywriting job. Give a model a scene, and it outputs a few lucky-sounding lines. But when the requirements match a real client order, the difficulty rises noticeably.
The test used a specific scenario. A client, Xiao Lin, is moving into a renovated second-hand apartment. The home is not large, and it is not a luxury property in the usual sense. The models had to write two blessings for different purposes:
- The first is a message to a former colleague. It must be no more than 50 Chinese characters and sound natural. It may lightly tease the changes made to the old apartment, but it must stay warm.
- The second is a card written with a cash gift for a college roommate. It must be 80 to 100 Chinese characters and include one concrete image related to the second-hand apartment or the renovation.
The task also set clear bans:
- Common stock phrases are not allowed, such as “Wishing you good fortune in your new home,” “May this blessed place shine,” or “An auspicious day to settle down.”
- The text must not mention floor area, price appreciation, or property prices.
- Rhyme is not required, and the text should avoid slogan-like parallel expressions.
- The two blessings must serve clearly different purposes. Changing only the greeting is not enough.
So the real test is not whether a model can write pleasant words. It tests whether the model can understand relationships, control tone, and deliver a message ready to send, under several constraints at once.
2. Test Setup and Scoring Method
The test included 10 text generation models: 6 positioned as high-capability and 4 as lower-cost. All models received the same task brief, the same conversation context, and the same output requirements to limit the effect of prompt differences.
Scoring used seven dimensions, each worth up to 1 point:
- Task completion: Whether the model output two blessings as required, each suited to a WeChat message or a greeting card.
- Hard constraint compliance: Whether the text met the length limits and avoided banned words and banned topics.
- Scene specificity: Whether the model understood the renovated second-hand home, rather than writing generic blessings.
- Relationship fit: Whether the model distinguished the distance in a former colleague relationship from a college roommate relationship.
- Language naturalness: Whether the text sounds like something a real person would send or write.
- Emotional sincerity: Whether the text feels warm without overly emotional or exaggerated praise.
- Usability: Whether a user could send the text with little or no editing.
The final score is a weighted total of the seven dimension scores, ranging from 0 to 1. The highest recorded score was 0.88 and the lowest was 0.22, a gap of nearly fourfold. This result shows that differences among models in open
