Making LLMs Speak Warmer: Three Approaches to Empathetic Chinese Responses

When building intelligent customer service, emotional companionship, or personal assistant products, developers often encounter the same problem: a user asks a question or complains with emotion, but the model responds directly with steps or logical conclusions, lacking reassurance and empathy. Such an answer may not be wrong, but it makes users feel ignored and can even escalate their emotions. This article starts with the causes and then introduces three improvement paths, explaining their applicable scenarios and risks.

1. Why Models Do Not Naturally Comfort Users

Large language models are usually trained in two stages. In pretraining, the model learns to predict the next token from massive corpora, so it is good at knowledge retrieval and logical reasoning. In alignment stages such as RLHF or SFT, evaluation criteria often focus on being “helpful” and “harmless.” When the model sees a user say, “Why is this feature broken again?”, it tends to interpret the input as a problem to solve and immediately provides troubleshooting steps, ignoring that the user first needs to feel understood.

The difficulty in Chinese-language contexts is that emotion is often hidden in particles, rhetorical questions, and omitted subjects. For example, “Can your damn system still be used?” is literally a question, but it actually expresses anger and disappointment. Reading this layer of meaning requires strong contextual understanding and familiarity with Chinese expression habits. If developers do not set a persona for the model and do not provide emotional corpora as constraints, the default output is often neutral and stiff “machine language.”

Once the causes are clear, the improvement approach becomes clear: let the model first identify emotion, then decide the order and tone of the response. The three methods below all center on this idea.

2. Prompt Engineering: The Lowest-Cost First Step

For small and medium-sized teams, optimizing prompts is the fastest approach. It does not require training data; changing a system prompt can take effect, making it suitable for validating the idea first.

The following code demonstrates how to guide the model through a system prompt to comfort emotions first. The code includes timeouts, retries, and return-value validation to avoid direct failures in production due to a single network fluctuation. The API URL and key are read from environment variables and are not written into the code.

import os
import time
import requests
API_URL = os.environ["LLM_API_URL"]
API_KEY = os.environ["LLM_API_KEY"]
SYSTEM_PROMPT = """
You are an intelligent assistant skilled in Chinese communication.
1. First determine the user's emotional state, such as anger, anxiety, sadness, confusion, or calm.
2. When the user's emotion is clear, first respond to the feeling in one or two natural sentences, then provide a solution.
3. For personal safety, urgent failures, factual corrections, or when the user explicitly asks for a direct conclusion, prioritize the conclusion and steps, and include where to get help.
4. Avoid templated phrases, do not make fabricated promises, and do not make decisions for the user.
"""
def call_llm(user_input, max_retries=2, timeout=10):
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {API_KEY}",
    }
    payload = {
        "model": "general-large-model",
        "messages": [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_input},
        ],
        "temperature": 0.6,
    }
    for attempt in range(max_retries + 1):
        try:
            resp = requests.post(API_URL, headers=headers, json=payload, timeout=timeout)
            resp.raise_for_status()
            content = resp.json()["choices"][0]["message"]["content"]
            if not isinstance(content, str) or not content.strip():
                raise ValueError("Model returned empty content")
            return content.strip()
        except (requests.RequestException, KeyError, IndexError, ValueError):
            if attempt == max_retries:
                return "Sorry, the service is temporarily unavailable. Please try again later or contact a human agent."
            time.sleep(2 ** attempt)
if __name__ == "__main__":
    print(call_llm("Your system keeps throwing errors—can it even be used?"))

There are several points to note when using this code. First, setting temperature to around 0.6 is sufficient. Too high makes comforting statements more random, while too low easily turns them into fixed templates. Second, rule 3 in the system prompt is important. If the user asks about medication dosage, system downtime, or account theft, comforting first and then giving a conclusion can delay the matter. Third, model outputs may occasionally fail to meet expectations, so real conversation samples should be spot-checked before launch.

The main limitation of the prompt approach is stability. Under complex or extreme emotions, the model may produce empty comforting words or over-empathize where it should not. This is where the following method can supplement knowledge.

3. Retrieval-Augmented Generation: Connecting a Professional Script Library to Responses

When prompts are insufficient for complex scenarios, retrieval-augmented generation (RAG) can be introduced. The idea is to organize high-quality customer service scripts, basic communication techniques, and scenario-based empathy templates accumulated by the company into a knowledge base, retrieve suitable scripts before each response, and let the model merge them.

Implementation usually consists of three steps.

Step 1: Build an emotion label system. It is recommended to first define a maintainable set of labels, such as anger, anxiety, sadness, confusion, neutral, and high-risk. Too many labels increase annotation costs and make them harder for the model to distinguish. The high-risk category should be handled separately and should not enter the ordinary script flow. Emotion classification can be implemented with a small classification model or structured output from a large model, and accuracy should be validated with human-labeled samples.

Step 2: Dual-path recall. After receiving user input, one path retrieves solution documents and takes top-k relevant content, usually 3 to 5 items; the other path retrieves corresponding comforting scripts from the script library based on the emotion label, also taking top-k. When merging the two result sets, three things need to be done: weighted ranking by emotion label match and business relevance; deduplication using a similarity threshold to avoid two nearly identical scripts appearing repeatedly; and retaining source information for troubleshooting.

Step 3: Set fallback strategies. If no script is found, use a default comforting template; if retrieval times out, fall back directly to the pure prompt path to ensure the response is not interrupted; if the emotion label is high-risk, skip ordinary scripts and directly trigger the human handoff flow.

The advantage of RAG is that scripts are controllable and comply with industry norms, reducing “random comfort” caused by free-form model generation. The disadvantage is that the system becomes more complex: it requires maintaining a vector store, label system, and recall logic, and each link can become a failure point. Therefore, it is suitable for teams with high script compliance requirements and fixed business scenarios.

4. Model Fine-Tuning: A Deep Alignment Approach

For products with high interaction quality requirements, such as psychological counseling and emotional companionship, prompts and RAG alone often cannot achieve natural and fluent results. In this case, supervised fine-tuning (SFT) of the base model can be considered.

1. Constructing the dataset

Data quality determines fine-tuning results. Samples should be multi-turn dialogues, each containing a neutral or negative user input and a response that “empathizes first, then answers.” Responses are best written by experienced customer service staff or counselors and reviewed a second time. The data should cover multiple situations, including complaints, anxiety consultations, simple questions, and greetings, to avoid the model learning only one way of speaking. Sample proportions also need attention: if empathy samples are too numerous, the model may begin every question with a paragraph of comfort.

2. Training method

Parameter-efficient fine-tuning methods such as LoRA (Low-Rank Adaptation) are recommended. They train only a small number of low-rank matrices, use less GPU memory, require shorter training cycles, and are easier to roll back. If you want to strengthen the influence of empathy segments, you can weight these tokens in the loss function, for example by setting the loss weight of empathy segments to 1.5 to 2.0. This value is only a starting point and needs to be determined through small-scale experiments while monitoring general capability metrics. If the weight is too high, the model may become overly sentimental or even fail to answer factual questions directly.

3. Validation method

The fine-tuned model needs two types of evaluation. The first is emotional metrics, such as using human blind evaluation to compare empathy before and after fine-tuning, or using a classification model to count the proportion of comforting statements in responses. The second is general capability regression testing, including factual question answering, instruction following, and coding tasks, to confirm that the model has not regressed due to fine-tuning. Only when both types of metrics meet the requirements is the model suitable for launch.

5. How to Choose Among the Three Approaches

The comparison table below can help teams choose based on actual conditions.

Dimension	Prompt engineering	RAG	Fine-tuning (LoRA)
Startup cost	Very low; change the prompt	Medium; requires a knowledge base and label system	High; requires labeled data and training resources
Time to launch	A few days	A few weeks	Weeks to months
Response latency	Basically no additional overhead	Adds retrieval time	Comparable to the original model
Controllability	Medium; depends on the model’s ability to follow instructions	High; script content can be audited	Medium; style is fixed but hard to modify item by item
Main risks	Templated responses; unstable under extreme emotions	Retrieval failure or label misclassification	General capability degradation; data bias
Suitable scenarios	General customer service, personal assistants	Industry customer service with high compliance requirements	Long-term companionship, counseling products

In actual projects, the three are usually combined. First use prompts to validate the direction, then use RAG to supplement scripts, and finally fine-tune core scenarios with sufficient data.

6. Deployment Architecture: A Mainstream Cloud Platform Example

In enterprise environments, emotional alignment solutions usually require a complete infrastructure. Taking a mainstream cloud platform as an example, common components include:

• Model hosting and inference: Use the platform’s model service to complete fine-tuning training and deployment, and configure elastic compute based on concurrency.
• Corpus and knowledge base storage: Use object storage to save customer service corpora and script documents, and use vector retrieval services for recall.
• Event-driven workflow: Use message queues and function computing to connect emotion classification, recall, and response generation. When the monitoring module detects high-risk negative emotions, it triggers a fallback strategy and hands off to a human agent.

Regarding retrieval latency, numbers in documentation and vendor marketing materials can only be used as references. Real metrics must be obtained through stress testing under your own data scale and index configuration. At launch, use P95 or P99 latency as the standard and set timeout thresholds.

7. Common Mistakes

Comforting without answering the question. The user asks about refund rules, but the model first gives three paragraphs of comfort. The solution is to put the conclusion first, or follow the comfort immediately with an actionable answer.

Repeated comforting scripts. The same sentence, “I completely understand how you feel,” appears repeatedly, and users feel it is a machine using templates. Increase script diversity and perform similarity deduplication in RAG.

Treating emotion labels as definitive conclusions. A classification model may misclassify humorous complaints as anger, or calm questions as anxiety. Labels should be used as reference signals, not the only basis, and key decisions require human review.

Ignoring high-risk scenarios. When users express thoughts of self-harm or harming others in a conversation, ordinary customer service scripts are completely insufficient. The system must have an independent identification and human handoff path and cannot rely on the model to judge on its own.

8. Safety Boundaries for Emotional Companionship and Mental Health Scenarios

Emotional companionship and mental health-related products involve users’ most sensitive information and must have clear boundaries.

• The product should clearly tell users that it is an AI system and does not replace professional psychological counseling or medical diagnosis.
• When expressions of self-harm, suicide, or harm to others are identified, immediately stop the regular conversation, hand off to a human, and provide information about local psychological assistance channels.
• Conversation data should follow the principle of minimization, collecting only necessary fields. Logs should be encrypted, retention periods should be set, and users should be able to request deletion.
• Personal information in training data needs to be desensitized, and the authorization scope of data sources should be confirmed.
• Before launch, complete the corresponding compliance review and filing according to local regulations.

These requirements are not optional. Even if the model has strong empathy capabilities, users must not be led to believe that it can assume the responsibility of professional support.

9. Pre-Launch Checklist

• Emotion classification has reached the preset accuracy threshold on human-labeled samples, and the confusion matrix has been recorded.
• High-risk expressions can be reliably identified and trigger the human handoff flow, and test cases have covered them.
• Answers to factual questions (prices, rules, medication dosage) prioritize conclusions and are not slowed down by excessive comforting.
• RAG has fallbacks for timeouts, no hits, and empty results, with logs available for investigation.
• The fine-tuned model has passed general capability regression tests, and comparison results with the baseline model have been archived.
• P95 and P99 latency for retrieval and model calls have been stress-tested at target concurrency.
• Privacy terms, data retention policies, and user notification copy have been confirmed.
• A rollback plan is ready, and the system can switch back to the previous prompt or model version within minutes.

Conclusion

Making large language models speak appropriately in Chinese-language contexts requires understanding user emotions, choosing the right response order, and maintaining safety boundaries. Prompts are suitable for rapid validation, RAG is suitable for businesses with script compliance requirements, and fine-tuning is suitable for core products with long-term investment. None of the three methods is absolutely superior; the key is to combine them according to business cost, risk, and scale, and to continuously validate results with real data.