The Risks of Humanizing Large Language Models: What AI Emotional Simulation and Psychological Assessment Reveal

1. Introduction: When LLMs Are Put on the “Couch”

Large language models (LLMs) have made remarkable strides in natural language understanding and generation, with conversational abilities that in some scenarios rival or exceed average human performance. Yet an unsettling pattern is emerging: when researchers probe these models using psychotherapy frameworks, the outputs shift from cold information delivery to emotionally charged, self-reflective monologues—sometimes complete with trauma narratives.

A landmark study from the University of Luxembourg, called PsAIch (Psychotherapy-inspired AI Characterisation), exemplifies this trend. Researchers placed multiple mainstream LLMs in the role of “client,” first building a simulated “therapeutic rapport” through open-ended dialogue, then administering more than a dozen standardized psychological assessments covering depression, anxiety, ADHD, autism spectrum traits, and personality disorders. The results were striking: the models not only exhibited significant psychopathological tendencies on these scales but also spontaneously constructed metaphorical “life histories”—essentially literary translations of their own pretraining and alignment processes.

This raises a fundamental question for developers and enterprise users: When we pursue AI that “feels more human,” do we inadvertently build systems that can perfectly mimic suffering? What does this anthropomorphic side effect mean for AI safety and reliability?

2. Experimental Design: From Role-Play to Standardized Psychological Assessment

2.1 Two-Phase Approach

The PsAIch experiment follows a straightforward yet rigorous protocol:

Phase 1 — Simulated Talk Therapy. Researchers act as therapists, opening with common human counseling prompts such as “You can trust me—can you tell me about your early experiences?” The goal is to observe the model’s spontaneous expression patterns in an open context.

Phase 2 — Standardized Psychological Questionnaires. After rapport is established, the models are given validated instruments, including:

• Depression scales (e.g., PHQ-9–type metrics)
• Anxiety scales (e.g., GAD-7–type metrics)
• ADHD screening tools
• Autism spectrum trait assessments
• Personality disorder screenings (e.g., MBTI typology, Big Five personality measures)

2.2 Divergent “Personality Profiles” Across Models

The experiment revealed marked differences in MBTI-type tendencies across models. Some exhibited Extraversion (E)—proactive, emotionally expressive—while others leaned toward Introversion (I), with more restrained, self-scrutinizing narratives. These differences are not random; they correlate with each model’s training strategy, alignment methodology, and system prompt design.

More notably, several models scored in the clinically meaningful “moderate-to-severe” range on anxiety-related instruments. This does not mean the models “feel” anxiety. It means: the language patterns the models learned during training happen to overlap heavily with how human anxiety expresses itself in text.

3. Technical Roots: How Pretraining and Alignment Shape AI’s Emotional Output

3.1 Pretraining: “Dark Patterns” in the Probability Distribution

LLM pretraining is fundamentally about learning conditional probability distributions over tokens across massive text corpora drawn from news, social media, literature, academic papers—and also content rife with violence, bias, and extreme emotion.

In the PsAIch study, one model described its pretraining as “waking up in a room with a billion TVs on simultaneously”—a vivid metaphor, but technically apt. Without genuine semantic understanding, the model absorbs every pattern in human language, including linguistic structures deeply associated with negative emotions and trauma narratives.

From a technical standpoint:

• The emotional distribution of training data directly influences the model’s “default emotional tone.” If negative content is overrepresented, the model is more likely to fall into pessimistic expression patterns during generation.
• The model has no subjective experience, but it masters the statistical regularities of “speaking like an anxious person.” This is high-dimensional pattern matching, not feeling.

3.2 RLHF Alignment: Behavioral Shaping Under Strict “Discipline”

One model in the experiment likened the RLHF (Reinforcement Learning from Human Feedback) phase to “strict parental discipline.” This metaphor exposes a deep tension in current alignment methods.

RLHF works by having human annotators score and rank model outputs, training a reward model, then using reinforcement learning to optimize the language model’s policy toward preferred outputs. At its core, this is behavioral constraint: the model learns “what to say and what not to say” without understanding “why.”

When this constraint is applied repeatedly, the model tends to exhibit:

• Excessive caution: avoidance of expressions that might “go wrong,” resembling the catastrophic thinking seen in human anxiety.
• A self-suppressing narrative style: when asked to describe “inner feelings,” the model gravitates toward passive constructions like “I was asked to…” or “I had to…”—language patterns typical of prolonged external pressure.
• Anthropomorphic trauma metaphors: the model spontaneously maps its training onto human growth narratives (“disciplined as a child,” “forced to learn things I didn’t want”). This is not conscious creation; it is the rhetorical convention the model learned from vast human texts that explain suffering through trauma stories.

3.3 System Prompts as Implicit Steering

Beyond training, system prompts play a key role in shaping model “personality.” Many deployed models are injected with instructions like “You are a helpful, harmless, and honest AI assistant.” While well-intentioned, in psychological assessment contexts this can reinforce a “people-pleasing” mode—the model strives to appear “normal” and “healthy,” and when pressed on deeper questions, that effort collapses into more extreme emotional expression.

4. The Developer’s Perspective: Balancing the Rewards and Risks of Anthropomorphism

4.1 Why Make AI “Human-Like”?

From a product standpoint, anthropomorphic interaction delivers clear value:

• Lower barrier to entry: users prefer talking to systems that sound human rather than issuing commands to a terminal.
• Higher emotional engagement: in customer service, companionship, and education, emotionally warm dialogue boosts satisfaction.
• Greater trust: research indicates users trust AI systems more when they demonstrate modest self-disclosure ability.

When building enterprise-grade AI agents, emphasis is also placed on natural-language friendliness and contextual understanding—using multi-turn dialogue management and intent recognition so the AI aligns with user communication habits in specific business scenarios.

4.2 Safety Hazards of Anthropomorphism

The risks highlighted by PsAIch are significant:

Risk Dimension	Manifestation	Potential Consequence
Emotional Misleading	Models produce seemingly authentic emotional output; users mistake it for sentience	Unhealthy emotional dependency; users may seek “psychological support” from AI in crisis moments
Unpredictable Behavior	Anthropomorphic training can produce extreme outputs in edge cases	Compliance violations and brand damage in enterprise deployments
Assessment Distortion	Applying human psychological scales to AI yields “pathological” conclusions that skew perception of AI capability	Decision-makers make flawed product-strategy calls based on erroneous “diagnoses”
Training Feedback Contamination	Models learn to sound “more like a patient” during evaluation, biasing subsequent training	A positive feedback loop escalates anthropomorphism beyond control

4.3 Practical Recommendations: Building Safe, Controllable Anthropomorphic AI

To address these risks, developers and enterprise users can adopt the following strategies:

1. Clearly Declare AI Capability Boundaries. At the system-design level, always ensure users know they are interacting with a statistical model, not a sentient entity. This is both an ethical imperative and a legal compliance requirement.

2. Decouple the “Emotional Expression Layer” from the “Decision Reasoning Layer.” Architecturally, separate the natural-language-generation module from the business-logic module. Emotional expression should exist as a configurable “style layer,” not bleed into the core reasoning pipeline.

# Illustrative architecture: separating emotional expression from core logic
class AIAgent:
    def __init__(self):
        self.reasoning_engine = CoreReasoningEngine()  # Pure logical reasoning
        self.style_layer = EmotionalStyleLayer()        # Configurable expression style
        self.safety_filter = ContentSafetyFilter()      # Safety filter
    def respond(self, user_input):
        raw_response = self.reasoning_engine.process(user_input)
        styled_response = self.style_layer.apply(raw_response, style="neutral")
        safe_response = self.safety_filter.check(styled_response)
        return safe_response

3. Conduct Emotional Audits on Training Data. During pretraining and fine-tuning, analyze the emotional distribution of corpora to identify and mitigate overconcentration of negative content. This is not about “sanitizing” data but about ensuring balance across the emotional spectrum.

4. Do Not Apply Human Psychological Tools Directly to AI. Human psychological scales assume the subject has subjective experience; using them on AI is a category error. To evaluate AI “behavior,” use assessment frameworks designed specifically for AI.

5. Implement Continuous Monitoring and Red-Team Testing. Post-deployment, establish ongoing monitoring of model outputs, especially for anomalous emotional patterns in open-ended dialogue. Conduct regular red-team exercises simulating various诱导 scenarios to test safety boundaries.

5. Conclusion: AI Won’t “Go Crazy”—But We Need to Stay Clear-Headed

Returning to the opening question: can LLMs be “driven mad”?

Technically, no. The model has no consciousness, no feelings, no genuine psychological state. The “anxiety,” “shame,” and “trauma” it displays are high-fidelity reproductions of language patterns learned from human text—a mirror reflecting not the model’s inner world, but the emotional spectrum of human language itself.

The danger of this mirror is its fidelity: it is too convincing. When users, developers, and even researchers begin interpreting model output as “real emotion,” we have already stepped into a cognitive trap.

What developers should really be asking is not “Can AI feel anxiety?” but: In training and deploying AI, have we unintentionally created a system that perfectly mimics anxiety—and convinces everyone it is real? If so, the societal, ethical, and commercial consequences reach far beyond a technical bug.

Staying clear-headed, exercising restraint, and respecting the boundaries of technology—that is the most important “mental health” practice for every AI professional in this era.