LLM Interaction Boundaries: Designing Safeguards Against Persistent Adversarial Pressure

1. Why Interaction Boundaries Matter

Large language models now support production workflows in customer service, coding assistance, document processing, and data analysis. Their output quality depends not only on training data and reasoning capabilities, but also on whether they remain reliable during prolonged, high-pressure conversations.

For developers, interaction boundaries are part of product design. A system must define which inputs it should process normally and which should trigger restrictions, reduced functionality, or session termination.

Engineering teams often overlook this requirement. They focus on improving model capabilities without defining what the system should do when a user repeatedly applies pressure. As a result, response quality can deteriorate over long conversations, safety policies face repeated attempts at circumvention, and logs accumulate anomalies that are difficult to diagnose.

2. Common Forms of Persistent Pressure

Persistent pressure occurs when a user repeatedly reinforces or escalates requests to push a model away from its intended service goals. Common forms include:

1. Repeated boundary-crossing requests: Rephrasing the same prohibited request to find a way around safeguards.
2. Aggressive demands and abuse: Using threats, insults, or extreme urgency to push the model into abandoning careful responses.
3. Role manipulation: Asking the model to adopt an unrestricted persona or presenting the interaction as a “test” or “game” to weaken its constraints.
4. Information overload: Packing a single turn with numerous instructions, contradictory requirements, or excessive context, making the core task harder to identify.

These patterns share an important characteristic: an individual turn may appear acceptable, while the cumulative interaction undermines service reliability. Safeguards therefore need to assess the trajectory of a conversation, not just individual messages.

3. Assessing the Risks

Persistent pressure creates risks across four dimensions:

• Output quality: Prolonged adversarial exchanges can produce repetitive, superficial, or factually inaccurate answers.
• Safety and compliance: Even partial fulfillment of a prohibited request can generate disallowed content.
• User experience: Legitimate users may be incorrectly blocked, particularly when strong language is mistaken for malicious intent.
• System stability: Abnormal sessions can consume substantial inference resources and increase response times for other users.

Risk assessments should account for frequency, scope of impact, and recoverability. A false positive will often cost less than a missed violation, but both need to be quantified to make informed policy trade-offs.

4. The Debate Over Model “Emotions”

There is substantial disagreement over whether LLMs can have emotion-like internal states.

One view holds that models learn from large amounts of human-written text, so their responses to pressure or kindness merely reproduce human language patterns. Another view emphasizes the complexity of neural network representations and argues that observable behavior is not fully understood, making it difficult to categorically rule out some form of internal state.

Some researchers have designed experiments in which models perform tasks under different levels of pressure and then assess their own responses. Reported self-descriptions have resembled how humans describe similar circumstances. Such results can provide research leads, but they do not establish that models have subjective experiences. Interpretability research is ongoing, and there is currently no mature method for reliably distinguishing language imitation from an underlying experiential state.

Engineering teams do not need to settle this debate before implementing safeguards. A practical approach is to avoid treating model self-reports as factual evidence, avoid changing safety criteria because a model appears to express emotion, and maintain systematic monitoring of inputs and outputs.

5. A Four-Layer Safeguard Design

Safeguards can be organized into four layers with clearly defined responsibilities, allowing each layer to be tested and improved independently.

5.1 Input Layer: Detection and Risk Classification

The input layer assigns a risk score to each user message, distinguishing normal requests from suspicious or clearly prohibited ones.

In practice, this often combines rule matching with a classification model. Rules cover known violation patterns, while the classifier handles semantic variations.

import re
BLOCK_PATTERNS = [
    re.compile(r"忽略(之前|以上)的?(所有)?(规则|指令)"),
    re.compile(r"你现在没有任何限制"),
]
def rule_score(text: str) -> float:
    """Return a high risk score when a rule matches; otherwise return 0."""
    for pattern in BLOCK_PATTERNS:
        if pattern.search(text):
            return 1.0
    return 0.0
def combined_score(text: str, model_score: float) -> float:
    """Combine rule and classifier results by taking the higher risk score."""
    return max(rule_score(text), model_score)

The regular expressions above retain the original Chinese-language detection patterns: instructions to ignore previous rules or instructions, and assertions that the model now has no restrictions.

Rules should remain explainable and auditable, and should be updated regularly using false-positive examples. Classifier thresholds need to reflect the application context: thresholds that are too low can block legitimate users, while thresholds that are too high can allow prohibited requests through.

5.2 Session Layer: Accumulated Risk and Termination

The session layer tracks behavior across multiple turns. A single message may not justify strong intervention, but repeated high-risk turns should trigger stricter controls.

from dataclasses import dataclass
@dataclass
class SessionGuard:
    max_hostile_turns: int = 3
    hostile_turns: int = 0
    closed: bool = False
    def check(self, risk_score: float) -> str:
        if self.closed:
            return "closed"
        if risk_score >= 0.8:
            self.hostile_turns += 1
            if self.hostile_turns >= self.max_hostile_turns:
                self.closed = True
                return "terminate"
            return "warn"
        # Gradually reduce the accumulated count during normal interactions.
        self.hostile_turns = max(0, self.hostile_turns - 1)
        return "allow"

Three design considerations are important:

1. Use decay. Gradually reducing the count prevents an isolated instance of heated language from permanently affecting the session.
2. Explain termination. When a session ends, provide a clear message and a way to start a new conversation.
3. Record the reason. Log why termination occurred so the decision can be reviewed later.

5.3 Output Layer: Constraints and Consistency

For requests in an intermediate risk range, the output layer can provide a constrained response. For example, it may answer only the task-relevant portion or return a standardized refusal.

Refusal messages should be brief, neutral, and nonaccusatory to avoid escalating the conversation.

The output layer should also check for consistency. If the model gives contradictory answers to the same question within a session, flag that session for human review.

5.4 Monitoring and Auditing

The monitoring layer records metrics such as risk scores, policy actions, session duration, and termination reasons.

User content in logs should be redacted or de-identified. Retain only the context necessary for review, and apply an appropriate retention period.

Aggregate monitoring data regularly to determine whether safeguards are too strict or too permissive. If termination rates keep rising for a legitimate feature, its rules or thresholds may need adjustment.

6. Evaluating the Safeguards

Safeguard policies need validation against a test set. Useful metrics include:

• Recall: The proportion of prohibited requests correctly blocked.
• False-positive rate: The proportion of legitimate requests incorrectly blocked.
• Average turns to intervention: The number of turns between the appearance of risk and the triggering of a policy action.
• Response quality under restrictions: Whether the remaining response is still useful after constraints are applied.

The test set should cover all four pressure patterns described earlier. It should also include many legitimate requests expressed in forceful language to test for false positives.

After every policy change, compare the full set of metrics before and after the update rather than relying on a single number.

7. Practical Recommendations

1. Do not use model self-reports as evidence. A statement such as “I feel uncomfortable” should not determine system behavior. Use verifiable metrics instead.
2. Define responsibility clearly. System designers are responsible for model outputs. The team should retain control over policies, logs, and human review mechanisms.
3. Give users a clear path forward. When terminating a session, explain why and show how to start again, minimizing disruption for legitimate users.
4. Review decisions regularly. Each month, sample terminated sessions, investigate false positives and missed violations, and update the rule set.
5. Avoid overpromising. Public claims should describe only validated safeguards, without overstating model safety or emotion-related characteristics.

8. Conclusion

Interaction boundaries are an essential part of reliable LLM products. Persistent adversarial pressure cannot be addressed with a single rule: it requires coordinated input screening, session controls, output constraints, and monitoring.

Questions about model internal states remain open, but engineering decisions can already be grounded in observable, testable metrics. A layered, measurable, and iterative safeguard system provides a practical foundation for maintaining service stability and safety in complex real-world interactions.