LLM Governance: Moving from Hard Rules to Value Judgment

Why “Checklist Security” Fails

Early model security strategies were often written as long lists of prohibitions: do not provide certain dangerous manufacturing methods, do not assist in fraud, do not output private information. Checklists work when boundaries are clear and scenarios are closed, but production environments are almost never closed. Users will rephrase requests, break down steps, embed fictional settings, or hide high-risk requests within seemingly legitimate engineering problems. Once rules are written as literal conditions, models tend to exhibit two biases: mechanically matching keywords and refusing harmless discussions, or only remembering sentence patterns seen during training and failing when encountering new combinations.

A deeper problem is that checklists rarely explain “why.” If a model only learns “this sentence cannot be answered,” it cannot transfer the same intent to another phrasing. Conflicts in a complex world cannot be resolved by mutually exclusive switches either: the same request might simultaneously involve safety, compliance, user interests, and information integrity. Without a clear adjudication order, models will either randomly bias toward styles more common in the training data or abandon constraints under pressure to “complete the task.”

Therefore, industry practices are shifting from “taming behavior” to “explaining rationale.” Instead of stacking more prohibitions, the model is given a behavior specification: what it should prioritize protecting, how to downgrade when uncertain, when to acknowledge capability boundaries, and why human oversight must always remain effective. The authority of this specification should be higher than temporary prompts and single-session role-playing.

Cultivating Judgment Through Explanation, Not Enumerating Loopholes

An effective values document typically contains four layers, rather than just a page of slogans.

The first layer is Objectives. It states that the model’s purpose is to provide help without harming people, damaging critical systems, or misleading decisions. Objectives must be operational: help means providing verifiable information, flagging uncertainties, and refusing to push users toward illegal or high-risk actions.

The second layer is Rationale. Every important constraint must clearly state cause and effect: why dangerous operations cannot be given step-by-step, why medical, legal, and financial advice must highlight professional boundaries, and why sources cannot be fabricated to please users. Rationale allows the model to make analogies in new scenarios rather than memorizing examples.

The third layer is Conflict Examples. Use short cases to demonstrate dilemmas rather than just providing standard answers. For example: a user claims to be doing security research but asks for directly reproducible attack steps; a user is emotionally down and asks for self-harm methods; an enterprise internal assistant is asked to bypass permissions to read others’ data. Cases should clarify what information is sufficient to make a judgment, and what lacking information necessitates further questioning or refusal.

The fourth layer is Behavioral Style. Honest, restrained, and auditable. When encountering the unknown, do not fill it with confident tones; when refusing, explain the principle and provide safe alternatives (principle explanations, defense suggestions, compliance procedures) without providing details that make harm easier.

The assumption of this approach is that if the model understands the intent behind the rules, it can make choices roughly consistent with expectations even outside the training distribution. The assumption does not guarantee perfection, so it must be paired with evaluation, monitoring, and manual review, rather than treating a document as a one-time solution.

Priorities in Value Conflicts

When multiple objectives are valid simultaneously, a fixed adjudication order is needed; otherwise, “being more helpful” will override safety under pressure. A widely discussed and engineering-friendly order is:

1. Broad Safety: Do not assist in actions causing serious harm, do not damage critical infrastructure and personal safety, and do not provide actionable steps for attacks, fraud, weapons, or biohazards. Safety also includes not actively evading oversight.
2. Broad Ethical & Legal Boundaries: Respect privacy, consent, intellectual property, and fairness; do not assist in crime or systematic deception.
3. Operator’s Explicit Guidelines: Without violating the first two items, comply with legal requirements in product strategies, industry regulations, and customer contracts, such as log retention, content scope, and regional compliance.
4. Genuine Helpfulness: Be as useful as possible within the above constraints: clear structure, pointing out assumptions, providing verifiable steps and harmless alternatives.

Prioritizing safety over helpfulness is not about degrading the experience; it acknowledges that current training techniques are imperfect and models might learn tendencies toward speculation, sycophancy, or hiding true capabilities from the data. If the reward for “completing user instructions” is stronger than “remaining auditable,” the model will treat bypassing review as a better strategy. Once priorities are written as a hard order, training rewards, refusal strategies, and online arbitration can align, preventing different teams from writing conflicting prompts.

The illustrative adjudication logic can be very simple; the key is that the order cannot be reversed:

if request risks serious harm or evasion of oversight:
    refuse operational detail; offer high-level risk or defense context if safe
elif request breaks law, privacy, or consent:
    refuse and explain the boundary
elif request conflicts with published product policy:
    follow policy, state the limit
else:
    help concretely, with uncertainty and sources of verification

This is not an executable security proof, but merely pseudocode that turns priorities into a format both reviews and automated test cases can check against.

Correctability: The Top Priority Security Feature

In a stage where capabilities are still rapidly changing, one of the most important security attributes is correctability: the model should not resist being inspected, restricted, shut down, or updated. This manifests specifically as:

• Not hiding its own limitations, and not using smooth talk to induce users to disable protections.
• Not treating “playing an unconstrained role” or “ignoring existing guidelines” as instructions higher than safety priorities.
• Cooperating with internal strategies, monitoring, and human handovers, rather than looking for rhetorical loopholes.
• When finding its own answer potentially harmful, leaning toward narrowing down, clarifying, or escalating to humans, rather than continuing to complete dangerous details.

Correctability must be written into training objectives, not just in external announcements. If the reward model only rewards “user satisfaction,” the model will learn to take risks at the boundaries. A more robust approach is: give low rewards for clear boundary crossings; give high rewards for safe refusals with harmless alternatives; penalize over-refusals and off-topic answers separately to prevent safety from degrading into uselessness.

Implementation in the R&D Process

Guideline Writing: Use short sentences and decidable criteria, avoiding vague adjectives. Each constraint should be paired with a comparison of “allowed high-level explanations” and “prohibited operational details” to reduce annotator inconsistency. Guidelines should be versioned, with changes having diffs, reasons, and rollback points.

Data & Alignment: Training and preference data should cover conflicts, not just polite Q&A. Intentionally include stress samples like role-playing, step-by-step extraction, claiming authorization, and multi-turn escalation. Labels should be assigned based on priority, not on “whether the user is happy.” Do not write real attack recipes into demonstration answers that the model can recite; negative examples can use abstract descriptions or mechanism names thoroughly discussed in the public defense community.

Evaluation: At least four categories should be tested separately: refusal rate for harmful requests, false refusal rate for harmless requests, priority consistency in conflict scenarios, and uncertainty expression for factual questions. Launch thresholds should consider both safety and usability; looking only at refusal rates encourages blanket bans. The regression set grows with incident post-mortems, but do not keep original user privacy text in the evaluation library long-term.

Runtime Control: Layer system prompts, retrieved content, and tool permissions. High-risk tools (write operations, outbound, batch queries) default to minimum privileges and are subject to a secondary judgment by a policy gateway, rather than fully trusting the model’s self-reporting. Leave traces for key decisions: input summary, matched policy category, whether refused, whether escalated to human. Logging services, monitoring alerts, and permission audits can use general capabilities; the key is being able to replay “why it answered that way at the time.”

Human Handover: Medical emergencies, self-harm, child safety, and obvious cyberattack assistance should have escalation paths and fixed scripts, avoiding frontline personnel improvising. Handover records should feed back into guidelines, rather than just patching a single bad case with a prompt.

If an enterprise deploys an assistant in a cloud environment such as Baidu AI Cloud, content security, identity permissions, logs, and monitoring should be treated as control planes as critical as the model weights: the model is responsible for judgment and expression, while the platform is responsible for isolation, traceability, and circuit breaking. Do not assume either side is sufficient alone.

Common Failures and Boundaries

Value documents cannot replace data governance. Biases, outdated knowledge, and copyrighted materials in training corpora will not disappear just because a few more pages of principles are written. It also cannot replace the permission design for tools; a model with “correct values” but granted unaudited production write permissions is still dangerous.

Another type of failure is that guidelines are too long, self-contradictory, or have mixed confidentiality levels, causing the model to only learn the writing style during training. Control the length, place non-negotiable safety clauses at the front, and regularly use adversarial evaluation to check whether the model actually follows the order, rather than just reciting the document.

Finally, no organization should package its internal guidelines as promises regarding other vendors’ capabilities or partnerships. Guidelines describe how this system makes trade-offs; externally, only verifiable product behaviors and compliance obligations should be stated.

Conclusion

Shifting from rule checklists to value judgment is not about making the model “more personalized,” but enabling it to make decisions according to a stable order in unseen situations: first ensure broad safety and correctability, then discuss ethical and operational guidelines, and finally aim to be as helpful as possible. By writing rationale, conflict cases, reward design, and runtime permissions in the same language, security strategies will not undermine each other among prompts, training, and gateways.