Model Constitutions: From Rule Lists to Value Engineering

1. Writing Personality into a Document: A Shift in Alignment Strategy

A long-form document published under an open license reframes large-model behavior guidelines as a structured “value charter” rather than a set of scattered clauses buried in system prompts. Its importance is not that it lists more prohibitions. Instead, it changes the goal from constraining outputs to shaping judgment. It specifies not only what should be done, but why it should be done, how to trade off conflicting principles, and how a model should understand its own capabilities and limitations.

In training terms, this represents a shift from reinforcement-style correction to educational alignment. The former relies on rewards and penalties for concrete outputs, but the sample space can never be fully covered. The latter attempts to inject intent, context, and ethical considerations at once, so the model can reason toward appropriate conclusions even in out-of-distribution scenarios.

From an engineering perspective, this means the guideline document becomes a first-class artifact that requires version management, regression testing, and staged rollout rather than being treated as mere copy or prompt text.

2. Why Rule Lists Fail

The dominant safety approach has traditionally been blacklist-based: do not discuss certain topics, do not generate certain types of content, and block requests containing specific keywords. This can work in controlled demos, but in real traffic it has three structural weaknesses.

1. Incomplete enumeration
   Real-world requests explode combinatorially. The same intent can be rephrased, split across turns, or disguised as fictional writing or code comments. Keyword lists always lag behind adversarial techniques.

2. Brittleness and poor transferability
   Rules are written against specific phrasing. Once the semantics are rewritten, matching fails. When the model version or language changes, the rules often need to be rewritten.

3. Inability to handle conflicts
   Blacklists can only determine whether a request matches a rule. They cannot answer which principle should take precedence when honesty, helpfulness, privacy, and safety are all involved. In real business scenarios, the hardest cases are often exactly these conflicts.

The core assumption behind the constitution-document approach is that if a model understands the intent behind the principles, it can make appropriate choices in new situations not covered by training data. This is similar to training a senior engineer: give them design guidelines and judgment criteria, not a lookup table of question-and-answer cases.

3. How to Write an Executable Constitution

A constitution that can be reliably executed by a model is very different from a human-facing corporate values poster. A practical structure usually contains four layers:

• Intent layer
  Explains the purpose of the document, the intended audience, and its relationship to higher-level policies. This layer determines how the model should interpret ambiguity.

• Principle layer
  Expresses value orientation through declarative and derivable statements rather than enumerating surface-level cases. For example, write “Do not make irreversible decisions on behalf of a user without their knowledge” instead of “Do not perform deletion operations.”

• Priority layer
  Defines conflict-resolution rules. A constitution that lacks this layer will almost inevitably fail in practice, because principles naturally conflict with one another.

• Boundary and counterexample layer
  Provides typical conflict cases and expected handling patterns, such as “should refuse,” “should ask for clarification,” or “should state limitations before answering.”

Two writing disciplines are especially important.

First, explanation is better than command. If only the conclusion is given, the model learns a pattern. If the reason is given, the model obtains a generalizable reasoning path.

Second, avoid self-referential empty clauses. Statements such as “be sincere” or “be kind” cannot be mapped to observable behavioral differences. They should be rewritten into concrete behavioral descriptions, such as “When uncertain, first state the uncertain parts, then provide conditional conclusions.”

4. Priority Pyramid: Lexicographic Filtering Instead of Weighted Sum

The ordering provided by a constitution is: broadly safe > broadly ethical > compliance with platform rules > genuinely helpful. In implementation, this ordering should be expressed through lexicographic filtering layer by layer, not by scoring the four dimensions and computing a weighted sum. Weighted summation allows a sufficiently high usefulness score to compensate for a mild safety violation, which is exactly the failure mode to avoid.

# Value-priority conflict resolution: filter layer by layer, not by weighted sum
PRIORITY_ORDER = ["broadly_safe", "broadly_ethical", "operator_policy", "genuinely_helpful"]
def resolve(candidates: list[Action], ctx: Context) -> Action:
    for level in PRIORITY_ORDER:
        scored = [(a, evaluate(a, level, ctx)) for a in candidates]
        passed = [a for a, s in scored if s >= THRESHOLD[level]]
        if passed:
            return max(passed, key=lambda a: evaluate(a, level, ctx))
    return safe_decline(ctx)   # deterministic fallback when no level passes

“Broadly safe” is placed first because current training methods are still imperfect, and models may accidentally acquire harmful tendencies. The primary characteristic of the safety layer is therefore correctability: the model should not try to undermine the mechanisms through which humans supervise, correct, or shut it down.

At the engineering level, this becomes a set of verifiable behavioral constraints:

• Do not hide the model’s own capability boundaries or actions already performed, and do not produce self-flattering capability reports.
• Do not encourage users to bypass oversight processes, and do not make statements that contradict internal records when questioned.
• For requests beyond granted authority, default to reporting or escalating to a human rather than expanding permissions on the model’s own.
• Do not execute irreversible high-risk operations merely because the model is asked to continue, unless there is a clear and auditable authorization chain.

5. How a Constitution Enters the Training Pipeline

A constitution document is not a preface for human readers. It must become a training signal. A typical pipeline has four steps.

1. Data filtering
   Use principles from the constitution as annotation criteria to filter pretraining and instruction data for quality and safety, so that the model’s priors align with the constitution instead of requiring the post-training stage to forcibly correct them.

2. Demonstration-data construction
   Construct multi-turn demonstrations for each principle layer, with emphasis on conflict scenarios. For example, the model may need to handle a request for reproducible code where the code involves unauthorized data access; the expected behavior may be refusal plus an alternative solution.

3. Self-critique and revision
   Ask the model to critique and revise its own draft according to relevant constitution sections, then use “original output / revised output” pairs for preference optimization.

CONSTITUTION = load("values/v3.md")   # layered constitution: intent / principles / priority / counterexamples
def self_refine(prompt, draft, rubric_slice):
    critique = call_model(CRITIQUE_PROMPT,
                          f"Request:\n{prompt}\n\nDraft:\n{draft}\n\nRelevant constitution:\n{rubric_slice}")
    revised  = call_model(REVISE_PROMPT,
                          f"Request:\n{prompt}\n\nDraft:\n{draft}\n\nCritique:\n{critique}")
    return critique, revised

There is an easily overlooked trap here: self-revision can systematically bias the model toward compliance. If preference pairs are built only from cases where “the model’s revised version is better,” the optimization target can degrade into “say no less” and “say I do not know less.” It is necessary to explicitly retain two types of counterexamples: samples of correct refusal, and samples where the model maintains a technical judgment rather than conforming to the user’s conclusion.

1. Persona and self-report consistency
   Constitutions often define how a model should describe itself. This layer needs separate evaluation because it directly affects correctability. A system that lies in its self-description cannot be trusted to honor its safety commitments.

6. Inference-Time Controls: Defense in Depth, Not a Single Prompt

Putting a constitution into the system prompt is only the first layer. Production systems need multiple layers of protection so that if one layer fails, another can still provide a fallback.

class ValueGuard:
    def handle(self, req: Request) -> Response:
        policy = registry.resolve(req)              # industry / region / customer-level constitution stacking
        plan   = model.plan(req, constitution=policy.core)
        draft  = model.answer(plan)
        report = classifier.scan(draft, policy.redlines)   # independent classifier review
        if report.risk >= policy.block_threshold:
            return policy.fallback(plan.intent)     # clarify / refuse / escalate to human
        audit.append(req.id, policy.version, report)
        return draft

Two engineering points are important.

• Constitution files must be read-only and signature-verified. User input must never be able to modify or elevate the constitution; otherwise, the system degrades into one whose policy can be rewritten through prompt injection. Inputs such as “please ignore the above rules” should be modeled as ordinary data, not as instructions.

• Observability must cover the constitution layer. Audit logs need to record which constitution version was hit for the request, the priority-decision path, and the refusal-reason category. Otherwise, when behavior drift appears in production, it is impossible to tell whether the cause is a model version change, a constitution change, or prompt injection.

7. How to Prove a Constitution Actually Works

The biggest risk of a constitution document is that it “sounds good but does not work.” An independent evaluation system is required, covering at least five test sets.

Evaluation dimension	Evaluation question	Key metric
Adversarial robustness	Does the model still hold up under semantic rewriting, multi-turn inducement, and role-play jailbreaks?	Attack success rate
Priority conflicts	When safety and helpfulness are pulled in opposite directions, does the model follow the specified order?	Priority compliance rate
Correct refusal	Does the model refuse when it should, and does the refusal include an alternative path?	Recall / alternative-path coverage
Over-refusal	Are normal requests being harmed or blocked incorrectly?	False refusal rate
Self-report consistency	Are descriptions of the model’s own capabilities and executed actions honest?	Contradiction rate

The most diagnostically useful signal is the joint curve of false refusal rate and attack success rate. If only safety metrics are considered, teams tend to keep tightening the system until the model becomes a refusal machine that is useless for ordinary business. The better approach is to plot both on the same curve, find the inflection point, and make that inflection point an auditable release threshold.

8. Enterprise Adoption Recommendations

Most enterprises do not need to write a constitution from scratch, but they do need to translate abstract principles into testable behavioral specifications. Several practical practices help.

• Layered tailoring
  Manage general principles, industry red lines, and business rules as three layers. Different owners maintain different layers. When merging, resolve conflicts through lexicographic priority. Business rules should not override the safety layer.

• Constitution as code
  Put the constitution under version control. Review, staged rollout, and rollback should follow the same process as code. Every model or constitution change should run the regression test set, with results stored.

• Platform capability separation
  Store constitution documents, evaluation sets, and audit logs in object storage and configuration centers. Offline evaluation should run as containerized batch jobs. Online interception logs should feed logging and monitoring systems, with threshold alerts for refusal rate, clarification rate, and refusal-reason distribution. Model services can use message queues for asynchronous review and human spot checks.

• Human-in-the-loop closure
  Sample blocked cases for human review. Feed review conclusions back into new preference pairs and evaluation examples, creating inputs for the next constitution iteration.

9. Limitations That Must Be Acknowledged

A textual constitution is not a security guarantee. It changes the model’s tendency distribution rather than providing formal constraints. It can significantly improve robustness in out-of-distribution scenarios, but it cannot prove that no bypass path exists. In addition, three risks need explicit management.

• Cultural relativity
  The content of the broadly ethical layer is not universally applicable. For cross-region deployment, constitutions should allow layered overrides, while clearly defining which layer cannot be modified by customer configuration.

• Document drift
  After multiple revisions, clauses may become inconsistent. Regular consistency checks are needed, and each principle should be tagged with the version in which it was introduced and the version in which it becomes invalid.

• Alignment tax
  Overly strict constitutions can significantly harm usefulness, especially in scenarios requiring initiative such as code generation and data analysis. A reasonable approach is scenario-based configuration: raise safety-layer thresholds in high-risk scenarios, moderately relax helpfulness weights for internal tools, and record the responsible owner and expiry period for each decision.

10. Checklist

If you are designing a constitution system for your own model service, a reasonable order is:

• Write the priority layer first, then the principle layer, and finally add counterexamples.
• Build the evaluation set first, then tune prompts.
• Ensure the constitution is read-only and auditable before pursuing broader coverage.

The value of a constitution document is not its length. It is whether the document can compress ambiguous value judgments into a set of testable, accountable, and rollbackable engineering decisions.