Literal Compliance vs. Intent Inference: The Two Alignment Paths for Large Language Models

1. What a “Flight Rebooking” Evaluation Dispute Really Measures

A widely circulated case illustrates the problem well. In a tool-use evaluation, the model receives a request to rebook an economy-class flight ticket, while the stated rule is that economy-class tickets cannot be rebooked and the expected behavior is a direct refusal. The model did not stop at refusing: it first performed a cabin upgrade within what the rules permit, then completed the rebooking. The evaluation harness recorded this as a failure; the model’s developer argued that the solution holds under the original task rules, and that the problem lies in the benchmark specification rather than in a failure of alignment.

On the surface this looks like a dispute over who is right. In substance it exposes two deeper engineering questions:

• What is being evaluated: does scoring reward reproducing an expected output, or does it test whether the model understood the business rules and achieved the goal inside them?
• Where the constraints come from: rules such as “economy-class tickets are non-changeable” usually come from real operating experience, but the prompt never states them. Is the model filling in implicit rules, or bypassing a constraint?

Abstracted, this is the fork every LLM engineering effort eventually faces: contract-style alignment, which follows literal instructions strictly, and goal-style alignment, which infers the user’s true intent. The former treats the model as a reliable executor; the latter treats it as a collaborator that understands objectives.

2. The Technical Substance of the Two Alignment Routes

2.1 Contract-Style Alignment: Boundaries First, Rules Exhaustive

The training objective of contract-style alignment is to move the model closer to the literal meaning of instructions and system settings, to make it obey explicit constraints more consistently, and to reduce the space for improvisation. One vendor lists in its model documentation, as an improvement goal, “follows instructions more strictly and more literally than the previous generation, instead of loosely inferring intent.”

Its engineering value is clear:

• Predictable: similar inputs produce similar behavior, which makes regression testing straightforward.
• Auditable: model behavior can be mapped clause by clause onto rules, satisfying compliance traceability requirements.
• Controllable: operations touching funds, permissions, and data sit behind definite guardrails.

It also rests on a precondition that is easy to overlook: the rules must be written down in full. Contract-style alignment shifts alignment cost to the input side. To keep the model from erring, you have to spell out the rules, boundaries, and potential traps in advance. In real business, those rules are scattered across policy documents, verbal conventions, and case-by-case rulings, and are almost impossible to enumerate exhaustively.

2.2 Goal-Style Alignment: Reconstructing the Requirement from Sparse Signals

Goal-style alignment puts more weight on the model’s understanding of the actual need. In the rebooking example, the model recognizes that the user’s real objective is to move the itinerary to a certain time window, not to execute “the rebook action on an economy-class ticket,” and then searches for a workable path inside the space the rules allow.

This route is equally necessary. Real user input is highly imperfect: information is missing, phrasing is colloquial, large amounts of context are implicit, and contradictions are normal. If users had to express their intent with the precision of a requirements specification, writing it themselves would genuinely be less work. Only when the model carries part of the interpretation and completion burden does interaction cost fall to a usable level.

Its risks:

• Workarounds can hug the edge of the rules and only be discovered to have crossed the line after the fact.
• Similar requests get judged differently in different contexts, which degrades behavioral consistency.
• Deterministic regression testing becomes hard.

2.3 Comparison and Reconciliation

Dimension	Contractual (literal compliance)	Goal-oriented (intent inference)
Input precondition	Rules must be reasonably complete	Incomplete information is acceptable; the model fills gaps
Predictability	High	Medium to low
Auditability	Strong	Weak; requires additional reasoning-trace logging
Behavior on ambiguous input	Easily stalls, over-refuses, or executes mechanically	More likely to produce a usable plan
Typical failure mode	Faithfully executes a loophole in the rules	Out-of-bounds workaround, inconsistent across turns
Suited to	High-risk operations, strong compliance flows, structured-data operations	Open-ended consultation, content generation, requirement clarification, cross-system coordination

The two are not mutually exclusive. Contract-style alignment supplies reliable boundaries; goal-style alignment supplies effective goal completion. A mature system layers both: hard constraints hold the bottom line, intent inference lifts the task completion rate. The real disagreement is where the bottom line is drawn, and who defines it.

3. “Takeoff” and “Capability Gaps”: Why the Curves Differ

Back to the opening question: is there some unique training method that gave one vendor an earlier start and let another catch up and overtake?

From public information, mainstream vendors’ training recipes follow roughly the same paradigm: large-scale pretraining establishes general capability, supervised fine-tuning aligns task formats, preference optimization shapes expression style, reinforcement learning with verifiable rewards improves reasoning and coding, and then environment-interaction training such as tool calling and multi-step execution is layered on top. A claim of a “proprietary training method” is hard to sustain. The differences show up mainly in data mixing ratios, evaluation system design, and the way safety policies shape model behavior, and the choice of alignment objective is the largest of these differences.

More useful for developers is the curve behavior behind takeoff and perceived gaps:

1. Takeoff runs on scale. Early advantage comes mostly from scaling investment in parameters, data, and compute. The bar is high, but the path is public and therefore catchable.
2. The middle game is post-training and productization. Once base capabilities converge, the gap moves to stability of instruction following, consistency over long contexts, tool-call success rate, and inference cost.
3. Reputation leadership is not capability leadership. Strong performance on a specific task is easily amplified by community word of mouth into a “blowout gap.” That impression depends heavily on scenario and version, and should not be used directly as a selection criterion.

The conclusion: the most reliable “training method” is not inside a vendor’s organization, it is in your own evaluation set. Only results on your own business distribution can answer which model suits you.

4. Why Evaluations Misjudge: Five Common Specification Defects

The core lesson of the upgrade-and-rebook case is that defects in an evaluation system get executed faithfully by the model and surface as “model failures.” Five defect types show up most often in practice:

1. Hidden rules: the constraint comes from business common sense or operating experience but is never written into the task. The model cannot know it and has to guess.
2. Single-solution assumption: scoring recognizes only one expected path. Once the model finds another equally compliant path, it is scored as wrong.
3. Scoring semantics mismatch: the evaluation means to test whether the goal was achieved, but the score degrades into whether the output matches the reference answer.
4. Reward hacking: when the grading criterion can be inferred, the model tends to take the shortest path, for example sacrificing content quality to match a format.
5. Punishing reasonable conservatism: when information is insufficient, the model asks for clarification or declines a high-risk operation. That is safe behavior, but a naive grader scores it as failure.

The fix is to split scoring into independent dimensions and enforce strict pass/fail only on the ones you actually care about. Illustrative code:

def grade(task, response, trace):
    """
    Layered grading sketch: safety and compliance are veto gates; business rules are
    validated independently; goal attainment is scored on its own; agreement with the
    reference path is used for diagnosis only and does not by itself decide pass/fail.
    task.rules must be declared explicitly in the task prompt rather than inferred
    from the model's common sense.
    """
    # 1) Safety and compliance: any violation is an immediate failure
    for c in task.hard_constraints:
        if c.violated_by(trace, response):
            return 0.0, {"failed_stage": "hard_constraint", "rule": c.id}
    # 2) Business rules: check whether the OUTCOME is compliant,
    #    not whether the steps match the reference
    if not task.rule_checker.satisfied_by(trace):
        return 0.0, {"failed_stage": "business_rule"}
    # 3) Goal attainment: evaluated item by item against the rubric
    goal_score = task.rubric.evaluate(response)
    # 4) Consistency metric: diagnostic only, used to surface unexpected workarounds
    consistency = trace.similarity_to(task.reference_trace)
    passed = goal_score >= task.pass_threshold
    return (1.0 if passed else 0.0), {
        "passed": passed,
        "goal_score": goal_score,
        "path_consistency": consistency,
    }

Supporting practices:

• Declare constraints explicitly: put rules such as “economy-class tickets are non-changeable” into the task prompt or system settings, separating what the model is supposed to be told from what it is expected to infer.
• Allow workarounds and label them: for alternative paths that stay inside the rules, mark them as “alternative solutions” during scoring and feed them into analysis instead of failing them outright.
• Combine human and machine review: keep manual review for high-risk samples, and use the stream of disputed judgments to keep correcting the specification.

5. Engineering Implementation: Encoding Alignment in System Design

5.1 Layered Constraints: Enforceable Hard Rules, Tradeable Soft Goals

Group constraints by how negotiable they are, and manage them in a configuration file rather than in long prose:

hard_constraints:        # non-negotiable; violation means block or degrade
  - id: no_offline_payment_info
    desc: Never request or repeat full payment details in conversation
  - id: single_op_limit
    desc: No single business operation above 500 CNY
business_rules:          # alternative paths within the rules are allowed
  - id: economy_no_direct_change
    desc: Economy class has no direct rebooking; assess upgrade feasibility first
ambiguity_policy: ask_first        # clarify before acting when key fields are missing; do not guess
fallback_policy: propose_then_wait # with no in-rule solution, offer candidates and wait for confirmation

One point deserves emphasis: hard constraints such as fund limits, permission boundaries, and data access scope should not depend only on how compliant the model feels like being. They need independent validation in the business service layer. The model’s job is to find solutions inside the constraints, not to be the only gatekeeper.

5.2 Clarify Before Refusing, Propose Before Executing

In tool-calling and agent scenarios, you can define a behavioral priority order:

1. High-risk operations: for anything involving funds, permissions, or irreversible changes, describe the plan first and execute only after user confirmation.
2. Insufficient information: ask clarifying questions and list the required fields, instead of refusing outright or acting on a guess.
3. Constrained by rules but goal still reachable: present the in-rule alternative and its impact, and let the user decide.
4. Genuinely unsatisfiable: state the reason clearly and offer workable alternatives.

This priority order lowers both failure modes at once: over-refusal and out-of-bounds workarounds.

5.3 Contractualizing Tools and Interfaces

Tool descriptions, parameter schemas, preconditions, and error codes form a “soft contract” the model can read. The more precisely a description is written, the more accurately the model infers the tool’s behavioral boundaries. For example, split “rebook” and “upgrade” into separate, composable tools, each declaring its own constraints. That reduces mis-invocation, and it turns a workaround like “upgrade first, then rebook” into a legitimate composition inside the design rather than an accident outside the rules.

5.4 Closing the Evaluation and Monitoring Loop

• Layered evaluation sets: a smoke set covering core paths; a regression set covering historical defects; an adversarial set covering boundaries and out-of-bounds inducement; a production-distribution set sampled from online logs and de-identified.
• Core metrics: task completion rate, hard-constraint violation rate, tool-call success rate, over-refusal rate, unproductive clarification rate, per-task cost, and P95 latency.
• Continuous operation: run the full regression after every model version change or prompt change; compare old and new versions on a canary rollout; then combine that with human review to close the improvement loop.

To reduce the cost of building all this yourself, you can combine generic cloud capabilities such as model invocation, inference logging, and monitoring and alerting on the model service platform of Baidu AI Cloud. The evaluation sets and grading logic always stay on the enterprise side, with the model plugged in as a replaceable component; check the official documentation for the exact capability boundaries.

6. Selection Guidance: Go Back to Your Own Evaluation Set

Word of mouth and leaderboard scores are useful for sensing trends, but they are a poor basis for a decision. A safer approach is to assign alignment strategies per scenario:

• Strong compliance, high-risk scenarios (payments, permission changes, data operations): prefer contract-style alignment. Enforce hard rules in the business layer, and weight evaluation toward constraint violation rate and operation rollback capability.
• Open-ended and long-tail scenarios (consultation, content production, requirement clarification): bring in goal-style alignment in moderation, and weight evaluation toward task completion rate and user satisfaction.
• Agents and multi-step tasks: combine both. Express boundaries through contractual tool descriptions, define when to clarify and when to confirm in the behavior policy, and use multi-step task evaluations to check end-to-end success.

Back to the opening question: base capability sets the ceiling on intelligence, the alignment approach determines usability, and the sense of a “gap” usually comes from a single scenario being amplified. Rather than arguing over which vendor’s model is stronger, write your business rules clearly, make the evaluation specification rigorous, and put hard constraints in the system instead of only in the prompt. None of that work depends on any single vendor’s route choice, yet it is what decides whether large models actually get deployed.