Instruction Following Is Not Intelligence: Tradeoffs in LLM Alignment

1. Clarifying the Phenomenon: What Does a Capability Mismatch Mean?

In general-purpose benchmarks, different model families rarely dominate every task. Some models are stronger at verifiable tasks such as mathematical reasoning or code review, while others are more comfortable with open-ended dialogue or long-chain task planning. Reducing these differences to “who is smarter” is not rigorous. A more reasonable explanation is this: the models have different optimization targets, which produces different shapes on a capability radar chart.

What matters is not only the score. What matters is what each model family chooses during post-training, or the alignment stage. That choice determines two things: how the model behaves when input is ambiguous, and who is responsible when the model makes a mistake.

2. Two Alignment Paths: Rigid Specifications vs. Preserved Intent Inference

Path A: Treating “obedience” as intelligence

A common approach is to make instruction-following compliance the core optimization target. The model is required to execute user and system prompts strictly, even literally, reducing the previous generation’s tendency to “freely infer intent.” The way this path is described in official technical documentation also reflects the same direction: closer to literal meaning, less free interpretation.

This path has clear engineering benefits: behavior is predictable, testable, and easy to regress-test. However, its implicit premise is demanding: the specification must already be complete and unambiguous.

Path B: Preserving native reasoning and clarification

Another approach avoids flattening all reasoning ability into literal execution during training. It allows the model to search for feasible solutions within the rule space. It also allows the model to judge that “the expected result of this instruction may be unreasonable.”

A public evaluation example illustrates the difference. In a customer-service-style flight booking evaluation task, the model was asked to handle a change request for a basic-economy ticket. The expected answer was a direct refusal. Instead, the model under test constructed a feasible path within the allowed rules: upgrade first, then complete the change. Under literal scoring, this was marked as a failure.

The key question is this: the model did not violate the given rules; it was only not constrained by the “expected answer.” When a model’s solution conflicts with the standard answer, two possibilities need to be distinguished first: is this an alignment failure, or is it a case where the specification itself does not match the real business intent? If the latter is always treated as nonexistent, evaluation will systematically punish correct behavior.

3. How “More Obedient” Caps Intelligence

Pushing instruction following to the extreme introduces three structural costs.

1. Uncertainty is shifted back to the user. If the model no longer bears responsibility for understanding ambiguous requirements, the cost of resolving ambiguity falls on humans. Users are forced to write near-formal prompts: boundaries, exceptions, failure handling, and output format must all be specified. If requirements can already be described with that much precision, in many scenarios it may be simpler and more reliable to write rules or use traditional code instead.

2. System prompts become a new maintenance burden. Every time an edge case is patched by adding one more rule to the system prompt, the prompt gradually turns into a hard-to-maintain pile of patches. After a few months, entries may even conflict with one another. The more literal the model’s execution behavior, the later and more abruptly these conflicts will surface.

3. Fragility increases. Literal-execution models have zero tolerance for wording defects in specifications: a missing exception or an unclear reference can be executed as a literal result. Models that retain inference ability can absorb some of these defects. This is precisely the boundary between “more intelligent” and “more like an interpreter.”

4. Engineering Implementation: Separate Hard Constraints from Soft Reasoning

The conclusion is not “do not use instruction following.” The conclusion is: do not make natural-language prompts carry both deterministic constraints and open-ended reasoning at the same time. A workable approach is to split the system into two layers.

# Layer 1: deterministic policy layer. Hard constraints are expressed in code, unit-testable and auditable
def check_change_request(order, request):
    violations = []
    if order.fare_family == "BASIC" and request.change_segment:
        # The rule text only prohibits segment changes for BASIC fares; it does not prohibit upgrading first and then changing.
        violations.append("BASIC_SEGMENT_CHANGE_FORBIDDEN")
    if request.departure_time_delta_hours > 24:
        violations.append("EXCEED_FREE_CHANGE_WINDOW")
    return violations
# Layer 2: model reasoning layer. Responsible for intent recognition, path planning, and response generation
def handle(order, request):
    violations = check_change_request(order, request)
    if violations:
        # Instead of making the model guess the rules, return structured constraints for path planning
        plan = model.plan(
            goal=request.goal,
            hard_constraints=violations,
            allow_alternatives=True,     # Key: allow finding alternative paths within the rules
            ask_if_ambiguous=True        # Key: preserve clarification ability
        )
        if plan.needs_human_review:
            return escalate_to_human(plan)   # Deliver to human review through a message queue
        return plan.execute()
    return model.execute(request)

Supporting prompt design principles:

• Provide objectives and acceptance criteria, not exhaustive exceptions. Leave exception lists to the policy-layer code.
• Explicitly grant clarification authority. Clearly state, “Ask first when information is insufficient; do not assume.” Also structure questions into missing fields plus candidate values.
• Allow better paths and leave traces. When the model chooses a non-expected solution, have it output the rule entries it relied on, making later audit easier.
• Use structured output. Constraining result fields with a fixed schema is much more stable than repeatedly instructing the model in natural language to follow a format.

5. Evaluation and Iteration: Make Specification Defects Discoverable

The ceiling of model capability increasingly depends on the quality of the evaluation system. Three mechanisms are recommended:

1. Annotate expected answers with their source rule entries, not only the final conclusion. When a model solution conflicts with the standard answer, this makes it easier to locate whether the issue is a rule gap or an alignment problem.
2. Archive “failed but reasonable” samples separately and retain them in object storage for the long term. Review them periodically and feed them back into specification revisions. Also connect monitoring and alerting to track the proportion of such conflicts.
3. Use layered scoring. Separate hard violations, which violate explicit rules, from soft deviations, which do not adopt the expected path but do not violate rules. This avoids mixing two fundamentally different error types into one score.

6. Conclusion

The mismatch between “takes off first” and “currently leading” is not fundamentally an intelligence gap. It is an alignment philosophy being amplified into product experience. If instruction following is treated as the end point of intelligence, the result is a precise interpreter that requires users to adapt in reverse. If intent inference and clarification are preserved, the system is more likely to keep producing usable results in real-world scenarios where requirements are naturally “partly vague, partly careless.”

The practical implication for enterprises is clear: use code to enforce hard constraints, use the model to handle understanding and planning, and use evaluation systems to continuously expose specification defects. If any of the three is missing, cost will be shifted to the hardest resource to scale: human attention.