When AI Models Are Restricted but Hard to Retire: The Governance Challenge of High-Risk AI

1. Why Restrictions and Dependence Can Coexist

A particularly concerning AI governance dilemma arises when a buyer believes that a model supplier’s usage boundaries undermine operational control, while the supplier argues that certain high-risk uses cannot be authorized solely through a procurement contract. If the parties cannot reach agreement, a commercial relationship may escalate into a supply-chain security dispute.

The source material describes a similar conflict, but its specific dates, administrative findings, negotiation details, and individual statements cannot be verified from a single account. This article therefore does not assign responsibility for any particular event. Instead, it focuses on the broader question the situation illustrates: Why might a model that is supposed to be restricted remain embedded in a critical business workflow?

The answer is not simply that the customer “cannot live without it.” AI has evolved from a standalone tool into a system component. A model may connect to knowledge retrieval, business rules, identity and access controls, data processing, and human approval workflows. Disabling one model may appear to be a configuration change, but in practice it can require revalidating the entire business chain.

Restrictions and continued use are therefore not always logically contradictory. An organization might stop making new purchases while creating a transition plan for existing systems. It might also disable high-risk functions first and replace other capabilities gradually. However, any transition must define the scope of authorization, its duration, and the applicable audit mechanisms. Otherwise, a “temporary exception” can become an open-ended dependency.

2. The Core Control Dispute: Who Decides How a Model Is Used?

Control over high-risk AI has at least three distinct dimensions. They should not be treated as the same thing.

1. Procurement Rights Do Not Mean Unlimited Usage Rights

Buyers typically want to configure technology according to their own policies and integrate it into existing business processes. Purchasing a service, however, does not automatically authorize every possible use. Actual usage remains subject to contracts, laws, security requirements, and deployment conditions.

Conversely, a supplier’s usage restrictions do not mean that the supplier can replace the buyer in making public or business decisions. Reasonable boundaries should clearly define which capabilities are available, which uses are prohibited, and which operations require additional approval. They should not use vague language to interfere with every aspect of the customer’s business.

2. Usage Restrictions Are Not Command Authority

At the engineering level, it is important to distinguish between refusing to provide a capability and deciding whether a specific real-world action should occur.

For example, preventing a system from exposing certain sensitive data is an access-control measure. Preventing a model from directly executing a critical operation is a form of capability isolation. Neither arrangement is equivalent to allowing the model supplier to decide whether a real-world action should be carried out.

Describing every restriction as the supplier entering the business command chain can obscure the questions that actually need to be answered: Who defines the restriction? Where is it enforced? How is it verified? What happens when the parties disagree?

3. Contractual Requirements Must Become System Constraints

“Prohibit improper use” is a policy principle, not a complete technical design.

For a boundary to be enforceable, the system must define the relevant subjects, resources, actions, and conditions. For example:

• Which roles may request access to sensitive information?
• Which tasks may produce recommendations only?
• Which operations require confirmation from two authorized people?
• When does an authorization expire?

Only after these requirements are translated into technical controls do governance obligations become more than contractual language.

3. Supply-Chain Risk Should Not Become a Label for Every Dispute

AI supply-chain risk should be assessed by category rather than treated as one broad label.

Risk category	Core question	Primary governance direction
Data risk	Could inputs, outputs, or logs expose sensitive information?	Data classification, minimal processing, and access auditing
Model risk	Are outputs reliable, and can updates change behavior?	Scenario-based evaluation, version management, and regression testing
Service continuity risk	Can the business continue if the service becomes unavailable?	Degraded operation, alternative validation, and recovery exercises
Permission risk	Can the model call business capabilities beyond its authority?	Least privilege, execution isolation, and human approval
Dependency risk	Is the system overly tied to one implementation?	Interface abstraction, portable assets, and an exit plan
Governance risk	Are usage boundaries and responsibilities clear?	Contractual terms, responsibility matrices, and dispute procedures

These risks may be related, but they are not equivalent. A disagreement between a supplier and a buyer about permitted use does not by itself prove that a data breach occurred. Similarly, a model passing a capability test does not prove that it is suitable for unsupervised execution of high-consequence tasks.

Risk assessments should correspond to specific evidence, impact scopes, and mitigation measures. The more serious the label, the more important it is to identify what exactly it concerns: the supplier, a model version, the deployment method, or a particular business use.

4. Why Replacing a Model Is Harder Than Changing an Interface

A common engineering assumption is that a model can be swapped seamlessly as long as the input and output formats are compatible.

In reality, interface compatibility is only the starting point of a migration.

1. The Same Task Can Produce Different Decision Behavior

Different models may handle ambiguous instructions, insufficient evidence, and conflicting information in different ways. Some are more likely to continue answering, while others may request clarification. Some can follow complex formats but may not consistently recognize high-risk requests.

These differences affect downstream processes. Parsers, approval rules, and human operating procedures that depend on the original model’s output patterns may all require adjustment.

2. Historical Evaluation Results Cannot Be Automatically Reused

The fact that the original system passed an evaluation does not mean that the replacement system inherits the same level of trust. Migration requires renewed validation of task quality, error types, permission boundaries, and exception-recovery capabilities.

Particular attention should be paid to failures that appear successful:

• The output is structurally complete, but its supporting references are wrong.
• The conclusion is fluent, but it omits a critical limitation.
• A business request is executed even though the requesting party lacks the necessary authority.

3. Organizational Workflows Also Create Dependency

Over time, employees adapt to a system’s response style, timing, and interaction patterns. Even if the replacement model performs similarly on average, it may increase review costs and operational mistakes.

Migration must therefore include both technical validation and user training. Interface integration alone is not sufficient.

5. Turning High-Risk Boundaries into an Enforceable Architecture

A reliable control system should not depend solely on a model’s ability to understand and follow rules. Deterministic constraints must exist outside the model.

1. Separate Reasoning Capability from Execution Authority

A model may analyze material, propose recommendations, or generate a draft plan for review. That does not mean it should automatically receive execution privileges.

A general architecture is:

Business request → Identity and permission checks → Model analysis → Policy checks → Required human approval → Restricted execution → Audit record

The key principle is that model-generated text must not become an authorization credential. The execution system should independently verify the caller’s identity, resource scope, approval status, and operating conditions.

2. Set Automation Levels According to Consequences

Not every task requires the same level of approval. Tasks can be grouped into information assistance, low-risk operations, and high-consequence operations according to their potential impact.

Information-assistance tasks should focus on accuracy and evidence sources. Reversible, low-risk operations may be automated within defined limits. Operations involving personal safety, significant rights, or irreversible consequences require stronger authorization and human control.

Human involvement must also be substantive. Reviewers should be able to see the evidence, constraints, and proposed action. They must have enough time and authority to reject or modify the plan, rather than merely clicking a confirmation button.

3. Treat External Content as Untrusted Input

Documents, messages, and retrieval results read by a model may contain manipulative instructions. The system must not promote such content to the level of business rules simply because it appears in the model’s context.

External data should be separated from system instructions, and the execution layer should verify the actual action. Even if a model is misled, an attacker should not gain new data access or operational privileges as a result.

4. Fail Safely in High-Risk Situations

When identity verification, policy services, or approval chains are unavailable, high-risk operations should stop by default rather than bypassing checks to keep the workflow moving.

At the same time, organizations should prepare for human takeover or read-only degraded modes. Stopping automated execution should not necessarily mean bringing the entire business to a halt.

6. Auditing Must Explain the Basis for an Action, Not Just Record the Answer

Saving conversation content alone is usually insufficient for accountability in a high-risk system.

For a critical operation, records should make it possible to reconstruct at least the following:

• Who initiated the request
• Which permissions were used
• Which model version was involved
• Which policies were applied
• Which evidence was referenced
• Who approved the action
• What was ultimately executed

These records should be linkable, but that does not mean retaining all raw data indefinitely. Audit design must also follow data-minimization principles, redact sensitive fields, and restrict query and export permissions.

In addition, a model’s explanation should not be treated as a literal record of its internal reasoning. Audits should prioritize verifiable inputs, retrieval evidence, policy results, and execution logs rather than relying on natural-language explanations as the sole basis for accountability.

7. Exit Capability Reduces the Cost of Control Disputes

The deeper an organization’s dependence on a single model, the more likely a disagreement can escalate into a business crisis. The solution is not to assume in advance that one party is inherently trustworthy. It is to build and validate the ability to exit.

First, business rules, permission policies, and knowledge assets should be decoupled from model-integration logic. Even when the underlying capability changes, the organization should retain control over its own data governance and execution controls.

Second, organizations should maintain evaluation sets that reflect real operating conditions. In addition to common tasks, these sets should cover insufficient evidence, conflicting instructions, sensitive-data access, unauthorized requests, and attempts to manipulate the model through external content.

Third, migration should proceed in stages along the business workflow. Organizations can begin with lower-risk scenarios, conduct comparison testing without affecting real decisions, and then expand the scope gradually. For high-risk functions that have not yet completed validation, manual processes or suspended automation should remain available.

Finally, organizations should regularly rehearse service outages, permission revocation, and supplier exit. The objective is not merely to verify that a fallback system can return an answer. It is to confirm that critical operations can continue within safe boundaries and that the organization knows when automated execution must stop.

8. The Goal of Governance Is Verifiable Responsibility

Debates about whether AI has consciousness or agency cannot replace concrete risk analysis. The need for stronger controls should be determined primarily by the system’s capabilities, permissions, deployment environment, and potential consequences—not by unverified speculation.

Likewise, contradictory information during a negotiation is not enough to establish the existence of a comprehensive forced-takeover plan or a particular decision-making motive. Technical governance must distinguish observable facts from interpretive claims.

High-risk AI does not need one party to gain unrestricted control. It needs an operating model with clear responsibilities, enforceable boundaries, auditable processes, and a viable exit path.

Suppliers should clearly define capability limitations and service conditions. Buyers should retain independent evaluation and business decision-making authority. Development teams should ensure that the mistake of any one party cannot directly bypass the security boundary.

When an organization can explain why a system acted, prevent unauthorized actions, and safely replace the system when necessary, AI has truly entered an engineering phase that can be governed.