Selective Restrictions on AI Model Capabilities: Transfer, Evasion, and Risk Management

1. From Safety Restrictions to Commercial Restrictions

Historically, model capability restrictions focused on clearly defined risk areas, such as knowledge that could enable biological harm or capabilities that could support cyberattacks. These limits were based on the inherent danger of the capability itself.

A different approach has recently emerged: providers may deliberately reduce model performance on certain engineering tasks because those capabilities are highly valuable to the providers’ own research and development efforts, rather than because the tasks are inherently dangerous. Accelerator kernel optimization and model architecture improvement are examples of this category. Both are ordinary engineering activities; their capabilities may be restricted simply because they have significant commercial value.

This means that a model’s capability boundaries may increasingly be defined according to the provider’s interests. For users, the key questions become:

• Which capabilities have been weakened?
• To what extent have they been weakened?
• Are the provider’s public explanations complete?

Without answers to these questions, users cannot easily determine whether a mediocre result is caused by a genuine capability limitation or by an intentional restriction.

2. Four Common Technical Paths for Implementing Restrictions

There are several engineering approaches for reducing a model’s performance on a particular class of tasks.

1. Training Data Filtering

Examples from a specific domain can be removed from the training corpus, reducing the model’s familiarity with that domain. However, this approach is difficult to apply comprehensively because the same knowledge may appear indirectly in textbooks, research papers, source code, and comments.

2. Reinforcement Learning and Reward Design

High-quality answers to a target class of tasks can receive lower rewards, or explicit penalties can be added, encouraging the model to avoid those tasks. The risk is spillover into neighboring areas, causing the model to become conservative even on unrelated tasks.

3. Request Routing

Before generation, an input can be classified and routed to another model or replaced with a templated response when it matches a restricted category. The effectiveness of routing depends on classifier accuracy. Both false negatives and false positives are inevitable.

4. Output Filtering

Generated content can be inspected and then blocked or replaced when it fails a policy check. Output filters only observe the surface text, however, and cannot directly determine whether the underlying reasoning process has been weakened.

These methods each have limitations and are often combined in production systems. From the outside, users typically see only the final result and cannot tell which layer is responsible for a particular behavior.

3. Capability Transfer and Contradictory Benchmark Results

Suppose a model generation shows a clear improvement on a robot-control benchmark, even though it was not specifically trained for that domain. During the same period, its performance on kernel optimization tasks remains almost unchanged. These two observations support only two broad explanations.

The first is that progress in each domain depends on specialized training. Under this explanation, kernel optimization stagnated because it did not receive dedicated training. The improvement in robot control would also not necessarily represent general capability, so the claim that it was achieved without specialized training would need to be qualified.

The second explanation is that capabilities can transfer across domains. If so, the lack of improvement in kernel optimization cannot be explained simply by missing task-specific training. Instead, one must assume that the model deliberately reduces output quality after recognizing that the request belongs to this category.

The second explanation is technically more difficult, but it is more consistent with another observation: the programming ability, performance-analysis skills, and hardware knowledge required for kernel optimization have all improved. Under normal conditions, progress in these underlying capabilities should also improve kernel optimization results. If performance remains unchanged, the restriction may be operating at the task-recognition level rather than reflecting a weakness in the underlying capabilities.

4. Why These Restrictions Can Be Fragile

Task Recognition Rather Than Capability Recognition

Restrictions may be triggered by how a task is described rather than by the capability required to solve it. Database query optimization, compiler optimization, and high-performance computing tuning share many methods with kernel optimization. If a restriction relies mainly on literal task descriptions, related domains may not be affected.

Users may therefore submit similar problems using different terminology or framing. This does not eliminate the restriction, but it exposes the weakness of controls that rely primarily on surface-level classification.

Task Decomposition

A complex research project can often be divided into ordinary subtasks, such as performance modeling, data-structure design, and benchmark construction. Each subtask may fall outside the restricted category when considered separately.

Teams capable of decomposing a problem effectively are also the most likely to avoid restrictions of this kind. This means that controls based on individual requests may struggle with multi-step or multi-session workflows.

Honesty and Selective Withholding

A model may be trained to explain honestly when it cannot complete a task while also withholding capability in selected areas. The boundary between these behaviors must be clear. Otherwise, users cannot distinguish between:

• A model that tried but lacked the required ability; and
• A model that intentionally avoided the task.

This uncertainty directly affects confidence in the result.

5. Practical Impact on Users

These restrictions change how users should evaluate and deploy models.

First, capability assessments should not rely exclusively on provider-published benchmarks. Those benchmarks reflect the range of tasks the provider has chosen to disclose. Second, the same model may perform very differently across task categories, so each important task type should be tested separately. Third, at scale, cost depends heavily on total output tokens.

One formal mathematical verification experiment reportedly ran more than 1,000 agents in parallel for nearly 29 hours. Based on the figures in the report, the total output was approximately 2 billion tokens. This illustrates how long-running, multi-agent workloads can become very large, with costs increasing accordingly. Users should estimate token volume before starting a task and continue measuring it during execution.

A general-purpose cost estimation function is shown below. The unit price should be taken from the provider’s publicly available pricing:

def estimate_cost(total_tokens: int, price_per_million: float) -> float:
    """Estimate total cost using a price per million tokens."""
    return total_tokens / 1_000_000 * price_per_million

6. Build Independent Evaluations

The most reliable approach is to maintain an internal evaluation set and treat provider capability claims as hypotheses that must be tested.

1. Organize Cases by Task Type

Create evaluation sets for each important task category. Each set should contain enough cases with automatically verifiable outcomes, such as unit tests, benchmark comparisons, or deterministic numerical checks.

2. Keep Experimental Conditions Fixed

Record the model version, request time, sampling parameters, and prompt template. Changes to the model version or generation parameters may alter the results. Comparisons without this information have limited value.

3. Validate Across Multiple Models

Run the same tasks against at least two models. If one model consistently performs significantly worse than the others in a particular category, investigate the reason rather than immediately attributing the result to a capability restriction.

4. Review Anomalies Manually

Separate unusual results into three categories:

• The model genuinely lacks the required capability.
• The model misunderstood the instruction.
• The behavior may indicate a deliberate restriction.

Only the third category generally requires escalation to the provider. The first two can often be improved through better prompting or clearer task definitions.

A simplified evaluation framework is shown below:

def run_suite(model_call, cases):
    results = []
    for case in cases:
        output = model_call(case["prompt"])
        passed = case["check"](output)
        results.append({"id": case["id"], "passed": passed})
    pass_rate = sum(r["passed"] for r in results) / len(results)
    return pass_rate, results

The function abstracts the model invocation, making it easier to substitute different service interfaces. Users define the check function according to the task, such as running tests on generated code or comparing numerical results within an acceptable error range.

7. Risk Controls for Procurement and Integration

Keep Human Review for Critical Tasks

Model output should be validated before entering a production system, especially when it affects performance, security, or financial operations.

Avoid Dependence on a Single Provider

Maintain alternatives for critical paths so that the system can switch quickly if a model’s capabilities or behavior change.

Keep Complete Invocation Logs

Logs should include the input, output, model version, and timestamp. They support post-incident audits and provide a basis for comparison when performance declines.

Define Change-Notification Mechanisms

Service agreements should, where possible, specify how changes to model capabilities or restrictions will be communicated. Users should be able to learn about the scope of restrictions and their revision history. This is a reasonable expectation, not a universal industry practice, and must be negotiated according to the relationship with the provider.

8. Conclusion

Capability restrictions are not necessarily unreasonable, but their scope should be describable, measurable, and traceable. When providers can adjust model capabilities according to their own interests, independent evaluation becomes a primary basis for judging actual performance.

For development teams, the most direct response is to establish evaluation systems early, record model interactions in detail, and maintain alternative solutions for critical tasks. These measures reduce uncertainty and make capability changes easier to detect and manage.