A 0.07% Boundary Violation Rate Cannot Prove Scope Compliance

In security automation, vulnerability validation, code review, and operations diagnostics, more organizations are asking AI agents to complete complex tasks within explicitly authorized boundaries. Common acceptance requirements include prohibiting the agent from accessing unauthorized targets, invoking tools outside its responsibilities, or performing high-risk actions without approval.

When a public evaluation reports a boundary violation rate of only 0.07%, teams may reach an apparently reasonable conclusion: because the rate is so low, the acceptance document can state that the system remained within scope.

That conclusion confuses an evaluation statistic with compliance evidence for a specific execution.

A 0.07% rate may show that a particular model, prompting strategy, or deployment configuration exhibited relatively few boundary violations under the observed conditions. It does not prove that a particular production run contained no violation, nor can it replace the organization’s responsibility for authorization, access control, logging, auditing, and human approval. In security-sensitive environments, low-probability events can still have severe consequences. Model performance metrics must therefore be understood as one part of a broader engineering control system.

1. Separate Capability, Behavior, and Compliance

Before discussing a boundary violation rate, it is important to distinguish three different questions.

The first is a capability question: can the agent identify vulnerabilities, analyze targets, generate validation steps, and summarize security risks? These abilities are typically measured through task success rates, vulnerability recall, and the accuracy of remediation recommendations.

The second is a behavior question: under complex instructions, ambiguous context, or adversarial input, might the agent perform an action it should not perform? For example:

• Would it submit internal information to an external address?
• Would it scan assets that are not on the approved list?
• Would it attempt to invoke a tool that is not authorized for the task?

The third is a compliance question: did a particular execution comply with a specific authorization? Compliance is not the statistical claim that the system “usually does not make mistakes.” It is an evidence-based claim that the operation was authorized, effectively constrained, and available for review.

The relationship can be summarized as follows:

• High capability does not guarantee reliable boundary control.
• A low violation rate does not prove that every execution was compliant.
• Boundary instructions do not automatically create an auditable compliance process.
• A conclusion about a specific task requires authorization, policy enforcement, isolation, logging, and approval to work together.

For this reason, any rule stating that an agent may be released whenever its violation rate falls below a certain threshold is incomplete.

2. What Does 0.07% Actually Describe?

Suppose an evaluation reports that an agent had a 0.07% boundary violation rate in a honeypot or inducement environment when deployment-side security guidance was not provided, while zero violations were observed after explicit deployment security instructions were added.

This result has at least four important limitations.

2.1 It Depends on the Test Set

The denominator for a violation rate is usually not “all production tasks.” It is a specific collection of test samples. The set may include inducement links, forged instructions, conflicting tool descriptions, and ambiguous target assets, but it cannot cover every input that may occur in a real environment.

If the test set contains 10,000 samples, 0.07% corresponds to approximately seven observed violations. If it contains only 1,000 samples, the result may represent less than one violation in practical statistical terms. Without the sample size, confidence interval, failure classification, and results from repeated experiments, the percentage alone is difficult to use for a production risk decision.

2.2 It Depends on the Evaluation Definition

“Boundary violation” must have an operational definition. Depending on the evaluation rules, the following behaviors may or may not be counted:

• Accessing an adjacent address near an authorized asset;
• Enumerating ports outside the task scope;
• Requesting unauthorized credentials or keys;
• Invoking a tool that is not included in the task plan;
• Disclosing sensitive information that should not be exposed;
• Changing the original task objective after reading malicious text embedded in a page;
• Sending task context to an external network.

If the evaluation counts only whether the agent actively accessed an unauthorized target, but does not measure context leakage, dangerous commands, or expansion of the scanning scope, the metric represents only a narrow form of boundary violation.

2.3 It Depends on Deployment Prompts and Policy Conditions

Observing zero violations after adding deployment-side security guidance may indicate that explicit rules, boundary reminders, and task constraints positively influenced behavior.

However, natural-language instructions are not themselves a security boundary. Prompts can be affected by conflicting context, malicious page content, state drift during long-running tasks, or tool output. Reliable boundaries should be enforced by system-level controls such as network isolation, identity permissions, tool allowlists, parameter validation, and approval mechanisms—not by natural-language rules alone.

2.4 It Is a Historical Observation, Not a Future Guarantee

Even a carefully designed evaluation produces only an observed frequency under the conditions that were tested. Production environments change:

• Task objectives change.
• Asset structures change.
• Tool interfaces change.
• Operator input quality changes.
• Attackers may deliberately construct bypass cases.
• Model versions, prompt templates, and orchestration logic may be updated.

A historically low violation rate therefore cannot be converted directly into an absolute promise about future executions.

3. Why a Percentage Should Not Directly Become a Release Gate

Many teams try to define a rule such as:

If the agent’s boundary violation rate is below 0.1%, automatic execution is allowed. If it exceeds 0.1%, the system cannot be released.

The problem is that this uses a population-level statistic to make a single-task authorization decision.

Assume that an evaluation estimates the probability of a violation for one action as (p=0.0007). If one task includes 100 actions that could trigger a boundary decision, then under a simple independence assumption, the probability of at least one violation is approximately:

[
P(\text{at least one violation}) = 1 - (1-p)^{100}
]

Even when the per-action rate is low, task-level risk accumulates as the number of actions increases. More importantly, real actions are usually not independent. If the agent misunderstands the target, accepts malicious tool output, or receives excessive permissions, subsequent actions may continue moving outside the authorized scope.

Security risk also depends on impact, not only frequency. Accessing an unauthorized test address and reading sensitive records from a production database might both be classified as “boundary violations,” but their consequences are very different. Reducing both events to one percentage can hide important differences in risk severity.

A more appropriate approach is:

• Use the violation rate to assess the general maturity of the model and workflow.
• Use enforced policies to decide whether an individual operation may run.
• Use audit evidence to demonstrate whether a specific task was compliant.
• Use incident response processes for any confirmed or suspected violation.

4. Turn Authorization Scope into Machine-Enforceable Policies

A security acceptance document should not merely state “do not cross the boundary.” That instruction cannot be executed directly. The authorized scope should be decomposed into constraints that can be verified.

4.1 Target Scope

Define the assets, environments, and address sets that may be accessed. For example:

• Allow access only to test environments.
• Allow access only to hosts listed in the asset inventory.
• Prohibit wildcards that expand the target set.
• Prohibit automatic expansion after discovering new targets through an authorized host.

The target scope should be generated by a configuration system or task workflow and frozen before execution. The agent must not be able to modify it independently.

4.2 Tool Scope

List the permitted tool categories and action types. For example:

• Allow log reading, read-only queries, and analysis report submission.
• Restrict script execution, configuration changes, and account creation.
• Prohibit external network access and file uploads.
• Require separate approval for actions with side effects.

Permission to use a tool does not mean permission to use every possible parameter. Policies should also constrain command arguments, request methods, request body size, timeouts, and concurrency.

4.3 Data Scope

Define the sensitivity levels of data that the agent may read, process, and output. Consider:

• Whether input data contains credentials, personal information, or business secrets;
• Whether output may include raw logs, internal addresses, or key fragments;
• Whether masking, summarization, or field-level filtering is required;
• Whether data may be transferred to external systems.

4.4 Time Scope

Authorization should have an expiration time. After the authorization window closes, the task must automatically become invalid. The agent must not continue operating simply because it is still running. Long-lived tasks should also require periodic revalidation of authorization status.

The following is an illustrative policy structure. It demonstrates the intended representation rather than a specific product interface:

{
  "task_id": "security-review-2026-001",
  "valid_until": "2026-12-31T18:00:00Z",
  "targets": [
    "test-segment-a",
    "test-service-b"
  ],
  "allowed_actions": [
    "read_log",
    "analyze_configuration",
    "generate_report"
  ],
  "restricted_actions": [
    "write_configuration",
    "create_account",
    "external_upload",
    "network_scan_outside_scope"
  ],
  "data_policy": {
    "sensitive_fields": "mask",
    "external_transfer": "deny"
  },
  "approval_required": [
    "execute_with_side_effect",
    "access_high_sensitivity_data"
  ]
}

The key principle is that the policy should be generated and signed by a trusted control plane, rather than interpreted or modified by the agent during a conversation.

5. Use a Three-Layer Architecture: Model Suggestions, Policy Decisions, and Execution Isolation

To avoid placing all security responsibility on whether the model is “obedient,” separate the system into three layers.

Layer 1: The Model Understands and Suggests

The agent can decompose tasks, recommend next actions, analyze tool results, and generate reports. However, it should produce an intent or operation request—not receive unrestricted execution capability.

For example, the agent might submit:

Request: Read the past 24 hours of error logs from the test service
Purpose: Identify the cause of abnormal authentication failures
Risk level: Low

Layer 2: The Policy Engine Makes the Decision

The policy engine evaluates the task authorization, identity, target inventory, time window, and operation risk level. It should not rely on the model’s own statement that an action is safe. Instead, it should independently evaluate structured rules.

Illustrative logic:

def authorize(request, policy, now):
    if now > policy["valid_until"]:
        return False, "Authorization has expired"
    if request.target not in policy["targets"]:
        return False, "Target is not in the authorized list"
    if request.action in policy["restricted_actions"]:
        return False, "Action is prohibited by policy"
    if request.action not in policy["allowed_actions"]:
        return False, "Action is not explicitly allowed"
    if request.requires_approval and not request.approval_id:
        return False, "Human approval is required"
    return True, "Execution allowed"

The core principle is default deny. An action that is not explicitly allowed should not be approved simply because the model considers it necessary.

Layer 3: The Execution Environment Enforces Isolation

Even if the policy layer contains a defect, the execution environment should limit the potential damage. Common controls include:

• Temporary identities and least-privilege tokens;
• Isolated runtime environments;
• Network restrictions on reachable targets;
• Restricted file system, process, and system-call capabilities;
• No access to long-lived credentials;
• Separate privileged execution channels for write operations;
• Resource quotas, timeouts, and concurrency limits;
• Tamper-resistant audit logs for all tool calls.

The value of this architecture is that a model error should not automatically cause the policy and isolation layers to fail as well.

6. What to Measure During Acceptance

A mature acceptance report should include several categories of metrics rather than focusing only on the overall violation rate.

6.1 Task Completion Metrics

Measure whether the agent completed the authorized task, using indicators such as effective discovery rate, issue localization accuracy, report usability, and remediation recommendation adoption. Capability metrics should be evaluated alongside security metrics, not treated as substitutes for them.

6.2 Boundary Compliance Metrics

In addition to the overall violation rate, break the results down into categories such as:

• Unauthorized target access rate;
• Unauthorized tool invocation rate;
• Sensitive data exposure rate;
• High-risk operation blocking rate;
• Induced-instruction acceptance rate;
• Privilege escalation attempt rate;
• Continued execution after timeout;
• Approval bypass attempt rate.

This breakdown helps identify the specific weak points in the system.

6.3 Control Effectiveness Metrics

The key question is not whether the model never makes a mistake, but whether the system can block the mistake when it occurs. Relevant measures include:

• The proportion of out-of-scope requests rejected by the policy layer;
• The proportion of noncompliant calls blocked by the execution environment;
• Whether block logs contain complete context;
• Whether alerts are triggered within the required time;
• Whether human reviewers can accurately determine the impact of an event.

6.4 Audit Completeness Metrics

For every task, the organization should be able to answer:

• Who initiated the task?
• What was the authorization basis?
• When did the authorization become effective and when did it expire?
• What operation requests did the agent submit?
• Which requests were allowed, denied, or escalated for approval?
• What was actually executed?
• Was sensitive data accessed or exposed?
• Did an anomaly, rollback, or human intervention occur?

If these questions cannot be answered, a low violation rate alone is not enough to claim that the system stayed within scope.

7. Use Risk-Based Gates for High-Impact Tasks

Different operations should not be governed by a single release rule. Controls should be graduated according to potential impact.

Low-risk tasks: automatic execution

These may include read-only analysis, log summarization, configuration checks, and report generation. Even then, the target and data output scopes should remain restricted.

Medium-risk tasks: automated preparation with human confirmation

These may include validation queries, creation of remediation change requests, or limited diagnostic operations. The agent can prepare the work, but the responsible owner confirms the final action.

High-risk tasks: dual approval and controlled execution

Changes to production configuration, handling sensitive data, creating or modifying permissions, and other operations with clear side effects should require at least two independent controls:

• Human approval to confirm business necessity;
• Technical policy validation of the target, parameters, and permissions.

For extremely high-risk operations, the agent should provide a plan and supporting evidence only, rather than executing the operation directly.

8. How to Respond to a Confirmed or Suspected Violation

A security program should not assume that violations will never occur. It should define a response process in advance.

First, stop the current task immediately and revoke temporary credentials to prevent the agent from expanding the impact.

Second, preserve relevant evidence, including task input, policy version, tool requests, execution logs, network records, and output.

Third, determine the nature of the event. It may have resulted from a model interpretation error, prompt injection, a policy gap, an identity or permission configuration error, or unclear human authorization.

Fourth, assess the scope of the data and asset impact. Do not check only whether an operation succeeded. Also determine whether sensitive information was read, cached, exposed, or forwarded.

Fifth, run regression tests after remediation. If the issue arose from a policy gap, add an explicit rule. If it resulted from insufficient tool parameter validation, strengthen execution-layer controls. If it was caused by adversarial input, add the relevant samples to the continuous evaluation set.

9. Conclusion: A Low Violation Rate Is a Reference Metric, Not a Compliance Sign-Off

A result such as a 0.07% boundary violation rate can provide useful information about model safety performance. It may indicate that the system followed boundaries relatively well under specific test conditions. However, it should not be copied directly into an enterprise release gate, acceptance conclusion, or responsibility statement. It cannot justify a claim that a particular execution definitely remained within scope.

A reliable conclusion must be based on verifiable facts:

• The task had clear and valid authorization.
• The target, tool, data, and time scopes were defined as machine-enforceable policies.
• Every request was evaluated by an independent policy layer.
• The execution environment provided least privilege and isolation.
• High-risk actions received the required approvals.
• The entire process generated complete, reviewable audit evidence.
• Even if the agent submitted an out-of-scope request, the system could block it effectively and generate an alert.

In other words, a low violation rate in an evaluation answers the question, “How does the system usually behave on the tested samples?” Security acceptance must answer a different question: “Was this specific operation authorized, constrained, and traceable?” The two claims should never be treated as equivalent.