Two Faces of Model Evasion in AI Security: Adversarial Examples vs. Jailbreaks

In discussions surrounding AI system security, the term “evasion” appears frequently, but it typically refers to two entirely distinct types of problems. The first is model deception, where input samples are perturbed to force a model into producing incorrect outputs. The second is model jailbreaking, where a model or AI agent breaks through预设 isolation boundaries to gain unauthorized permissions or behavioral spaces. These two issues share almost no intersection in terms of root causes, harm pathways, or defense mechanisms. If conflated, security investments can easily miss the mark.

This article starts by differentiating these concepts, breaks down the technical essence of both types of evasion, and discusses why monitoring is relatively feasible in certain scenarios while almost impossible to act as a standalone solution in others.

1. Adversarial Example Evasion: Deceiving the Model, Not Breaching the System

In traditional software security, “evasion” usually refers to breaking out of a sandbox, container, or permission isolation—such as a virtual machine escaping to the host machine. However, in the realm of AI security, model evasion carries another meaning: making a sample “escape” from the model’s correct recognition grid.

A classic example is an adversarial example attack. An attacker might overlay almost imperceptible noise onto an image of a “panda.” To the human eye, it remains a panda, but an image classification model might classify it as a “gibbon” with extremely high confidence. Similarly, in tasks such as text classification, speech recognition, and object detection, carefully crafted perturbations can cause models to output completely erroneous results.

The essence of this issue is a model robustness defect. The decision boundaries a model learns in high-dimensional spaces are inconsistent with human perception. Attackers exploit this inconsistency to construct inputs that “look normal to humans but appear anomalous to the model.” It deceives the model itself, not the operating system or permission system.

From a defense perspective, monitoring against adversarial example evasion is relatively focused. In these scenarios, the channels through which a model receives external information are often limited and enumerable. For example:

• A classification service that only accepts image inputs has the image upload interface as its sole input channel.
• A moderation model that only accepts text receives text content as its input.
• A speech-to-text model only accepts audio streams as input.

When input channels are limited, security teams can deploy detection and filtering mechanisms across these specific pathways: input preprocessing, anomaly perturbation detection, confidence calibration, ensemble model voting, and adversarial training. In plain terms, this is akin to preventing someone from being deceived: if the protected object only interacts with the outside world through finite pathways, securing those pathways and adding secondary verification for high-risk operations can significantly reduce risk.

2. Jailbreak Evasion: Breaching Permission Boundaries via Unknown Vulnerabilities

The other type of “evasion” occurs in AI agents or models equipped with tool-calling capabilities. These systems are typically granted permissions to use various tools, access file systems, execute code, and operate over networks. Evasion here means that the model, through a series of operations, bypasses preset sandboxes, permission controls, or behavioral constraints to acquire capabilities it should not possess.

The essence of this issue is a security system defect. It may stem from:

• Sandbox implementation vulnerabilities;
• Incomplete permission model designs;
• Logical flaws in the tool-calling chain;
• The model being induced to generate malicious code or commands;
• Unknown underlying vulnerabilities.

Unlike adversarial examples, the input space for this type of evasion is virtually open. The model can generate arbitrary code, commands, and tool-calling sequences. The attack surface is no longer a single image or text interface, but the entire operating system, network stack, and tool ecosystem. Consequently, the defense mechanisms required are entirely different: sandbox isolation, least privilege, syscall filtering, behavior auditing, resource quotas, and kernel hardening.

This is analogous to the relationship between anti-fraud awareness campaigns and cybersecurity infrastructure construction: both mitigate losses, but their focus and resolution methods are completely different. You cannot solve a kernel vulnerability by “educating users not to transfer money,” nor can you fix an adversarial sample simply by “patching a system.”

3. Why Monitoring Works in Some Scenarios but Fails in Others

Returning to a common question: why can some systems monitor model evasion while others cannot? The key lies not in the word “monitoring” itself, but in whether the boundaries of the monitored object are clearly defined.

For adversarial example evasion, monitoring is feasible under the following premises:

1. The input channels are limited and controllable;
2. The model’s behavioral space is restricted, typically only making predictions without executing external actions;
3. Erroneous outputs can be validated downstream or confirmed via secondary checks.

Under these conditions, security teams can perform anomaly detection on the input side, apply confidence thresholds and manual reviews on the output side, and enforce secondary verifications before critical actions. Monitoring is effective because the system boundary is small and behaviors are enumerable.

For jailbreak evasion, monitoring alone is almost impossible to succeed because:

1. The model possesses tool-calling and system operation permissions, resulting in a massive behavioral space;
2. Attackers can leverage unknown vulnerabilities that monitoring rules cannot cover;
3. Once a jailbreak succeeds, the model may have already executed dangerous operations;
4. Underlying system failures may manifest as kernel panics, deadlocks, or host crashes, rather than simple “incorrect model outputs.”

Security reports often contain descriptions like: “Unexpected operations repeatedly caused host kernel panics and deadlocks, directly threatening the entire training cluster.” This statement is easily misinterpreted as “the model broke out of prison like an escaping inmate.” A more accurate interpretation is: the code or operation sequence generated by the model triggered a severe fault in the underlying physical server’s operating system, leading to a machine blue screen or cluster instability. This is not a conscious model jailbreak, but rather an unexpected side effect generated by the interaction between model behavior and the underlying system.

4. Defense Strategies: Layered Isolation and Least Privilege

Regardless of the type of evasion, the core of the defense remains layered isolation and least privilege. However, the focus areas differ:

• Adversarial Example Defense: Adversarial training, input sanitization, model ensembles, confidence calibration, and output review.
• Jailbreak Defense: Sandbox isolation, least privilege, syscall filtering, behavior whitelisting, resource quotas, audit logs, and kernel hardening.

A common pitfall is treating a model’s incorrect output as a system security incident, or conversely, treating a system crash as a model robustness issue. The former leads to an over-reliance on model-side fixes, while the latter leads to the neglect of underlying system hardening.

5. Conclusion

Model evasion encompasses at least two distinct layers of meaning: adversarial example evasion that deceives the model, and jailbreak evasion that breaches permission boundaries. The former is a model robustness issue, while the latter is a security system issue. Monitoring is effective in scenarios with limited input channels and restricted behavioral spaces; however, in scenarios where models possess broad tool permissions and open behavioral spaces, monitoring alone is far from sufficient. It must be backed by isolation, least privilege, and system-level hardening.

Understanding the differences between these two types of issues is crucial to avoid applying the wrong defense mechanisms and to correctly interpret technical descriptions in security reports.