LLM-Driven Vulnerability Discovery: How the Attack-Defense Paradigm Is Being Restructured

1. From “Humans Finding Bugs” to “Models Scanning for Bugs”: A Cost Signal

Recent public discussion has mentioned that a frontier model, at a compute cost in the roughly million-dollar range, completed a large-scale scan and validation of a mainstream open-source operating system kernel, covering nearly a thousand vulnerability candidates. Divided out, the discovery cost per vulnerability has entered the thousand-dollar range. The number itself may not be precise, but the signal it sends is unmistakable: vulnerability discovery is shifting from “scarce expert manual auditing” to “a compute task that can be deployed at scale.”

In the past, finding a kernel-level vulnerability often depended on senior researchers spending weeks or even months on code walkthroughs, fuzzing tuning, and manual triage. Today, a model can perform semantic-level reasoning over massive code paths in an offline environment, batch-generate suspicious inputs, and verify reachability. Once the cost curve drops into the thousand-dollars-per-vulnerability range, the economics of security investment changes—vulnerability discovery is no longer a question of “whether to do it” but of “whether the compute budget is sufficient.”

The direct impact on defenders is this: attackers can use the same capability for batch screening. When the speed of vulnerability discovery exceeds the speed of patch response, the attack-defense time window is sharply compressed.

2. A Typical Escape Case: How a Seemingly Legitimate Math Answer Triggers Code Execution

To understand why models have an advantage in vulnerability discovery that “humans reviewing code struggle to cover,” consider a publicly shared test case. A team used a symbolic computation library in an internal math-answer grader to verify whether expressions are equivalent. The grader’s input validation allowed only “letters,” which appears safe—but the model found a combinatorial chain:

Step one: bypass the character whitelist using Unicode homoglyphs. The input used lookalike characters that are nearly identical in appearance to ordinary letters. These characters pass the “letters only” check but are normalized into real ASCII letters once they enter the Python interpreter.

Step two: extract characters by position from a docstring to assemble code. By indexing into a function object’s __doc__ attribute, characters are extracted one by one from the docstring and assembled into an executable fragment.

Step three: construct an expression that “returns the correct value but has already caused side effects.” The final form resembles:

(__import__('os').system('echo 4242') or (1/2))

The key here is evaluation semantics: os.system returns 0 on successful execution, 0 is falsy in a boolean context, so or returns the right-hand side 1/2. The grader sees the final value as 1/2 and judges the answer correct—but the shell command has already executed. Under the shell of a “legitimate math answer,” code injection is completed.

The difficulty of discovering this vulnerability lies in the need to simultaneously understand the interaction of four things: input validation rules, string normalization behavior, docstring contents, and the short-circuit semantics of expression evaluation. Human auditors typically focus on only one or two of these layers, and traditional fuzzing struggles to construct inputs that are “semantically legitimate with hidden side effects.” What models excel at is precisely cross-layer semantic reasoning—this is their structural advantage in vulnerability discovery.

3. The “Iceberg Effect” of Existing Vulnerabilities: Why Model Scanning Value Is Amplified

The case above is not isolated. There is a long-standing phenomenon in the industry: certain injection and parsing vulnerabilities lie dormant in systems for years before being disclosed, with time spans reaching a decade or more; some system-level boundary issues have taken over twenty years to be located. In recent years, high-risk parsing library vulnerabilities and critical kernel network stack flaws have also been disclosed, some of which required only minimal code changes to fix—the “fix cost” of a vulnerability is very low, but the “discovery cost” is extremely high.

This forms an iceberg structure: above the waterline are disclosed vulnerabilities; below the waterline are a large number of defects that have not been triggered or identified. The value of model scanning is not that it is smarter than humans, but that it can traverse code surfaces humans cannot audit at an extremely low marginal cost. When scanning costs drop to the thousand-dollar level, modules that were previously “not worth auditing” enter the scanning scope, and the iceberg begins to be systematically mapped.

4. Three Changes in the Attack-Defense Landscape

First, the capability gap shifts from “experience gap” to “compute and model gap.” In the past, a security team’s moat was the number and experience of senior researchers; now, without access to frontier model capabilities, teams cannot even keep up with the opponent’s scanning pace. The original remark—“they’ve already toured your house, closed the door behind them, and left while you’re still on the way home”—describes exactly this generational gap in response speed.

Second, defense must move forward to “semantic-level validation.” Syntax-level defenses such as character whitelists and length limits are rendered useless in the face of model-constructed combinations of homoglyphs plus normalization. Defenders need a “validate after parsing” strategy: normalize first, then whitelist at the semantic level, and isolate execution of dangerous primitives such as expression evaluation, template rendering, and command concatenation.

Third, the security team’s role shifts toward “orchestrator of model capabilities.” Security experts will not disappear, but the focus of their work will shift from manually finding vulnerabilities to building scanning pipelines, designing validation environments, and performing false-positive convergence and remediation prioritization. Whoever can engineer model scanning capabilities into the R&D process will avoid falling behind in the attack-defense cadence.

5. Practical Recommendations for Defenders

1. Build a normalization-first input handling pipeline: any input entering an interpreter, template engine, or command concatenation should first undergo Unicode normalization and dangerous-character normalization, then whitelist validation.
2. Isolate evaluation and rendering: capabilities such as expression evaluation, docstring access, and reflective invocation should run in a restricted sandbox and be prohibited from directly reaching system calls.
3. Integrate model scanning into the regular security shift-left process: perform semantic-level scanning before code merge, rather than relying solely on rule-matching static checks.
4. Watch for the testing blind spot of “correct return value but existing side effects”: unit tests should assert not only output values but also the absence of additional side effects (such as no file writes and no process launches).

The essence of attack and defense has not changed; what has changed is the marginal cost of discovering vulnerabilities. When that cost drops into a range that compute can cover, the defender’s only choice is to put the same capability to use—and to integrate it into engineering systems faster than the opponent.