Improving Vulnerability Research Efficiency and Controllability with Security LLMs

1. From “Discovering Numerous Vulnerabilities” to “Proving Security Value”

When LLMs begin to participate in code auditing, anomaly analysis, and patch generation, the efficiency boundaries of security research are indeed shifting. Tasks that previously required researchers to repeatedly read code, construct inputs, and analyze execution states can now be partially handled by automated systems.

However, when discussing such technologies, a critical clarification is needed: the number of suspected vulnerabilities output by a model does not equal the number of actual vulnerabilities in the system; the number of triggered crashes does not equal the number of independent security flaws discovered.

Performance figures lacking verifiable sources, test conditions, and disclosure records should not be directly treated as technical facts. Conclusions like “completing decades of work in a short time” or “discovering hundreds of unknown critical vulnerabilities” only hold discussion value when code versions, deduplication rules, reproduction conditions, and independent confirmation mechanisms are explicitly defined.

What truly deserves developers’ attention is not whether a model possesses mysterious “hacker capabilities,” but the following three questions:

First, can it more accurately locate defects in complex code?

Second, can it provide repeatable verification evidence for defects under authorized and isolated conditions?

Third, can it help teams complete remediation and regression rather than generating more pending alerts?

Focusing on these three questions is the only way to establish a practical security development strategy.

2. The Core of Capability Enhancement: Translating Code Understanding into Testable Hypotheses

1. From Local Pattern Matching to Cross-Function Reasoning

Many traditional detection methods excel at identifying explicit dangerous patterns, such as missing boundary checks, unreleased resources, or unvalidated inputs directly entering sensitive operations.

However, real vulnerabilities are often scattered across multiple functions and execution stages. For instance, an object might have clear ownership upon creation, which transfers after asynchronous processing, while another branch retains the old reference. Any single function might look reasonable, but combined, they result in a lifecycle error.

The potential value of LLMs lies in assisting in establishing these cross-location associations:

• What transformations external inputs undergo;
• Where permission checks occur;
• Who creates, holds, and releases an object;
• Whether exception branches compromise state consistency;
• Whether concurrent operations rely on execution orders that are not explicitly guaranteed.

But “being able to describe a risk” does not mean “having proven a vulnerability.” Every inference must still be anchored to specific code locations, necessary conditions, and observable outcomes.

2. Constraining the Analysis Process with Security Invariants

Rather than directly asking the model to “find all vulnerabilities,” a more effective approach is to have it analyze around security invariants.

Security invariants are conditions that must be continuously met during legitimate system execution. For example:

• Unauthorized requests cannot modify protected states;
• Resources should not continue to be accessed via old references after release;
• Length fields must remain consistent with the actual accessible data range;
• Error handling must not skip necessary cleanup or validation;
• Data crossing isolation boundaries must undergo necessary validation.

The benefit of this approach is that audit conclusions no longer stop at “this code looks dangerous,” but can explain: which constraint was violated, under what conditions it was violated, and why existing checks are insufficient to prevent the issue.

3. Correcting Judgments through Tool Feedback

Models cannot determine runtime results solely through text reasoning. Compilation feedback, test results, runtime logs, and anomaly diagnostics are all crucial evidence for validating hypotheses.

A reasonable analysis loop should be:

Propose hypothesis → Extract relevant code → Construct controlled test → Observe results → Revise hypothesis → Form evidence.

The most important aspect is not how many loops were executed, but whether each loop reduced uncertainty. Repeatedly generating similar descriptions or submitting identical inputs does not automatically improve research quality.

3. Multi-Agent Collaboration: The Key is Role and Handoff, Not Quantity

Dividing complex tasks among multiple agents can expand coverage but also increases the risk of redundant work, context conflicts, and error propagation. Therefore, multi-agent systems require clear role boundaries.

1. Asset Inventory Role: Defining the Research Scope

This role is responsible for organizing module structures, external interfaces, permission boundaries, and key dependencies, while flagging high-risk code areas.

For example, parsing paths handling untrusted data, cross-permission calling interfaces, and modules involving complex object lifecycles usually warrant priority inspection.

Asset inventory here should be confined to authorized code and test environments; automatically probing real external systems should not be a default step.

2. Defect Analysis Role: Proposing Falsifiable Candidate Issues

Analysis results should at least include:

• Corresponding code version and location;
• Potentially violated security invariants;
• Prerequisites for triggering;
• Known protective measures;
• Current evidence and unconfirmed hypotheses.

Requiring the model to actively list “under what conditions this issue does not constitute a vulnerability” helps reduce one-way inferences and over-interpretation.

3. Verification Role: Prioritizing Negation over Affirmation

The verification role should not merely supplement reasoning for the analysis role; it must independently check if the issue holds.

It needs to confirm whether the test reached the target path, whether the anomaly was triggered by the candidate defect, and whether the result is merely a configuration error, test framework issue, or expected defensive termination.

For high-risk boundary issues, minimal-impact verification methods should be prioritized over pursuing a complete attack chain. The goal of verification is to prove a security constraint was violated, not to expand the actual impact.

4. Remediation and Review Role: Closing the Loop

After the remediation role proposes a patch, a relatively independent review process must check:

• Whether the patch fixes the root cause;
• Whether it only covers a single test input;
• Whether it changes interface semantics;
• Whether it introduces resource leaks, deadlocks, or performance degradation;
• Whether adjacent code paths have similar issues.

Roles can be handled by different agents or jointly by models and humans. What determines effectiveness is evidence quality and review independence, not the number of agents.

4. Establishing an Evidence Chain: Alerts, Crashes, and Vulnerabilities Must Be Counted Separately

The most common statistical bias in automated research is accumulating outputs from different stages into a single “vulnerability count.”

A more rational approach is to establish a tiered funnel:

Tier 1: Candidate Alerts

The system suspects a potential security issue. At this stage, higher coverage can be pursued, but a certain proportion of invalid leads must be accepted.

Tier 2: Reproducible Anomalies

Anomalies that can be stably observed under explicit environments and inputs. Anomalies might manifest as process exits, assertion failures, state inconsistencies, or access violations.

Note that anomalies might just be robustness issues or normal functioning of protective mechanisms, and should not be directly labeled as critical vulnerabilities.

Tier 3: Independent Defects

After root cause analysis and deduplication, defects confirmed to be distinct code errors. The same defect might produce different stack traces, error messages, and triggering inputs, so counting solely by anomaly signatures is incorrect.

Tier 4: Confirmed Security Vulnerabilities

Defects that can compromise explicit security properties, such as crossing permission boundaries, leaking protected information, or causing unintended state modifications.

Risk levels must also factor in attacker privileges, reachability, environment configurations, and impact scope.

Tier 5: Fixed and Verified Issues

Patches that have passed retesting and regression and are incorporated into the normal delivery pipeline. This tier reflects the actual security benefits gained by the enterprise.

It is recommended to maintain a unified record for each issue, including code version, environment summary, minimal trigger conditions, observed results, root cause explanation, deduplication relationships, risk justification, and patch status. Records should avoid containing real credentials, customer data, and unnecessary sensitive information.

5. How to Design a Credible Capability Evaluation

1. Fixing Experimental Conditions

When comparing different approaches, keep code versions, computing resources, time budgets, tool permissions, and human intervention levels as consistent as possible.

If one approach runs more tests, accesses more source code, or receives more prompts, its results cannot be simply attributed to the model being inherently stronger.

2. Controlling Historical Information Interference

Models may have already been exposed to public vulnerability descriptions or remediation discussions. Therefore, “successfully identifying known issues” and “discovering previously unknown vulnerabilities” should be reported separately.

Evaluations can use time isolation, hidden remediation information, and manual spot checks to reduce interference, but should not claim to completely eliminate the influence of the model’s existing knowledge.

3. Reporting Effectiveness and Workload Simultaneously

Recommended focus areas:

• Confirmation precision rate: The proportion of confirmed independent vulnerabilities among candidate issues;
• Effective reproduction rate: The proportion of successfully independently retested issues among pending retests;
• Cost per valid output: Total investment divided by the number of confirmed independent vulnerabilities;
• Manual review time: Hours researchers invest in filtering, deduplication, and confirmation;
• Remediation closure rate: The proportion of issues that have been fixed and verified;
• Severity distribution: The composition of issues across different impact levels.

Real codebases usually do not have a total vulnerability count, making it difficult to accurately calculate recall. If using test sets with artificially injected defects, their representativeness limitations must be clarified and not directly extrapolated to all real-world projects.

6. Cost Accounting: Cheap Alerts Do Not Necessarily Bring Cheap Security

The cost of an automated system includes not only model API call fees but also test computing, environment maintenance, storage, manual review, and patch verification.

The following formula can be used:

Total Cost = Inference and Computing Cost + Environment Maintenance Cost + Manual Review Cost + Remediation and Regression Cost.

Unit Effective Cost = Total Cost ÷ Number of Confirmed and Deduplicated Vulnerabilities.

To evaluate R&D ROI, further observation is needed on whether issues enter the remediation process and whether fixes actually reduce the real attack surface.

One cannot simply multiply the number of potential vulnerabilities by a bounty ceiling to derive a so-called guaranteed return. Whether a vulnerability meets reward rules, is a duplicate, or whether the impact is valid will all change the actual outcome.

Furthermore, a higher volume of API calls is not always better. When the system continuously generates duplicate candidates, lacks new verification evidence, or manual review backlogs exceed processing capacity, tasks should be paused and strategies adjusted.

7. Isolation and Permission Governance: Stronger Capabilities Require Clearer Boundaries

Agents capable of reading code, executing tests, and modifying files inherently possess operational capabilities. Their security governance cannot rely solely on prompts.

1. Adopting Tiered Permissions

The process can be divided into stages such as read-only analysis, restricted testing, patch generation, and release approval. Each stage should only be granted the minimum permissions required to complete the task, and release and production changes must retain independent approval.

2. Default Isolated Execution

Tests should be conducted in resettable, independent environments using desensitized or synthetic data, with restricted external network access. For research that might affect underlying systems, isolation boundaries must be provided by mechanisms independent of the object under test, and the boundary being verified cannot serve as the sole line of defense.

3. Treating Repository Contents as Untrusted Input

Inductive content can appear in source code comments, documentation, and build scripts. Agents must not bypass permission policies, send data, or execute extra operations simply because they read an “instruction” in a file.

Task rules and materials to be analyzed must be separated, and high-risk tool calls must pass independent policy checks.

4. Retaining Audit and Halt Mechanisms

The system should log task scopes, key judgments, tool calls, file modifications, and approval results. When encountering out-of-bounds access, anomalous data transfers, runaway resource consumption, or isolation environment anomalies, it should be able to promptly halt execution and preserve the scene.

When dealing with public infrastructure controls or other high-impact systems, research should be confined to specially designed simulation or experimental environments and should not conduct autonomous probing in real production control chains.

8. Enterprise Implementation: Gradually Moving from Assisted Auditing to a Closed Loop

Phase 1: Start with read-only code auditing, allowing the model to assist in mapping sensitive paths and security invariants, with humans judging the quality of leads.

Phase 2: Introduce isolated verification, linking static analysis, dynamic testing, and manual review, establishing unified issue recording and deduplication rules.

Phase 3: Allow the model to participate in patch suggestions and regression test case generation, but maintain independent review to prevent “passing current tests” from being mistaken for “having fixed the root cause.”

Phase 4: Scale up based on accumulated data. The basis for expansion should be the proportion of valid issues, review burden, remediation quality, and operational risk, rather than the maximum output from a single demo.

Conclusion

The value of security LLMs lies not in generating surprising attack descriptions, but in transforming risks in complex code into verifiable, fixable, and sustainably manageable issues.

A mature solution must connect code understanding, controlled verification, root cause deduplication, patch regression, and permission governance. Only when the discovery process is explainable, the verification process is repeatable, the execution process is constrainable, and the remediation results are confirmable, can automated vulnerability research truly become an asset for R&D security rather than a new source of risk.