AI in Cybersecurity: The Dual Challenge of Vulnerability Discovery and Defense

AI Is Becoming Part of Core Cybersecurity Research

Cybersecurity research is not simply a matter of searching code for suspicious strings. It is an end-to-end process involving understanding, validation, and remediation. Researchers must understand how software operates, identify code paths that may violate security assumptions, determine whether a flaw can be triggered in realistic conditions, and propose practical mitigations or fixes.

As large language models and automated analysis improve, AI is moving beyond code-reading assistance into multiple stages of the security research workflow.

This capability growth has two implications. On one hand, automated analysis can help researchers navigate large codebases, construct test inputs, reproduce issues, and evaluate patches more efficiently. On the other hand, without authorization and appropriate constraints, the same technologies could reduce the effort required for attackers to discover and exploit vulnerabilities.

Security models should therefore not be evaluated only by whether they can complete a task. Evaluation must also consider the conditions under which they operate, whether they follow scope restrictions, and how their outputs are reviewed and managed.

From Identifying a Flaw to Validating Exploitability

Vulnerability analysis generally consists of several connected stages.

1. Understanding the Program and Its Attack Surface

A model must combine source code, configuration, dependencies, and runtime behavior to identify how external input enters a system and which processing paths it follows.

For example, network requests, file contents, or user-submitted data may affect parsers, authorization checks, memory management, or database operations. Judging a problem from an isolated code fragment can easily produce false positives. Restoring context and analyzing data flows are therefore essential.

2. Determining Whether an Issue Has Security Impact

Not every software defect is a vulnerability. Whether issues such as out-of-bounds access, missing authorization checks, or insufficient input validation create real risk depends on trigger conditions, execution privileges, attack-surface reachability, and potential impact.

Models can help summarize hypotheses and locate relevant code paths, but conclusions still require testing and human review.

3. Validating in a Controlled Environment

Public evaluations of vulnerability-exploitation capabilities indicate that newer security models achieved a 48.5% success rate on designated tasks, compared with a 32.1% baseline for earlier models. In testing scenarios without source-code access, where models interacted only with applications, vulnerability recall reached 40.3%, compared with 32.0% for the control model.

These figures reflect results from specific benchmarks, task configurations, and experimental environments. They should not be treated as direct measures of vulnerability discovery rates or attack success rates in real production systems.

The validation stage requires particularly strict boundaries. Test targets should be explicitly authorized, environments should be isolated from production systems, and network egress, data access, and persistent operations should be restricted. Execution records should be retained after testing to ensure that validation activities do not affect real users, business data, or third-party systems.

4. Supporting Remediation and Regression Testing

The objective of security work is not to generate exploit examples, but to reduce risk. Models can help researchers explain root causes, suggest remediation directions, write regression tests, and check whether a patch introduces new edge-case issues.

However, remediation recommendations must go through code review and testing. A patch should not be merged simply because a model has produced a seemingly reasonable fix.

How to Interpret Benchmark Results

Vulnerability-exploitation benchmarks can measure a model’s ability to complete complex tasks under constrained conditions, but benchmark scores have clear limits.

Different test suites may vary in vulnerability categories, target configurations, definitions of success, and resource constraints. Strong performance on one benchmark does not mean that a model can reliably handle every programming language, framework, or deployment environment.

Benchmark tasks also typically provide researchers with predefined targets and execution conditions. Real-world systems have more complex dependencies, access controls, and defensive mechanisms. Models may encounter incomplete information, environmental differences, or ambiguous prompts, leading to false negatives, false positives, or behavior that does not meet expectations.

Evaluation reports should therefore describe task scope, sample composition, success criteria, and failure cases rather than presenting a single percentage in isolation.

For organizations, a more practical approach is to run limited pilots within their own security processes, using authorized code and testing environments. In addition to the number of issues identified, teams should assess the cost of confirming findings, time spent handling false positives, adoption rates for remediation recommendations, and whether the model consistently follows task boundaries.

Governance Requirements for Stronger AI Capabilities

As models become more effective at helping identify and validate vulnerabilities, security teams should establish governance controls before deploying these capabilities rather than adding them afterward.

• Define authorization boundaries: Use written policies to specify which code repositories, systems, and testing environments may be assessed. Unauthorized targets must be excluded.
• Isolate execution environments: Run high-risk analysis in sandboxes or dedicated testing environments, with restrictions on credentials, sensitive data, and external network access.
• Require human review: Apply tiered review to vulnerability assessments, exploit validation, and remediation recommendations. Critical conclusions should be confirmed by personnel with the appropriate responsibilities.
• Maintain audit records: Record the source of each task, input scope, model output, human decisions, and follow-up actions to support accountability and post-incident review.
• Establish a vulnerability disclosure process: When an issue is found, notify the relevant maintainers according to organizational policy and applicable coordinated disclosure practices. Avoid publishing details before mitigations are available.

These controls are not intended to limit security research. They ensure that research remains controlled, traceable, lawful, and within authorized boundaries.

A Practical Adoption Process for Development Teams

Organizations and development teams can introduce AI-assisted security analysis in stages.

First, select authorized code and testing environments with limited impact, and clearly define the task as defensive security assessment only. Next, use the model to map attack surfaces, generate testing ideas, or summarize suspicious paths. Security personnel should then decide whether dynamic validation is appropriate.

Confirmed issues should be converted into defect records that include reproduction conditions, impact analysis, remediation plans, and regression tests. Finally, teams should use code review and continuous testing to confirm that the fix is effective, while documenting the model’s false positives and false negatives for the task.

Throughout this process, the model should be treated as a research tool rather than as the party responsible for security decisions. It can reduce the time required for information gathering and repetitive analysis, but it cannot replace the system owner’s authorization decisions, professional risk assessment, or the maintenance team’s responsibility for remediation.

For high-impact operations in production environments, organizations should apply approval workflows, isolation, and least-privilege principles to prevent automated outputs from directly triggering irreversible changes.

Conclusion

Progress in AI for cybersecurity is extending vulnerability research from code understanding to a degree of validation and remediation support, with the potential to significantly improve security team efficiency.

However, stronger capabilities do not automatically produce stronger security. The same technology can support defense or be misused for attack. A sustainable approach is to combine model capabilities with authorization controls, isolated environments, human review, and responsible disclosure. Within clearly defined boundaries, automation can deliver meaningful value while continuous evaluation and auditing help manage risk.