Security LLMs in Practice: Why Stability Matters More Than Peak Recall

    A large-scale evaluation of security-focused language models shows that the highest aggregate vulnerability recall does not necessarily translate into reliable production performance. This analysis examines how repeated runs, single-run recall, consistency, cost, false positives, and validation standards should shape the evaluation and deployment of AI for code security.

    In recent years, large language models have entered workflows for code auditing, vulnerability analysis, patch generation, and security operations. Unlike general-purpose question answering, cybersecurity tasks require models to understand complex code context, trace data and control flows, identify subtle defects, and produce technically verifiable conclusions. Comparing conversational quality alone therefore cannot accurately measure a model’s real-world security capabilities.

    Recently, a security technology company published the results of a large-scale evaluation. The research team used approximately 11.7 billion tokens and selected 32 recently disclosed vulnerabilities associated with real-world codebases. Multiple leading models were tested under the same conditions.

    Across three runs, one model identified 28 vulnerabilities, achieving the highest aggregate recall. However, its performance in any individual run was less impressive: it found only 17 cases in the first run, and just 10 vulnerabilities were detected consistently across all three runs.

    This result deserves close attention. It shows that, in cybersecurity, the model with the highest recall is not necessarily the model best suited for production.

    What Did the Evaluation Actually Measure?

    Based on the publicly available information, the evaluation covered several models designed for code reasoning and security analysis. Each model processed 32 real vulnerability cases, with every case executed three times, resulting in 96 total runs. A single task allowed the model to perform up to 30 rounds of exploration, while internet access was disabled.

    The test data was not made up of simple, manually written questions. It consisted of recently disclosed vulnerabilities linked, where possible, to real code repositories. This design has three important advantages.

    First, models had fewer opportunities to reproduce a standard answer directly from their training data. The evaluation focused on whether a model could independently locate a risk by analyzing code, configuration, and call relationships, rather than whether it remembered a published vulnerability description.

    Second, real-world code contains substantial noise. A vulnerability may be distributed across multiple files, functions, or even modules. The model must build a complete contextual picture to determine whether an input is controllable, whether a dangerous operation is reachable, and whether effective protections are present.

    Third, repeated runs make it possible to observe model randomness. Large language models have probabilistic outputs, so the same prompt may produce different results in different runs. Testing only once can easily mistake an unusually strong result for stable capability.

    How Should “Highest Recall” Be Interpreted?

    For vulnerability detection, recall can generally be expressed as:

    1. Recall = Number of correctly identified vulnerabilities / Total number of vulnerabilities in the test set

    If the test set contains 32 vulnerabilities and a model identifies 28 of them, its aggregate recall is approximately 87.5%.

    However, this figure must be interpreted in the context of the evaluation method. The reported result used an aggregate across three runs. In other words, a vulnerability counted as detected if the model correctly identified it in at least one of the three runs.

    This metric helps answer one question: how many risks can the model potentially cover after multiple attempts?

    It does not directly answer a more important production question: can the model identify the same vulnerability reliably every time?

    At least three metrics should therefore be considered together:

    1. Single-run recall
      This measures the model’s actual detection capability in one execution and is more relevant to automated scanning or interactive code review.

    2. Aggregate recall
      This measures the potential coverage achieved across multiple runs and helps determine whether repeated analysis could be worthwhile.

    3. Stable hit rate
      This counts the vulnerabilities detected in every run and reflects consistency and reproducibility.

    A model might identify 17, 22, and 24 vulnerabilities across three runs, producing high aggregate coverage. But if the detected sets differ significantly, engineering teams still need additional executions, result merging, and manual review.

    Three Signals Revealed by the Results

    1. High Recall May Come From Multi-Round Exploration

    The highest aggregate result was 28 out of 32, compared with 25 to 26 identified by other evaluated models. This suggests that the leading model can discover risks that are easy to miss during an initial analysis when it is given more time to explore.

    This capability may involve revisiting previously overlooked functions, trying alternative input paths, tracing cross-module calls, or continuing to gather evidence when the initial conclusion is uncertain.

    However, multi-round exploration also increases latency and computational cost. Repeated execution may be reasonable for batch code audits. For online development workflows or high-concurrency security gates, teams must balance latency, resource usage, and the additional coverage gained.

    2. Limited Stability Increases Operational Costs

    Only 10 vulnerabilities were detected in all three runs, indicating significant uncertainty on some cases. This uncertainty is especially important in security workflows.

    Development teams generally expect security detection to be explainable and reproducible. If the same code is classified as high risk today but the issue is not detected in the next run, it becomes difficult to determine whether the code changed or the model simply produced a different probabilistic result.

    Production systems should therefore store more than whether the model generated an alert. They should also retain:

    • The code location identified by the model;
    • The input source that triggered the risk;
    • The propagation path and dangerous operation;
    • The evidence snippets used in the analysis;
    • Confidence and uncertainty information;
    • Differences between repeated runs.

    This information allows security professionals to review and track findings more effectively.

    3. Lower-Cost Models May Offer Better Value

    The evaluation also showed that a lightweight or lower-cost model achieved 25 out of 32 in the formal version—only one fewer case than the leading model—while costing approximately 69.8% less.

    This indicates that model selection should not be based solely on the top position in a leaderboard. For enterprises, a more useful objective is often the number of effective findings per unit cost. One possible comparison is:

    1. Value per unit cost = Number of correctly identified vulnerabilities / Cost per run

    If a model has lower cost and latency while delivering recall close to that of a larger model, it may be better suited for routine scanning. A more capable model can then be reserved for high-risk repositories, complex vulnerabilities, and final review.

    Limitations of This Type of Evaluation

    First, 32 cases remain a limited sample. Vulnerability types vary considerably in difficulty. Memory-safety issues, access-control problems, injection risks, business-logic defects, and configuration errors place different demands on a model. A single test set cannot represent all enterprise codebases.

    Second, disabling internet access reduces the possibility that a model will retrieve a known answer, but it also differs from many production environments. Real security analysis may incorporate vulnerability databases, dependency information, version history, and internal knowledge bases. Offline evaluation is useful for measuring a model’s own code-reasoning capability, but it cannot fully represent the performance of an integrated security platform.

    Third, recall does not capture the false-positive rate. A model that reports large numbers of suspicious code segments may increase its apparent discovery count while creating a heavy burden of invalid alerts for developers. Precision, false-positive rate, the practicality of remediation suggestions, and the time required for human review are equally important.

    Finally, the definition of “vulnerability discovered” must be explicit. A valid hit should identify the correct risk type, the accurate location, and a verifiable explanation. A vague statement such as “this code may contain a security issue” should not be treated as a completed vulnerability analysis.

    How Should Enterprises Build Their Evaluation Framework?

    Organizations adopting security-focused LLMs can use a layered evaluation process.

    The first layer is offline benchmarking. Using historically confirmed vulnerabilities, teams can compare recall, precision, stable hit rate, and average analysis time.

    The second layer is replay testing. The model can be run against historical code versions to determine whether it detects an issue before the vulnerability is fixed and stops reporting the same issue after remediation.

    The third layer is staged deployment. The model initially operates in a non-blocking, side-by-side mode. Teams can then measure the proportion of valid alerts and the rate at which engineers accept or act on the findings in real projects.

    The fourth layer is production gating. Only after the model has demonstrated stable performance over time, with acceptable false-positive rates and latency, should it be considered for automatically blocking high-risk changes.

    From an architecture perspective, a large language model should not independently make every security decision. A more reliable approach combines rule-based scanning, dependency analysis, static analysis, dynamic validation, and model reasoning. Traditional tools can handle high-confidence findings, while LLMs analyze complex context and humans confirm high-risk conclusions.

    How Should This Evaluation Be Judged?

    The most valuable lesson from this evaluation is not that one model will remain permanently ahead. It is that security model assessment has a common trap: the highest aggregated recall does not automatically mean stable, low-cost, production-ready capability.

    A more balanced conclusion is:

    • Large language models already provide meaningful assistance in analyzing vulnerabilities in real-world code;
    • Multiple runs can improve potential coverage;
    • Single-run performance and repeatability remain important weaknesses;
    • Lower-cost models with near-leading performance may be better suited to large-scale deployment;
    • Every leaderboard result should be interpreted alongside the test set, number of runs, evaluation criteria, and false-positive behavior.

    For developers and security teams, the priority should not be chasing a particular leaderboard position. Instead, they should build an evaluation framework aligned with their own codebases, vulnerability types, and delivery processes.

    Only by considering recall, stability, cost, explainability, and human review efficiency together can an organization determine whether a model is merely impressive in a laboratory demonstration or genuinely capable of supporting enterprise security operations.