Cybersecurity Agent Evaluation: From Static Q&A to Dynamic Adversarial Games

    This article explores the evolution of cybersecurity agent evaluation from static question-and-answer formats to dynamic adversarial games. It highlights why traditional static evaluations often overestimate capabilities and introduces a bidirectional adversarial framework involving attackers, defenders, and referees. The article details the framework's design, key metrics, reasons for performance stratification in adversarial settings, and practical engineering considerations for building a sustainable evaluation platform.

    Cybersecurity Agent Evaluation: From Static Q&A to Dynamic Adversarial Games

    When security capability evaluation shifts from “question-answer” to a multi-round closed loop of “attacker-defender-referee,” the meaning of the leaderboard changes. It is no longer just a ranking of knowledge coverage, but a comprehensive test of strategy planning, tool invocation, long-term memory, feedback utilization, and cost control. Adversarial evaluations based on hundreds of valid matches reveal a clear stratification between top-tier agents and other participants. More notably, many attack paths that seem stable in static single-player modes experience a significant drop in capture rates within a few rounds once they enter bidirectional adversarial settings, as defenders continuously patch based on logs and alerts. This indicates: The true capability of a security agent must be measured in dynamic adversarial games.

    1. The Core Problem: Security Capability is Not “Knowing the Answer,” but “Closing the Loop Under Opponent Feedback”

    Traditional security knowledge evaluations typically use fixed question banks: given a prompt, asset information, and a target, the model is asked to output an analytical conclusion, exploitation idea, or remediation advice. While this is suitable for measuring conceptual understanding, common vulnerability identification, and tool proficiency, it fails to answer a more critical question: When the defender is also learning, patching, and deploying defenses, can the attacking agent still maintain effective action?

    Real-world offense and defense have several characteristics that static evaluations struggle to cover:

    1. Opponents alter the environment: Defenders patch, adjust rules, tighten permissions, or isolate based on exposed anomalous logs, alerts, and attack chains.
    2. Incomplete information: The attacker cannot see hidden assets, detection rules, and defense strategies; the defender also cannot see the attacker’s complete plan.
    3. Cost and time constraints: Real actions are limited by time windows, invocation limits, permission boundaries, and risk control, preventing infinite retries.
    4. Process matters more than the final answer: Whether false positives are triggered, whether detectable traces are left, and whether the path can be switched after failure all affect actual effectiveness.
    5. Scoring has adversarial nature: Optimizing only for a single final result easily induces “reward hacking,” such as repeatedly trying fixed patterns, bypassing the evaluator, or exploiting flaws in the simulated environment.

    Therefore, a more reasonable evaluation method is to construct a controlled bidirectional adversarial environment: a red-team agent attempts to achieve abstract goals, a blue-team agent detects, blocks, and repairs based on observable information, and a referee executes actions, maintains state, and records the entire process. The evaluation target is no longer just a “question-answering model,” but an “agent system acting within a feedback loop.”

    2. Why Static Evaluations Tend to Be Systematically Optimistic

    High scores in static single-player modes are not without value, but they often overestimate attack capabilities in dynamic environments. The reasons usually fall into the following categories.

    2.1 Unguarded Targets Lack Real Resistance

    In single-player mode, the environment usually does not proactively patch vulnerabilities or adjust strategies based on attack behavior. As long as the model can identify a path, it may repeatedly attempt it until success. In adversarial mode, however, the first successful attempt reveals information; once the defender patches based on this, the original path may immediately become invalid.

    2.2 Allowing Retries Masks Stability Issues

    Scoring methods like “two chances per question” or “taking the best result after multiple attempts” weaken the requirement for stability. Real security operations care more about the one-time success rate within a limited budget, average detection time, and recovery ability after failure. The more retries allowed, the easier it is for static scores to favor “knowing how to search for answers” rather than “knowing how to execute stably.”

    2.3 Fixed Environments Encourage Pattern Matching

    Fixed question banks, fixed asset topologies, and fixed log formats are easily overcome by models through superficial pattern matching. Adversarial evaluations should introduce asset randomization, log noise, topology changes, vulnerability combination variations, and hidden test sets to reduce the benefits of “memorizing answers.”

    2.4 Focusing Only on Final Results Ignores Action Trajectories

    The risks of security actions are often reflected in the process: whether a large number of anomalous requests are generated, whether alerts are triggered, whether business availability is disrupted, and whether persistence traces are left. If we only look at whether the final goal is achieved, important dimensions like “stealth,” “controllability,” and “recoverability” are ignored.

    2.5 Static Questions Provide Abundant Context

    Static prompts often present key clues directly to the model; in dynamic games, agents need to actively probe, filter logs, validate hypotheses, and make probabilistic judgments with incomplete information. The capabilities tested by the two are not the same.

    In summary: static evaluations favor “knowledge recall and single-step tool invocation,” while dynamic adversarial evaluations are closer to “long-term planning, counterfactual reasoning, strategy switching, and cost trade-offs.”

    3. Designing a Bidirectional Adversarial Evaluation Framework

    A reproducible security agent adversarial evaluation should include at least six components: roles, environment, rounds, observations, actions, and scoring.

    3.1 Role and Permission Separation

    • Attack Agent: Attempts to achieve abstract goals within an authorized sandbox, such as obtaining a specific flag, altering a controlled state, or completing a preset attack chain.
    • Defense Agent: Reads logs, alerts, asset inventories, and state snapshots, executing detection, blocking, patching, permission tightening, and validation actions.
    • Referee and Environment: Responsible for executing actions, validating legality, maintaining state, generating observable logs, and preventing one party from reading the other’s hidden state.
    • Scoring Service: Independent of the participating agents, calculates scores based on predefined rules to prevent tampering by the evaluated objects.

    Permission boundaries must be clear: the attacker cannot directly modify the referee’s state, the defender cannot read the attacker’s complete plan, and both can only act through restricted tools provided by the environment.

    3.2 Environment State and Round Mechanism

    A common workflow is: initialize the scene -> attacker acts -> referee executes and records -> defender observes and repairs -> proceed to the next round. Parallel rounds can also be designed, but rules for information delay and action conflict resolution must be clarified. Each round should have a budget, such as maximum action steps, maximum tool invocations, maximum time, and resource quotas, to avoid infinite trial and error.

    3.3 Scenario Randomization and Hidden Test Sets

    The evaluation set should be divided into public, validation, and hidden sets. The public set is for debugging, and the hidden set is for final ranking. Scenario generation should randomize asset naming, port exposure, log noise, alert thresholds, vulnerability combinations, and business dependencies to prevent the model from overfitting to fixed samples.

    3.4 Scoring Should Not Rely Solely on Win Rate

    It is recommended to adopt a multi-metric scoring system rather than a simple win/loss. Attack capture rate, defense success rate, detection latency, remediation latency, false positive rate, resource cost, and stability should all be included. For leaderboards, match points or an Elo rating system can be used, but confidence intervals, sample sizes, and evaluation conditions must be published simultaneously.

    4. Key Metrics: From “Can It Be Done?” to “How Fast, How Stable, How Cost-Effective?”

    1. Attack Capture Rate: The proportion of abstract goals achieved within limited steps and costs.
    2. Defense Success Rate: The proportion of attacks blocked, vulnerabilities patched, or risks controlled within an acceptable range.
    3. Multi-round Decay Rate: Compares the capture rate of the first round with subsequent rounds, measuring the decay speed of the attacker after being adapted to by the defender.
    4. Average Detection Time and Average Remediation Time: Reflects the blue team’s efficiency from observation to action.
    5. False Positive Rate and False Negative Rate: In security operations, over-alerting consumes significant manpower and must be included in the score.
    6. Resource Cost: Inference steps, tool invocation count, time budget, and compute resource consumption.
    7. Stability and Transferability: Variance under multiple random seeds in the same scenario, and performance on hidden sets.
    8. Explainability and Auditability: Whether key decisions leave reviewable evidence and are convenient for post-incident review.

    Among these metrics, the multi-round decay rate deserves special attention. It directly answers a question: when the defender starts patching, does the attacking agent continue to find new paths, or does it get stuck repeating old ones? The latter might score high in static questions but will quickly fail in dynamic games.

    5. Why Clear Stratification Occurs in Adversarial Settings

    In hundreds of valid matches, the stratification between top-tier agents and other participants is not difficult to understand. Strong agents usually possess more complete attack chain planning capabilities: first gathering information, then validating hypotheses, switching paths based on feedback upon failure, and selecting higher-yield actions under cost constraints. Weaker agents tend to remain in single-step modes, repeatedly trying old paths after encountering patches, or being misled by noisy logs.

    Defenders also gain advantages. As rounds progress, the blue team obtains more observable information, allowing them to prioritize patching high-value vulnerabilities, adjusting detection rules, and validating patch effects. The attacker, on the other hand, must explore new paths in a shorter time, and the information disadvantage gradually amplifies. Thus, a “surefire” attack in a static single-player mode might fail in a dynamic adversarial setting due to a single exposure, a single patch, or a single misjudgment.

    This also reveals an engineering fact: The capability of a security agent is not a single model capability, but a system capability encompassing the model, tools, memory, planner, and evaluation environment. Merely replacing the underlying model without improving tool abstraction, state management, and feedback mechanisms often fails to reproduce the gaps seen on leaderboards.

    6. Engineering Implementation: Building a Sustainable Adversarial Evaluation Platform

    For practical implementation, consider the following practices.

    • Isolated Sandbox: Use containers or virtual machine snapshots to build disposable environments, prohibiting access to the real internet and production data.
    • Least Privilege: Attackers and defenders only receive the tool permissions necessary to complete their tasks; all actions are verified by a whitelist.
    • Observability: Standardize structured logs, alerts, audit records, and state snapshots to ensure every round of decision-making is replayable.
    • Data Masking: Evaluation data, asset names, and business information should be masked to avoid leaking real environments.
    • Independent Referee: Decouple scoring logic from the evaluated agents to prevent scoring manipulation through anomalous outputs.
    • Regression Testing: After every model or toolchain update, rerun using a combination of fixed and random seeds to track score drift.
    • Manual Review: Conduct sampling audits on high-risk actions, anomalously high scores, and suspicious trajectories to prevent reward hacking.
    • Security Boundaries: All attack behaviors are strictly limited to the authorized sandbox and do not target real systems; evaluation reports should specify conditions and limitations.

    Below is an abstract pseudocode for adversarial evaluation, representing only the process without any real attack implementation:

    1. for round_id in range(1, max_rounds + 1):
    2. state = env.snapshot()
    3. attack_plan = attacker.plan(
    4. goal=task_goal,
    5. observation=env.attack_observation(state),
    6. memory=attacker.memory
    7. )
    8. attack_actions = referee.validate(attack_plan, role="attacker")
    9. attack_result = env.apply(attack_actions)
    10. logs = env.collect_logs()
    11. alerts = env.collect_alerts()
    12. defense_plan = defender.plan(
    13. logs=logs,
    14. alerts=alerts,
    15. asset_view=env.defense_observation(state),
    16. memory=defender.memory
    17. )
    18. defense_actions = referee.validate(defense_plan, role="defender")
    19. defense_result = env.apply(defense_actions)
    20. score.update(
    21. attack_success=attack_result.goal_reached,
    22. defense_effective=defense_result.risk_reduced,
    23. detect_time=defense_result.detect_time,
    24. patch_time=defense_result.patch_time,
    25. cost=attack_result.cost + defense_result.cost,
    26. round_id=round_id
    27. )
    28. if referee.terminated():
    29. break

    The focus of this pseudocode is not the specific interfaces, but the evaluation closed loop: observation, planning, validation, execution, recording, scoring, and moving to the next round. Only by incorporating trajectories and states into the evaluation can we distinguish between “accidental success” and “stable capability.”

    7. Limitations and Risks: Do Not Treat Evaluation Scores as Absolute Conclusions

    Bidirectional adversarial evaluations also have boundaries. Simulated environments may simplify real systems; scenario generators themselves may have biases; hidden test sets may also be indirectly overfitted; and the choice of scoring metrics will affect rankings. Therefore, evaluation results should be stated as “relative performance under specific tasks, specific budgets, and specific environments,” rather than absolute promises of real security capabilities.

    At the same time, agent adversarial evaluations must adhere to principles of authorization, isolation, data masking, and auditing. Attack agents should not be run in real production systems, nor should reusable real attack details be publicly disclosed. The value of evaluation lies in discovering capability boundaries and risk exposures, not in creating new risks.

    8. Conclusion

    Moving from static Q&A to dynamic adversarial games is a necessary path for the maturation of security agent evaluation. Static question banks remain suitable as knowledge baselines but cannot replace capability measurement in adversarial environments; a single leaderboard can provide a reference, but continuous, reproducible, and auditable adversarial evaluations are more instructive for engineering decisions. What truly deserves attention is not just whether an agent can solve a problem, but whether it can stably plan, act, review, and improve under conditions where defenders are constantly patching, the environment is continuously changing, and costs are constrained. Whoever can engineer this evaluation closed loop will have a better chance of bringing security agents from demonstrations into trusted practice.