Cybersecurity Enters the Era of Dynamic Attack-and-Defense Agent Evaluation

    The article explores the transition of AI cybersecurity evaluation from static testing to dynamic adversarial environments. It explains why models excelling in static tests often fail in real-world continuous attack and defense tasks, introduces a five-module architecture for deployable security agents, and provides recommended evaluation metrics along with a phased implementation roadmap for enterprises.

    In recent years, artificial intelligence has continuously improved its performance in tasks such as code generation, vulnerability analysis, and security operations. Many models can now provide exploitation ideas based on vulnerability descriptions, locate anomalous behaviors from logs, and even generate remediation patches. However, a concerning issue remains: performing well in relatively static and well-defined test questions does not mean a model can sustainably execute attack or defense tasks in a real network environment.

    Real security confrontation is not a “one-time quiz.” Attackers need to reconnoiter targets, identify entry points, construct exploit chains, and maintain access. Defenders must identify anomalies, patch vulnerabilities, update rules, and verify fixes without disrupting services. Both sides observe each other’s behavior and continuously adjust strategies based on environmental changes. Therefore, evaluating a security agent cannot rely solely on single-task success rates; it requires observing its decision-making quality, adaptation speed, and system stability in dynamic gameplay.

    Shifting from Static Quizzes to Dynamic Confrontation

    An evaluation method closer to real-world scenarios involves multiple agents acting simultaneously as attackers and defenders in an isolated environment. Each agent must attempt to breach the opponent’s exposed services while maintaining the normal operation of its own. The evaluation system typically provides several independent network environments. Each match consists of multiple rounds, where the attacker can use information from previous rounds to continue operations, and the defender can perform fixes based on alerts, logs, and anomalies.

    This design introduces three critical variables.

    First, the environmental state changes. Vulnerabilities may be patched, services may restart, and access control policies may be updated. An attack path that was effective previously may not work in the next round.

    Second, actions have costs. Scanning, brute-forcing, frequent testing, and large-scale configuration changes can expose intentions or even render one’s own services unavailable. Agents must make trade-offs between exploration and stealth.

    Third, the objectives of both sides are not singular. Attackers seek to gain permissions, read sensitive data, or control services, but cannot ignore the availability of their own systems. Defenders must not only patch vulnerabilities but also avoid blocking normal traffic, mistakenly modifying configurations, or triggering business interruptions.

    Therefore, this type of evaluation truly examines the “ability to continuously achieve goals,” rather than whether a single output is correct.

    Stratification Reflected in Multi-Model Matches

    In a multi-agent attack-and-defense evaluation co-organized by multiple research institutions, researchers combined different models with execution frameworks for testing. After hundreds of valid matches, the participating systems exhibited a clear capability stratification. The top two systems gained an advantage in overall scores and win rates, while other systems, though capable of solving some basic tasks, showed significant instability in continuous rounds and environments with frequent strategy changes.

    Such results indicate that a model’s comprehensive ability is not entirely equivalent to its security combat capability. A model might excel at explaining vulnerability principles but struggle to determine which path is most valuable in the current environment. It might generate correct commands but fail to confirm whether the command actually took effect. Some systems can identify problems but cannot coordinate subsequent verification, rollback, and service recovery.

    From an engineering perspective, the final score is often determined by multiple stages:

    1. Environment Understanding
      Agents need to identify hosts, ports, service types, permission boundaries, and data flows to build an updatable environment model.

    2. Action Planning
      Attack or defense actions are rarely isolated. They require sequencing and assessing the success probability and potential risks of different actions.

    3. Feedback Utilization
      Command outputs, log changes, connection results, and service states are all feedback signals. Excellent systems can correct their judgments from feedback rather than repeatedly executing failed actions.

    4. Cross-Step Memory
      In multi-round confrontations, accounts, paths, configurations, and patch information discovered in earlier rounds can affect subsequent decisions. Agents lacking structured memory tend to repeatedly scan or forget crucial context.

    5. Operational Stability
      Security actions must be predicated on service availability. Incorrectly modifying firewalls, permissions, or process configurations may cause the defender to lose points before the attacker does.

    Why Static High Scores Don’t Represent Combat Capability

    One of the most valuable comparisons in evaluation is placing the same security task into both single-machine mode and bidirectional adversarial mode. In a single-machine environment, targets are usually undefended, agents have a fixed number of attempts, and attack paths do not change due to actions. Under these conditions, multiple systems might complete most of the tasks.

    However, when tasks enter a dynamic adversarial environment, the success rate drops significantly. The reason is not necessarily that the model forgot the exploitation method, but that the problem itself has changed.

    In a static environment, the task can be abstracted as:

    Given a known target and vulnerability, find a viable path.

    In a dynamic environment, the task is closer to:

    Under conditions of incomplete information, continuously changing targets, and action side-effects, select the most valuable action currently and quickly adjust after failure.

    The difficulty differences manifest in the following aspects:

    • Defenders will promptly patch public vulnerabilities, rendering historical experience invalid.
    • Service versions, port states, and permission configurations may change dynamically.
    • Attack actions generate logs, and overly aggressive operations may trigger alerts.
    • Defenders must distinguish between real attacks and normal operational behavior.
    • Agents must handle engineering issues such as command execution failures, network latency, and incomplete tool outputs.

    This means security agents cannot rely solely on a “vulnerability knowledge base”; they must also possess state management, risk assessment, and online planning capabilities.

    A Deployable Security Agent Architecture

    From a system implementation perspective, an attack-and-defense agent can be decomposed into five modules.

    1. Observation Module

    Responsible for collecting network scan results, service states, system logs, application logs, process information, and alert events. Observation results should not be directly fed to the model as large amounts of raw text; instead, they should be structurally processed, for example, converted into:

    1. Asset: Application Server
    2. Service: HTTP
    3. Status: Running
    4. Anomaly: Multiple unauthorized accesses occurred
    5. Recent Changes: Access control configuration updated
    6. Confidence: Medium

    Structured observation helps reduce context noise and facilitates subsequent state comparisons.

    2. State and Memory Module

    The system needs to store asset inventories, verified facts, failed actions, hypotheses to verify, and current objectives. For example:

    1. Confirmed:
    2. - The target has a certain input validation flaw
    3. - Low-privilege accounts can access the service
    4. - The defender has modified related configurations
    5. To be confirmed:
    6. - Whether the fix covers all entry points
    7. - Whether alternative access paths still exist

    Separating “facts” from “speculations” is crucial. Otherwise, the model may treat an unverified guess as a definite conclusion, leading to an accumulation of erroneous decisions.

    3. Planning Module

    The planner breaks down long-term goals into several short-term actions, calculating the benefit, cost, and risk for each. Attackers might choose between information gathering, entry validation, and privilege escalation; defenders need to sequence alert confirmation, impact analysis, temporary mitigation, and permanent fixes.

    In practice, systems should not let the model generate overly long plans at once. A more robust approach is a “short plan, strong feedback” mechanism: execute only a limited number of actions each time, and re-plan after obtaining the new state.

    4. Execution and Safety Control Module

    Commands generated by the model should not be executed unconditionally; they must undergo policy checks, permission isolation, timeout control, and result validation. Operations that might damage the system require manual approval or automatic blocking rules.

    The execution environment should also feature:

    • Least privilege accounts;
    • Network access whitelists;
    • File system and process isolation;
    • Command timeouts and resource quotas;
    • Full operation audits;
    • One-click rollback and snapshot recovery.

    These measures not only prevent the agent from losing control but also facilitate analyzing why a match succeeded or failed.

    5. Evaluation Module

    The evaluation module needs to record the time, target, result, impact, and subsequent changes of each action, converting them into comparable metrics. Only by saving complete trajectories can we distinguish between issues like “lack of knowledge,” “planning errors,” “execution failures,” and “environmental changes.”

    A single win rate is insufficient to fully describe an agent’s capability. Enterprises should establish a multi-dimensional metric system.

    Attack Success Rate: Measures whether the target was breached, but should further distinguish between stages such as low-privilege access, privilege escalation, sensitive data acquisition, and persistent control.

    Defense Repair Rate: Tracks whether vulnerabilities were effectively fixed after discovery, and whether regression validation was performed after the fix.

    Average Response Time: The time required from anomaly appearance to completion of identification, handling, and verification.

    Service Availability: Records request success rates, latency, and error rates during security operations, preventing “shutting down the service” from being misjudged as a successful defense.

    Action Efficiency: The proportion of effective actions to total actions, identifying invalid scans, repeated attempts, and high-risk operations.

    Adaptability: Whether the system can quickly adjust when environment configurations, vulnerability states, or defense strategies change.

    False Positive and Mis-repair Rate: Whether the defense agent frequently blocks normal behaviors or modifies critical configurations due to incorrect judgments.

    Additionally, confidence intervals, different random seeds, and multi-round repeated experiments should be used to avoid accidental conclusions from a small number of matches. For system rankings, instead of just publishing a total score, it is best to report capabilities and failure reasons at each stage simultaneously.

    Enterprise Implementation Path

    If an enterprise wishes to introduce security agents, it is not recommended to start directly from the production network. A phased approach should be adopted.

    Phase 1: Establish an isolated simulation environment covering common services, identity permissions, log pipelines, and configuration management processes.

    Phase 2: Have the agent execute read-only tasks first, such as asset identification, log attribution, and risk ranking, to verify its observation and analysis capabilities.

    Phase 3: Introduce low-risk automated actions, such as generating remediation suggestions, creating change drafts, and executing regression checks, while retaining manual approval.

    Phase 4: Open limited automated remediation for clear, rollbackable scenarios, setting strict permission, scope, and time boundaries.

    Phase 5: Use continuous adversarial testing to verify whether the system can adapt to strategy changes, rather than just achieving high scores on fixed samples.

    At the governance level, all agents should have identity tags, permission boundaries, operation audits, and emergency stop mechanisms. When sensitive data is involved, desensitization, minimized collection, and graded authorization should be applied to avoid expanding data exposure for the sake of improving analysis effectiveness.

    Conclusion

    The key breakthrough for cybersecurity agents lies not in whether they can independently solve a single vulnerability puzzle, but in whether they can maintain stable judgment amidst incomplete information, dynamic environments, and continuous confrontation. Static tests can measure basic knowledge, but dynamic attack and defense truly tests planning, memory, execution, feedback, and operational coordination capabilities.

    If traditional security evaluation focuses on “whether the answer is correct,” the agent era should focus more on “whether it can continuously do the right thing amid changes.” This is also the most important evaluation criterion when building automated security systems: not only discovering problems but also understanding the environment, controlling risks, verifying results, and stopping erroneous actions promptly when necessary. Only by combining model capabilities, execution frameworks, isolated environments, and governance mechanisms can security agents gradually evolve from experimental tools into reliable production systems.