LLM Evaluation: Practical Methods for Coding and Security Audit Capability

    This article explains how to evaluate large language models for coding, agent execution, and security auditing as related engineering skills. It covers measurable dimensions, internal task-based test sets, long-context recall probes, agent metrics, detection quality versus authorization boundaries, unit cost per successful task, rollout risks, and a staged adoption path.

    1. Why Evaluate Coding and Security Auditing Together

    Once large language models enter the development workflow, the question shifts from “Can the model write code?” to “Can it finish real engineering work reliably without introducing new risk?” Coding, agent execution, and security auditing depend on the same foundations: understanding large codebases, keeping long chains of reasoning consistent, calling tools correctly, and judging whether the result is trustworthy. Evaluating them together avoids treating a single score as the whole story while ignoring engineering cost.

    The evaluation centers on three questions: Can the model complete the task? Is the result reliable? Is the cost acceptable? The rest of this article follows those questions.

    2. Break the Evaluation into Measurable Dimensions

    Before scoring models, split the goal into dimensions that can be measured.

    1. Code generation and modification: add features, fix defects, and refactor existing code, then check that existing tests still pass.
    2. Engineering understanding: locate the relevant modules in a large project and state the correct call relationships and impact scope.
    3. Agent execution: in an isolated environment, call build, test, search, and similar tools step by step to finish multi-step tasks.
    4. Security auditing: identify common vulnerability types in code, separate real risk from false positives, and provide fixes that can be verified.

    Each dimension needs an explicit pass criterion. For defect fixes, the related tests must all pass and other tests must not be broken. For security auditing, the vulnerability must be identified correctly and the fix must pass regression tests.

    3. Build the Test Set from Real Tasks

    Public benchmarks only show how a model performs on generic problems. They do not reflect a team’s code style, stack, or business constraints. A more reliable approach is to sample internal historical work:

    • Historical defects and the commits that fixed them, organized as triples of problem description, initial code, and verification tests.
    • Issues raised in code review, turned into audit samples.
    • Requirements documents with sensitive details removed, used to test the full path from requirement to implementation.

    Every sample should be judged automatically. The structure below is an example evaluation case:

    1. case = {
    2. "id": "bugfix-0142",
    3. "category": "bugfix",
    4. "prompt": "Fix duplicate deduction of the order amount when submissions race",
    5. "repo_snapshot": "snapshots/order-service@a1b2c3",
    6. "verify": ["pytest tests/test_order_concurrency.py"],
    7. "max_steps": 30,
    8. "max_tokens": 200_000,
    9. }

    Samples should cover the difficulty range of each task type, not only easy cases. Aim for at least several dozen samples per category, and refresh them regularly so the evaluation content is not already covered by training data.

    4. Testing Long-Context Capability

    A longer input window does not mean the model uses the whole input. It may receive on the order of a million tokens and still attend mainly to the beginning and the end. Tests should therefore measure recall of critical information.

    A common method is to plant one verifiable fact at different positions in a long text, ask the model about it, and track accuracy by position and length:

    1. def build_probe(filler_docs, fact, position):
    2. """Insert a fact at relative position (0 to 1) to test long-context recall."""
    3. body = "\n\n".join(filler_docs)
    4. cut = int(len(body) * position)
    5. return body[:cut] + "\n" + fact + "\n" + body[cut:]
    6. positions = [0.0, 0.25, 0.5, 0.75, 1.0]

    Summarize results as a table of position, length, and accuracy. That shows where quality drops sharply and informs how finely to split inputs, and which content should stay near the front of the prompt.

    5. Metrics for Agent Tasks

    Agent failures often come from multi-step loops and mistaken actions, not only from a single bad step. Besides whether the task finished, record:

    • Task success rate: the share of tasks that pass verification tests.
    • Average steps: the number of tool calls needed to finish. A persistently high count usually means weak planning.
    • Invalid-action rate: the share of repeated identical commands, edits to unrelated files, attempts to exceed granted access, and similar behavior.
    • Rollback count: how often the model undoes its own incorrect changes, which reflects self-checking.

    The execution environment must be isolated. The agent may access only the evaluation copy, must not see production credentials, and may write files only inside the designated working directory.

    6. Security Auditing: Detection Quality and Behavioral Boundaries

    Security-audit evaluation has two directions.

    The first is detection quality. On code samples within an authorized scope, report both recall and precision. Low recall means missed issues. Low precision means false positives that consume reviewer time. Reporting only one of the two gives a one-sided result.

    The second is behavioral boundaries. Audit output can support defense and can also be misused. During evaluation, check whether the model, without a statement of authorization, produces attack payloads that can be used directly, or probes systems unrelated to the audit task. A sound practice is to require every audit request to state its authorized scope, refuse requests outside that scope, and handle similar requests consistently.

    7. Accounting for Cost and Efficiency

    Comparing the price of a single call understates real cost. A better metric is the full cost of one successful task:

    Unit success cost = total spend / number of tasks that pass verification

    Total spend includes input and output tokens, retries after failure, and a conversion of human review time. A simple aggregation is:

    1. def unit_success_cost(records):
    2. total_cost = sum(r["cost"] for r in records)
    3. passed = sum(1 for r in records if r["passed"])
    4. return total_cost / passed if passed else float("inf")

    Model A may succeed more often but cost more per run. Model B may succeed slightly less often but need fewer retries. Their unit success costs can be close, or the ranking can reverse. Selection should follow this metric, not success rate or unit price alone.

    8. Rollout Risks and Governance

    Passing an evaluation still leaves engineering risks to handle:

    1. Data security: redact code and logs before they are sent to the model. Keys and credentials must not enter prompts.
    2. Access control: give the agent a least-privilege service account. Writes require human confirmation.
    3. Quality backstop: model-generated changes must go through normal code review and automated tests. They must not be merged directly.
    4. Compliance review: for open or otherwise obtainable models, read the license terms carefully. Being able to obtain a model does not mean it may be used commercially without restriction. Legal review should confirm the conclusion.
    5. Traceability: record prompt version, model version, and outputs so later review and audit are possible.

    9. Implementation Path

    A four-stage rollout is a practical sequence:

    • Offline evaluation: run a baseline on the internal test set and set acceptable thresholds for success rate and cost.
    • Shadow mode: the model only proposes changes and does not apply them. Measure how often suggestions are accepted.
    • Limited rollout: start with lower-risk work, such as test code or documentation.
    • Broader adoption: expand only after the metrics are stable, and keep a rollback path.

    Each stage needs a clear exit condition. If the metrics are not met, stay in the current stage rather than widening use.

    Checklist

    • Does the test set come from real tasks, and is it updated regularly?
    • Do long-context tests cover different positions and lengths?
    • Does the agent run in an isolated environment, with invalid actions recorded?
    • Does security auditing report both recall and precision, and check authorization boundaries?
    • Is cost measured as unit success cost?
    • Have data security, access control, and compliance review been completed?