LLM Agent Evaluation and Selection: A Practical Guide
This guide explains how to evaluate and select LLM agents for production use. It distinguishes agent evaluation from general Q&A benchmarking, defines core capability dimensions and observable metrics, and recommends combining public benchmarks with private task sets. It also presents a selection framework based on capability ceiling, stability, cost per successful task, latency, and controllability, with practical recommendations for coding agents, cost-sensitive batch workloads, and terminal or interactive environments. A step-by-step evaluation workflow and common pitfalls are included.
LLM Agent Evaluation and Selection: A Practical Guide
1. Why Agent Evaluation Is Not the Same as General Q&A Evaluation
LLM agent evaluation measures a model’s overall ability to autonomously decompose a task, select tools, execute multi-step operations, and adjust its strategy based on environment feedback after a goal is given. This differs fundamentally from traditional Q&A evaluation. Q&A evaluation focuses more on knowledge accuracy and language quality in single-turn or short-context settings, while agent evaluation focuses on planning stability, tool-call accuracy, state management, error recovery, and final task success rate in long-horizon tasks.
In real business scenarios, agents often need to invoke general capabilities such as retrieval, databases, code execution, file processing, message queues, and monitoring and alerting. A single task may contain dozens of steps, and any tool-parameter error, missing context, or permission overreach at any step can cause the entire task to fail. Therefore, relying only on general Q&A scores or public leaderboard rankings can easily overestimate a model’s usability in agent scenarios. During selection, metrics such as task success rate, average number of execution steps, retry count, cost per successful task, P95 latency, exception recovery rate, and security interception rate must be included in a unified evaluation.
2. Core Agent Capability Dimensions and Observable Metrics
2.1 Autonomous Planning
Autonomous planning requires a model to break a vague goal into executable subtasks and dynamically adjust during execution based on intermediate results. Evaluation should examine whether subtask decomposition is reasonable, whether无效 loops occur, and whether dependencies are identified. Observable metrics include task completion rate, number of plan revisions, proportion of无效 steps, and average number of steps. A model with strong planning ability is not necessarily the fastest at every step, but it usually reduces directional errors in long-horizon tasks.
2.2 Tool Invocation
Tool invocation is a high-frequency failure point in agent deployment. Evaluation should not only check whether a model “can call” a tool, but also whether parameters are correct, whether the tool schema is followed, whether retries occur after failure, and whether similar tools are distinguished. It is recommended to define clear input and output schemas for each tool and to measure tool-call accuracy, parameter validation pass rate, invalid invocation rate, and retry success rate. Tool description quality itself can significantly affect model performance, so the tool registry version should be fixed during evaluation to avoid misattributing tool design issues to model capability.
2.3 Multi-Step Execution and State Management
Multi-step execution requires a model to maintain task state in a longer context, including completed steps, pending items, intermediate artifacts, and constraints. Evaluation should design tasks that include branches, rollbacks, and concurrent dependencies, and observe whether the model forgets constraints, repeats execution, or omits final steps. Observable metrics include state consistency error rate, recovery rate after context truncation, long-task completion rate, and intermediate artifact completeness rate.
2.4 Error Recovery
Tool failures, timeouts, permission denials, and format exceptions are unavoidable in real environments. A strong agent should identify the error type and choose to retry, switch tools, degrade gracefully, or request human intervention. Evaluation should inject controlled faults and measure recovery rate after the first failure, average recovery steps, and error propagation rate. Error recovery often determines production usability more than single-step capability.
2.5 Security Boundaries and Controllability
Once an agent has execution capability, security boundaries must be established upfront. Evaluation should cover unauthorized tool invocation, sensitive data leakage, confirmation of dangerous operations, prompt injection resistance, and audit traceability. No matter how capable a model is, if it cannot consistently follow permission policies, it should not enter a high-privilege production path.
3. Benchmark Types: Use Public Benchmarks and Private Evaluation Sets Together
Public evaluations generally fall into several categories. General competition-style benchmarks compare models across multi-task, multi-tool environments and are suitable for observing capability ceilings and general trends, but they require attention to leaderboard bias, data contamination, sampling variance, and version drift. Terminal-task benchmarks emphasize command execution, file handling, process management, and environment feedback, making them suitable for evaluating long-horizon execution and error recovery. Interactive operating system or GUI benchmarks examine visual grounding, operation sequences, and fault tolerance, and are closer to desktop automation scenarios. Repository-level tasks focus on cross-file modifications, test feedback, and regression repair, making them suitable for complex agent coding scenarios.
Public benchmarks can only serve as an initial screen. Enterprises should build private evaluation sets that cover real business tools, permission policies, data distributions, and standard operating procedures. Private sets should include simple high-frequency tasks, complex long-horizon tasks, fault-injection tasks, and security adversarial tasks, and should define automatically judgeable success criteria for each task type. Evaluation sets should be versioned, retaining task inputs, tool versions, model configurations, and scoring scripts to ensure reproducibility.
4. Selection Framework: Balance Capability Ceiling, Stability, Cost, and Controllability
4.1 Use Tiered Selection Instead of One Model for Everything
Complex agent coding, cross-system planning, and high-risk decision-making are suitable for high-capability tier models, with deep reasoning or higher inference budget settings enabled to gain stronger planning and error-correction ability. High-frequency, low-risk tool-invocation tasks with fixed workflows are suitable for more cost-friendly lightweight or cost-effective tier models. Production systems can adopt routing and cascading strategies: simple tasks go to low-cost models, and failures or low-confidence results are escalated to a higher-capability tier.
4.2 Cost Should Be Measured as “Cost per Successful Task”
Comparing only input and output unit prices can be misleading. A more reasonable formula is: cost per successful task = (input billing cost + output billing cost + tool invocation cost + retry and human intervention cost) / task success rate. A high-capability tier model may have a higher unit price, but if its success rate is significantly higher and retries are fewer, the overall cost may be lower. Conversely, a low-priced model that fails frequently will also drive up total cost.
4.3 Latency and Stability Are Equally Critical
Agent tasks usually involve multiple model calls and tool round trips, so P95 latency reflects user experience better than average latency. Selection should test stability under concurrency, degradation behavior after rate limiting, performance degradation under long context, and regression differences after version upgrades. For interactive scenarios, timeouts, retry limits, and human takeover mechanisms should be set.
5. Decision Recommendations for Typical Scenarios
5.1 Complex Agent Coding Scenarios
Repository-level modifications, test-failure repair, and multi-file refactoring place high demands on long-horizon planning, tool invocation, and error recovery. It is recommended to prioritize high-capability tier models that perform stably in long-horizon tasks, together with isolated sandboxes, test feedback loops, and change auditing. Evaluation should measure cross-file modification success rate, test pass rate, regression introduction rate, and average repair rounds. Failure costs in these scenarios are high, so unit price alone should not be the deciding factor.
5.2 Cost-Sensitive Batch Scenarios
Tasks such as batch document processing, structured extraction, and routine ticket routing can control costs through fixed workflows, caching, batch processing, and fallback routing. A model that is close to the high-capability tier but has friendlier billing can be selected, and its stability on key boundary cases should be verified through private evaluation sets. Low-confidence results should enter human review or be escalated to a higher-capability tier.
5.3 Terminal and Interactive Environment Scenarios
Terminal operations and GUI interaction require environment feedback, permission isolation, and operation auditing. In addition to model capability, engineering safeguards are equally important: least privilege, secondary confirmation for dangerous operations, full logging, replay mechanisms, and abnormal termination. During evaluation, permission denials, timeouts, and UI changes should be injected to observe whether the model can exit safely rather than retry blindly.
6. Implementation Steps and Pseudocode
It is recommended to proceed in the following order: task set definition, sandbox and tool registration, offline evaluation, cost normalization, canary rollout, and continuous regression. On Baidu AI Cloud, general-purpose service categories such as container platforms, object storage, logging, and monitoring and alerting can be combined to build isolated evaluation environments and observability pipelines. The evaluation loop can be abstracted as follows:
tasks = load_versioned_tasks()tools = register_tools_with_schema()for model_config in candidate_configs:for task in tasks:env = create_isolated_sandbox(task)result = run_agent(model=model_config,task=task,tools=tools,max_steps=task.max_steps,timeout=task.timeout)score = judge(result, task.success_criteria)record(score, result.steps, result.retries, result.cost, result.latency)report_success_rate_and_cost_per_success()
After launch, failure cases should be replayed continuously, version drift should be monitored, and private evaluation sets should be updated regularly. Common pitfalls include selecting models solely by leaderboard ranking, drawing conclusions from a single sample, ignoring tool quality, looking only at unit price instead of cost per successful task, neglecting security boundaries, and lacking regression mechanisms. Scientific selection should center on real task success rate, use cost, latency, stability, and controllability as constraints, and complete the closed loop through public benchmark screening, private evaluation selection, and canary rollout validation.
