LLM Capability Evaluation: From Framework Design to Model Selection
Evaluating large language models requires more than just looking at leaderboard scores. This article explores a systematic approach to LLM evaluation, covering five core dimensions, scenario-based model selection, a five-step implementation process, and potential risks to ensure reliable integration into your business workflows.
I. Why Systematic Evaluation is Needed
A common pitfall in model selection is relying solely on leaderboard scores. While these scores reflect performance on specific question types, they do not indicate whether the model is reliable in your actual business processes. A model that excels in general Q&A might struggle significantly with industry terminology-heavy tasks like contract review or code evaluation. Therefore, evaluation should start from real-world tasks, breaking down capabilities into observable dimensions, scoring them separately, and then making a comprehensive judgment based on business priorities.
The evaluation results directly impact technical selection, budget planning, and deployment timelines. If evaluation criteria are ambiguous, teams are prone to making decisions based on subjective impressions, leading to extensive rework later. Documenting the evaluation process as repeatable steps ensures that different individuals arrive at similar conclusions for the same model.
II. Five Core Evaluation Dimensions
The table below outlines example tasks, scoring focus, and reference weights for each dimension. The weights and thresholds are merely a starting point; teams should adjust them according to their business objectives.
| Dimension | Reference Weight | Example Task | Scoring Focus |
|---|---|---|---|
| Reasoning | 25% | Multi-step math word problems, cross-disciplinary policy analysis | Completeness of steps, verifiability of conclusions |
| Coding | 25% | Generate functions based on requirements, fix buggy code | Executability, boundary handling, readability |
| Multi-turn Dialogue | 20% | 20+ turns of dialogue, inserting constraints midway and testing recall | Information retention, correct adoption of corrections |
| Domain Knowledge | 20% | Industry regulation Q&A, time-sensitive questions | Factual accuracy, indication of uncertainty |
| Creative Writing | 10% | Writing three versions of the same topic for different audience styles | Distinct style differences, relevance to the topic |
It is recommended to use a unified five-point scoring scale for each dimension: 5 indicates fully meeting requirements, 3 indicates basically usable but requires manual modification, and 1 indicates unusable. The scores for each task are then weighted and summed to obtain the model’s score in that dimension.
Reasoning. Tasks should transition from simple information retrieval to multi-step reasoning. You can design open-ended questions requiring the integration of economic principles and environmental policies, asking the model to provide a solution and list its assumptions. Scoring should not only look at the conclusion but also whether there are obvious jumps in the intermediate steps and whether assumptions are explicitly stated.
Coding. Evaluation should first check if the code passes unit tests, followed by boundary conditions and exception handling coverage. For code repair tasks, you can intentionally embed a couple of hidden defects to observe whether the model can locate the issue and explain the cause, rather than just fixing surface syntax.
Multi-turn Dialogue. The key to long conversation testing is telling the model a constraint in early turns, such as “budget limit is 50,000,” and then asking related questions in later turns to check if the model still remembers this information. You can also intentionally introduce a false premise midway to see if the model accepts it directly or points out the contradiction.
Domain Knowledge. The evaluation focus is on the model’s expression of its own uncertainty. For issues with rapidly updating policies, the model should state the time range of its knowledge and recommend checking official documents. Fabricated regulations, statistics, or cases should result in direct point deductions.
Creative Writing. You can ask the model to write copy targeting technical staff, general consumers, and management for the same topic, and then manually check whether the styles are practically distinct and if there is stereotyped, templated expression.
III. Understanding Differences Across Model Generations
Different versions from the same vendor often exhibit recognizable differences during evaluation. Recording three versions anonymously (Model A, Model B, Model C) helps avoid preconceived judgments. In most teams’ tests, earlier versions tend to deviate from the topic in multi-step instructions, and the generated code requires more manual correction. Intermediate versions usually show significant improvements in reasoning and long conversations but may still fabricate information on extremely complex questions. Newer versions are often more stable in cross-domain integration and expressing uncertainty.
These descriptions are only common trends and cannot directly replace test results. When comparing different versions, record the model name, version number, test date, number of questions, and reviewers. If the sample size is too small, such as only five questions per dimension, the conclusions may fluctuate significantly. It is recommended to prepare at least 30 questions per dimension and retain the raw outputs for later review.
IV. Scenario-Based Model Selection Method
Different business scenarios focus on very different metrics. The table below can serve as a starting point for selection discussions.
| Scenario | Core Metrics | Parameter Scale Considerations | Latency & Cost | Canary Threshold Example |
|---|---|---|---|---|
| Intelligent Customer Service | Intent recognition accuracy, multi-turn consistency | Mid-size models usually offer better cost-performance | First token latency must meet real-time dialogue requirements | Human handoff rate below 80% of baseline |
| Coding Assistant | Test pass rate, project context understanding | Versions with stronger coding-specific capabilities preferred | Completion latency affects developer experience | Adoption rate reaches team agreed value |
| Content Creation | Style diversity, long-text consistency | Depends on content length | Unit cost sensitivity during batch generation | Reduction in manual editing time |
| Professional Q&A | Factual accuracy, source traceability | Needs to be used with Retrieval-Augmented Generation (RAG) | Retrieval chain adds latency | Error answer rate below agreed upper limit |
When selecting a model, first identify the part of the process where errors are least tolerable, and then rank models based on the metrics for that part. For example, in a professional Q&A scenario, even if a model is more outstanding in creative writing, as long as its factual error rate is high, it should not be assigned the primary answering task.
V. Five Steps for Evaluation Implementation
Step 1: Define Responsibilities and Standards. Clearly document what tasks the model will undertake in the process and what constitutes a passing result. Engineers, operations, and managers have different focuses; these should be unified before the evaluation begins.
Step 2: Design a Reusable Evaluation Set. For each question, it is recommended to record the following fields: task type, input content, expected output, scoring rules, boundary cases, and owner. The evaluation set should be jointly reviewed by domain and technical experts and updated regularly as the business changes.
Step 3: Multi-model Double-blind Comparison. Run the same inputs, replace the outputs with codes, such as “Output A07,” and have reviewers score them without knowing the model source. Training should be conducted before the review to unify the scoring standards. If the score difference between two reviewers exceeds 1 point, a third reviewer should arbitrate. Cohen’s kappa can be used to calculate inter-rater reliability as a reference; if it is below 0.6, it indicates that the scoring standards need further refinement.
Step 4: Small-Traffic Canary Verification. Allocate a small amount of traffic in the real business environment, recording user feedback and the number of manual corrections. During the canary period, rollback conditions should be set in advance, such as switching back to the original solution if the error rate exceeds the threshold for three consecutive days.
Step 5: Continuous Monitoring and Iteration. After deployment, it is still necessary to track changes in output quality, especially after model version updates or business data changes. Monitoring metrics should remain consistent with the standards defined in Step 1.
VI. Risks and Boundaries
No model can guarantee 100% professional accuracy. The focus of evaluation is to understand under what circumstances the model is prone to errors and design corresponding fallback mechanisms, such as manual review, retrieval verification, or refusal-to-answer strategies.
Using AI to assist in generating evaluation questions is a feasible approach that can increase the quantity and diversity of questions. However, the generated questions may be biased toward types the model excels at or contain implicit biases. Therefore, AI-generated questions need manual review and should be mixed with manually written questions.
Deploying a model also requires compliance review, including data sources, privacy protection, content safety, and output transparency. These requirements vary by region and industry and should be confirmed by the legal and compliance teams.
Selection decisions should comprehensively consider capability, cost, compliance, and risk, rather than just looking at a single score. The evaluation report should explicitly state the test conditions, sample size, and known limitations so decision-makers understand the scope of the conclusions.
VII. Frequently Asked Questions
How to evaluate LLMs? First, define business standards, then design an evaluation set covering reasoning, coding, multi-turn dialogue, domain knowledge, and creative writing. Score them using a double-blind method, and finally confirm the effectiveness through canary testing.
How to select an LLM? Start from the part of the business where errors are least tolerable, determine core metrics based on the scenario, compare candidate models’ performance on the same set of questions, and make a comprehensive decision considering cost, latency, and compliance requirements.
What to pay attention to during evaluation? The sample size must be sufficient, scoring standards unified, reviews anonymized, and conclusions annotated with test conditions. Continuous monitoring is required after deployment, as model performance changes with versions and data.
