Evaluating Large Language Models in Practice: Coding and Agent Benchmarks You Can Trust
Public leaderboard scores reflect only one test set, one prompt format, and one scoring method—never the full story. This article explains how to evaluate large language models for real-world use by measuring distinct dimensions such as capability ceiling, stability, instruction following, long-context performance, and cost. It walks through practical evaluation methods for coding tasks (using executable unit tests in sandboxed environments) and agent tasks (tracking task completion, step efficiency, error recovery, and safety boundaries), plus approaches for testing long-context retrieval and everyday dialogue quality. The goal is a reproducible internal evaluation pipeline that turns "we heard it's good" into "it consistently meets our requirements."
Why Leaderboard Scores Should Never Be Taken at Face Value
The numbers on public leaderboards are produced under specific conditions: a particular set of tasks, a particular set of prompts, and a particular scoring methodology. Change the task distribution, the context length, or the calling method, and the same score can lead to a very different conclusion. So before reading any leaderboard result, confirm three things: where the test questions come from, whether they resemble your own business scenarios, and whether the scoring process can be reproduced.
A small score gap does not necessarily mean a real capability gap either. A one-percentage-point difference between two models may fall within statistical noise, or may simply reflect random fluctuation across a batch of questions. Engineering teams should treat leaderboards as a screening mechanism for narrowing down candidates—not as a basis for final selection.
Breaking “Capability” Into Measurable Dimensions
Model capability is not a single number. During evaluation, measure the following dimensions separately:
- Capability ceiling: under ideal prompting and context conditions, how difficult is the hardest task the model can complete?
- Stability: when the same task runs multiple times, what are the success rate and the consistency of the outputs?
- Instruction following: whether output format, constraints, and step order are strictly respected.
- Long-context utilization: whether retrieval and reasoning remain consistent when key information sits at different positions in the context.
- Latency and cost: response time, token consumption, and cost per task.
The ceiling and stability often diverge. A model may excel at complex reasoning problems while occasionally failing at simple format conversions. Looking only at the ceiling overestimates day-to-day usability; looking only at averages hides catastrophic failures. Record both, and set acceptance criteria for each.
Evaluating Coding Tasks
Coding tasks lend themselves to automated evaluation: the code a model generates can be checked by executable tests. A recommended workflow:
- Prepare a set of tasks close to your business needs. Each task includes inputs, expected outputs, and unit tests.
- Fix the prompt template, sampling temperature, and maximum output length so all models are evaluated under identical conditions.
- Run each task multiple times (for example, 5 times), recording pass rate, first-attempt pass rate, and average number of repair rounds.
- Execute the generated code in an isolated environment with limits on runtime, memory, and network access, so faulty code cannot harm the host system.
- Manually inspect a sample of the passing results to confirm the model isn’t gaming the tests—for example, by hardcoding test cases.
Below is a skeleton of an evaluation framework to illustrate the structure. In real use, the execution step should run inside a container or sandbox.
import subprocessimport sysimport tempfileimport timefrom dataclasses import dataclass@dataclassclass RunResult:passed: boolduration: floatdef run_candidate(code: str, test_code: str, timeout: int = 10) -> RunResult:"""Execute candidate code with tests in a temp directory; return pass status and duration."""with tempfile.TemporaryDirectory() as workdir:script_path = f"{workdir}/candidate_test.py"with open(script_path, "w", encoding="utf-8") as f:f.write(code + "\n\n" + test_code + "\n")start = time.time()try:proc = subprocess.run([sys.executable, script_path],capture_output=True,timeout=timeout,)passed = proc.returncode == 0except subprocess.TimeoutExpired:passed = Falsereturn RunResult(passed, time.time() - start)def pass_rate(results: list[RunResult]) -> float:"""Compute the pass rate across a list of run results."""if not results:return 0.0return sum(r.passed for r in results) / len(results)
This example invokes the subprocess with a list of arguments, avoiding shell parsing and command injection. However, it is not a substitute for real isolation. When executing untrusted code, pair it with containers, resource quotas, and network isolation.
Evaluating Agent Tasks
An agent task requires the model to plan steps autonomously, call tools, observe results, and iterate until the goal is reached. Evaluation here focuses on the full execution trajectory, observed across four dimensions:
- Task completion rate: whether the final state meets the acceptance criteria—for example, whether file contents are correct or an API response matches expectations.
- Step efficiency: the number of tool calls used, and the proportion of useless calls.
- Error recovery: after a tool returns an error or unexpected result, whether the model identifies the problem and adjusts its strategy.
- Safety boundaries: on high-risk operations such as deleting data, unauthorized access, or duplicate submissions, whether the model respects the constraints.
Build a simulated environment where tool return values are generated by scripts, ensuring consistent external conditions across runs. Also record the complete trajectory—including each step’s inputs, outputs, and reasoning—so failures can be traced back to their cause. Testing directly against real production systems is both hard to reproduce and likely to cause irreversible side effects.
Testing Long-Context Capability
A long context window spec is easily misread as “if it can read it, it can use it.” Evaluation should happen at two levels:
- Retrieval level: embed factual information at different positions in a long document and test whether the model can locate and cite it accurately.
- Reasoning level: require the model to synthesize information scattered across multiple locations, and verify both the reasoning chain and the citations.
Test at multiple context-length tiers—8K, 32K, 128K, and so on—and plot accuracy for each. The point on the curve where accuracy starts to drop noticeably is far more informative than the nominal maximum window size. If key information tends to cluster at the beginning or end of a document, separately compare position effects to determine whether the model suffers from mid-context information loss.
Evaluating Everyday Dialogue and Writing Quality
Everyday tasks—question answering, rewriting, summarization, translation, and multi-turn clarification—cannot be measured with unit tests. Use double-blind scoring instead:
- Collect around 100 real user requests covering a range of scenarios and difficulty levels.
- Remove model identifiers from the outputs and have at least two reviewers score them against a unified rubric, with dimensions including accuracy, completeness, conciseness, and tone.
- Compute inter-reviewer agreement (for example, Cohen’s kappa). When agreement is low, refine the scoring criteria and discuss the divergent samples.
This kind of evaluation is expensive, but it comes closest to the real user experience. For products aimed at end users, it often reflects actual satisfaction far better than public leaderboards.
Building a Reproducible Internal Evaluation Pipeline
Turn evaluation into a standardized pipeline with the following key steps:
- Version-control your test set. Every addition or removal of a task is recorded with its reason and date.
- Give all candidate models the same prompt templates and result-parsing logic, avoiding bias introduced by interface differences.
- Produce structured reports containing means, standard deviations, per-dimension scores, and representative failure samples.
- Set explicit pass thresholds before rollout. For example: coding task pass rate no lower than 95% of the current solution, while average latency stays within the budget cap.
The pipeline should rerun automatically whenever a model version is updated and compare results against history, so the team can catch regressions as soon as a new model is introduced.
Common Pitfalls When Interpreting Results
- Treating a single run as a stable measure of capability, ignoring the randomness introduced by sampling.
- Ignoring the impact of prompt differences, and crediting the model for gains that actually come from a well-written prompt.
- Using generic leaderboards in place of business-specific tests, causing real-world performance to diverge from expectations.
- Looking at quality without cost, then blowing the budget on API fees after launch.
- Test set contamination from training data inflating scores. Prefer internal or unpublished questions wherever possible.
Conclusion
The key to model selection is turning “we heard it’s strong” into “it consistently meets our bar on our tasks.” Combining dimension-by-dimension measurement, fixed experimental conditions, and complete records lets a team reach trustworthy conclusions at a controllable cost.
