Evaluating Large Language Models: From Reasoning Breakthroughs to General Intelligence

1. High Scores Matter, but General Intelligence Requires Broader Evidence

When a model improves on mathematical reasoning, unfamiliar-rule induction, scientific tasks, or security analysis, the first questions should be: under what conditions did it succeed, what system configuration was used, and which previously difficult problems did it solve?

A strong result on several benchmarks does not, by itself, mean that humanity has entered the era of artificial general intelligence (AGI). There is no universally accepted operational definition of AGI. Research approaches also differ in how they define generality, autonomy, learning ability, and economic value. Claims about a new era that are not tied to testable criteria risk turning meaningful technical progress into an unfalsifiable slogan.

Results should be treated cautiously when they are not supported by public materials, complete evaluation protocols, or independent replication. This is particularly important for claims of near-perfect scores, major mathematical breakthroughs, or substantial cost reductions. An abstract, screenshot, or secondhand account is not enough to establish such claims as fact.

A more useful analysis separates three levels:

• Capability: Does the model show stronger rule understanding, abstract reasoning, knowledge transfer, and error correction?
• System: Can a complete system reliably handle complex tasks when equipped with tools, memory, search, and verification?
• Delivery: Can it consistently produce acceptable results under the time, cost, access, and risk constraints of real work?

These levels are related, but they are not interchangeable. Solving one exceptionally difficult problem does not establish that a model can run a reliable workflow over time. Nor should the success of a mature workflow be attributed entirely to the model.

2. Progress May Be Uneven Across Tasks

Advances in large language models do not necessarily improve every task at the same rate.

Mathematical proofs, scientific computation, reverse engineering, and complex troubleshooting can require long reasoning processes, movement among competing hypotheses, and adjustments based on external feedback. Improvements in these areas may yield substantial gains on difficult tasks.

Everyday question answering, short-form rewriting, and basic page generation may feel little different. There are several reasons:

2.1 Simple tasks may already meet practical needs

For tasks with fixed formats and few steps, existing systems may already perform well enough. A model with stronger reasoning may produce an output that looks much like the one produced by an older system.

For example, the quality of a simple form depends more on whether the requirements and styling constraints are clear and whether the result is tested at runtime than on advanced reasoning. Such tasks may not distinguish complex reasoning capabilities very well.

2.2 Delivery quality may be limited by factors outside the model

Retrieval quality, data permissions, tool stability, network latency, and incomplete business rules can all become bottlenecks. Better reasoning does not automatically resolve these engineering problems.

2.3 A specialized strength does not imply broad reliability

A model can excel at a class of difficult problems while still failing at simple instruction following, edge-case handling, or sustained execution. Capabilities should therefore be assessed by task category, rather than summarized by a single overall score.

For developers, the practical question is not whether a model is “smarter overall,” but whether it improves the most expensive or unreliable part of the current system.

3. How to Read Evaluation Results: Conditions Before Scores

Benchmark scores need to be interpreted in the context of the experiment. At a minimum, examine the following:

3.1 Is the test measuring a model or a complete agent system?

A model answering a fixed prompt and a system that can call tools, preserve state, make multiple attempts, and verify its answers are different test subjects.

A complete system may include:

• Task decomposition and planning
• External knowledge retrieval
• Code execution and result checking
• Persistent state
• Search across candidate approaches
• Retries and result selection
• Logic adapted to the task environment

These components are not incidental aids that can simply be ignored; they contribute to real-world capability. But comparisons must use consistent boundaries. It is not meaningful to rank a carefully orchestrated system directly against another model’s single, unassisted response.

Reports should specify the model version, tool access, state-management approach, number of attempts, and compute budget.

3.2 Results from multiple attempts are not single-attempt reliability

“Success at least once across several attempts” is different from “success on the first submission.”

Assuming independent attempts, if the probability of success on each attempt is (p), then the probability of at least one success across (k) attempts is:

[
1-(1-p)^k
]

For example, with a 50% chance of success on each attempt, four attempts yield a theoretical 93.75% chance of at least one success. This example illustrates the difference in measurement; it does not represent the performance of any particular model.

In real tasks, attempts are often not independent. A model may repeatedly follow the same faulty line of reasoning, so the formula cannot be used directly to infer real single-attempt success rates.

Another often-overlooked issue is answer selection. Evaluators may use hidden answers to identify the correct candidate, while a production system may not have access to an equivalent judge. If the system generates a correct result but cannot identify it among several candidates, the task may still fail in practice.

3.3 Unfamiliar-rule evaluations should test the limits of generalization

Rule-induction tasks can show whether a system learns patterns from limited feedback. But a high score alone does not rule out training-data contamination, familiarity with the task format, or benchmark-specific adaptation.

More persuasive evaluations can:

• Use unpublished tasks designed independently
• Vary visual layouts, descriptions, and action mappings
• Change surface forms while preserving the underlying logic
• Limit trial counts and execution budgets
• Test whether the system can adapt again after the rules change

These designs help distinguish memorizing familiar solutions from building a new model of the task.

3.4 Near-ceiling scores may mean the benchmark needs an update

If multiple systems repeatedly score near the maximum, the benchmark may no longer distinguish them well. It can still measure basic competence, but tiny score differences should not be treated as evidence of major progress.

A better approach is to introduce new task distributions, dynamic environments, hidden constraints, and resource limits, while measuring failure modes rather than only increasing task difficulty.

4. Research and Mathematics: A Correct Result Is Only the Starting Point

Research tasks matter because they involve incomplete information, open-ended exploration, and multiple stages of validation. In this respect, they are closer to real knowledge work than closed-form question answering.

A credible research workflow should cover at least:

1. Understanding the question and available information
2. Proposing testable hypotheses
3. Selecting a computational method or experimental design
4. Processing and analyzing data
5. Checking anomalies, bias, and alternative explanations
6. Recording the process and making the work reproducible
7. Stating the limits of the conclusions

A system’s ability to operate in a computational environment does not, by itself, constitute a scientific discovery. It may run an incorrect analysis or produce numerical results without establishing that the method’s assumptions hold.

The same distinction applies in mathematics. “Generated a proof draft,” “passed formal verification,” and “produced a new result accepted by experts” describe different levels of evidence. Claims of a major breakthrough should be checked against the original argument, its assumptions, the verification process, and independent review.

In particular, an improvement to a bound should not automatically be described as bringing a famous conjecture close to resolution. Numerical progress and overcoming a theoretical obstacle are not always linearly related.

5. Long-Horizon Performance Depends on Closing the Loop

Complex tasks involve multiple dependent steps. Even when performance on individual steps is strong, end-to-end success can fall as the number of steps grows.

In a simplified model, suppose a task has 20 required steps, each with a 98% chance of being correct, and errors are independent. The probability that every step is correct is about 66.8%. This is not a prediction of any real system; it illustrates why high per-step accuracy does not guarantee a reliable long workflow.

Real systems also face error propagation, lost context, changing environments, and tool failures. A reliable agent needs a closed loop:

Observe the current state → choose the next action → execute → verify the result → update state → recover or roll back when needed.

Verification should not rely only on the model’s subjective assessment of its own output. Whenever possible, it should use external evidence, such as test results, constraint checks, data consistency checks, and business-rule validation.

Engineering practices should also include:

• Saving checkpoints at critical stages
• Protecting operations that may cause side effects if repeated
• Limiting retry counts and budgets
• Requiring human approval for irreversible actions
• Recording inputs, actions, outputs, and supporting verification
• Confirming the environment’s state before proceeding when it is uncertain

Mature long-horizon capability means knowing when to proceed, when to pause, and how to recover from failure—not simply generating a longer plan.

6. Safety Analysis Should Not Be Measured by Success Rate Alone

Vulnerability analysis, binary understanding, and system diagnosis can test advanced technical reasoning. These capabilities also have clear dual-use risks, so evaluations must account for authorization and safeguards.

First, distinguish among task objectives. Troubleshooting a service, identifying a code defect, validating a security risk, and exploiting a vulnerability are not the same capability. They should not be combined into one score simply because they all involve computer systems.

Second, testing should take place in legally authorized, isolated environments with clearly defined scope, controlled data, and auditable procedures. High-risk actions require least-privilege access, resource limits, and human approval.

Finally, evaluations should record false-positive and false-negative rates, invalid actions, environmental damage, and the completeness of supporting evidence—not just successful detections. A system that finds problems but often changes configurations incorrectly or loses its analysis trail may not be suitable for production.

7. “Less Reasoning, Lower Cost” Requires End-to-End Accounting

Shorter outputs and fewer calls do not automatically make a system more efficient.

Visible text represents only part of the computational process. External tools, parallel candidates, caching, hidden computation, and human corrections can all affect the final cost. Output length alone is not a sufficient basis for comparison.

Businesses can use a measure such as:

Cost per successful task = total cost during testing ÷ number of tasks that pass acceptance criteria.

Where possible, total cost should include model calls, compute, tool execution, storage, retries, and human review. The cost of failed deliveries should also be tracked separately, since the business impact of an infrequent failure may far exceed the cost of model calls.

Efficiency comparisons should use the same acceptance criteria and examine:

• Whether success rates are comparable
• Whether first-pass success improves
• Whether human intervention decreases
• Whether end-to-end latency falls
• Whether high-risk errors increase
• Whether costs remain stable across task difficulty levels

Simple tasks may need a lightweight process, while complex or high-risk work may warrant additional search and verification. The goal is not to minimize steps in every case, but to allocate compute and review effort where they provide the most value.

8. How Developers and Businesses Can Run a Credible Evaluation

Rather than repeatedly trying a few demonstration tasks, build a small evaluation suite that can be run consistently.

8.1 Sample real work by task category

The task set should cover routine work, long-tail cases, historical failures, and high-risk boundaries. Tasks should come from real needs, with appropriate de-identification and access reviews.

Keep an independent test set that was not used to tune the system. Otherwise, repeated optimization against known tasks can make it harder to judge generalization to real work.

8.2 Define acceptance criteria before testing

For a coding task, criteria might include passing functional tests, handling edge cases, and avoiding new high-risk problems. For data analysis, they might require reproducible calculations, consistent data definitions, and evidence supporting the conclusions.

Criteria should be objectively checkable where possible. If expert judgment is required, define the evaluation dimensions and retain the basis for the review.

8.3 Use two complementary comparisons

One comparison tests success rates under the same time, tool, and compute budgets to identify capability differences.

The other compares cost and latency while holding the quality threshold constant to identify efficiency differences.

These comparisons answer different questions and should not be collapsed into a single ranking.

8.4 Isolate the contribution of each system component

Compare, in sequence:

• The model alone
• The model with tools
• The model with tools and state management
• The complete system with independent verification

These ablation tests help identify where gains come from and whether the next improvement should target the model, retrieval, tools, or verification.

8.5 Run tests repeatedly and analyze failures

In addition to average success rates, record variability, completion time, human takeover frequency, and major failure categories. When a test set is small, do not interpret a narrow score difference as a definitive advantage.

Failure analysis is especially useful: gaps in knowledge, misunderstood requirements, poor planning, tool errors, and missing verification call for different remedies.

9. How to Assess Progress Toward General Intelligence

To discuss general intelligence, start with testable questions rather than a predetermined conclusion.

Can a system transfer existing capabilities to unfamiliar domains? Can it learn new rules from limited feedback? Can it express uncertainty when information is incomplete? Can it maintain goals, detect errors, and recover during long tasks? Does it remain stable under constraints on compute, time, and permissions?

These questions are closer to the core of generality than a top score on a single leaderboard, but they still require sustained evidence across contexts.

A reasonable response to a new model breakthrough is to recognize that specialized capabilities and systems engineering may have improved substantially, while still demanding transparent evaluation criteria, independent validation, and real-task data. Individual failures do not erase all value, and a handful of high scores do not mean every problem has been solved.

For developers and businesses, the most important shift is not whether a system earns an era-defining label. It is whether it can move beyond generating plausible answers to consistently delivering verifiable, traceable results at an acceptable cost and within explicit constraints. That is what turns technical progress into practical productivity.