Evaluating AI Systems and Assessing AGI: A Practical Framework

1. A Leap in Capability Does Not Mean AGI Has Arrived

When a new AI system is described as achieving leading benchmark results—or even labeled artificial general intelligence—the first questions should not focus on the label. Instead, the key issues are which tasks improved, how the evaluations were designed, whether the results are reproducible, and whether those capabilities transfer to real work.

AI capabilities can improve significantly over a short period. This is particularly important when a system can do more than answer questions: it may observe an interface, plan a sequence of steps, invoke tools, and adjust its actions based on feedback. At that point, the system is moving beyond content generation toward task completion.

However, the industry does not have a universally accepted definition of AGI that can be established through a single benchmark score. A more rigorous approach is to distinguish among capability improvement, task automation, and general intelligence.

The central questions can be divided into three levels:

1. Does the system perform better on a specific task? Examples include mathematical derivation, user interface operation, and code analysis.
2. Do its capabilities transfer? Does performance remain stable when the task category, software environment, or wording changes?
3. Can the system operate reliably in an open environment? When goals are ambiguous, unexpected events occur, or actions carry significant risk, can it recognize uncertainty, ask for help, and avoid causing harm?

Targeted benchmarks can provide evidence for the first level. The second and third require broader, longer-term evaluations that more closely reflect real-world use.

2. How to Interpret Different Types of AI Evaluations

2.1 Interactive Tasks: Continuous Observation and Error Correction

Traditional question-answering tests usually provide an input and evaluate the resulting answer. Interactive tasks require a system to receive environmental feedback over multiple steps. For example, it may need to inspect an interface, determine the current state, perform an action, and then verify that the action had the intended effect.

These tests are closer to real work than single-turn question answering, but their results still depend heavily on the test environment, task scope, and execution framework. Final performance may change depending on whether the system can retain relevant history, how timeouts are handled, and whether retries are permitted after a failure.

Evaluation reports should therefore describe the task set, available permissions, retry policy, and testing process. A percentage score alone does not show whether the result came primarily from the model or from the combined effects of the model, execution framework, and supporting mechanisms.

2.2 Mathematics and Reasoning: High Scores Do Not Guarantee Reliable Reasoning Everywhere

Mathematical evaluations can test multistep reasoning, symbolic manipulation, and complex problem-solving. Strong performance on difficult problems may indicate improved ability to handle tasks in a particular formal structure. It does not mean the system will be equally reliable across every real-world problem that involves reasoning.

Practical tasks often contain missing information, conflicting constraints, and facts that must be verified. A model may make subtle errors during a derivation or produce a conclusion that sounds logically coherent but lacks sufficient evidence.

Evaluations should therefore examine more than answer accuracy. They should also analyze:

• The types of errors the system makes
• Whether confidence levels are well calibrated
• How the system responds to missing information
• Whether it can identify the assumptions on which its conclusions depend

These properties help determine whether a system can reason responsibly under imperfect real-world conditions.

2.3 Security and Vulnerability Analysis: Stronger Capabilities Require Stronger Safeguards

Tasks involving software weaknesses or security issues can demonstrate a system’s ability to identify risks, but they can also expose opportunities for misuse. As capabilities improve, evaluation cannot focus only on whether the system can complete a task. It must also determine whether the system can distinguish defensive analysis from requests that could cause real harm.

A reasonable security assessment should cover:

• Permission controls
• Sensitive information handling
• Audit logs
• Human approval requirements
• Emergency stop and abnormal termination mechanisms

For high-risk operations, the system should have a clearly defined scope of authority. If authorization is unclear or an action could have irreversible effects, it should pause rather than proceed solely because it received a natural-language instruction.

Evaluation results should also explain the permissions and safeguards under which testing occurred. Performance in a controlled environment should not be interpreted as a guarantee of safety in an open environment.

3. Why Computer Use Matters—and Why Success Rate Is Not Enough

Computer interaction allows AI systems to assist with form entry, file organization, data retrieval, and repetitive workflows in specialized software. Compared with systems that only generate recommendations, direct interface interaction can shorten the path from instruction to execution. It may also make complex tools more accessible to users without specialized expertise.

Accurate visual targeting, however, does not make an entire workflow reliable. A typical interface-based task includes several stages:

1. Identify the current interface state.
2. Interpret the user’s goal.
3. Select the correct control or object.
4. Perform the action.
5. Wait for feedback.
6. Verify the result.

An error at any stage can cause later actions to build on an incorrect state. A system might click a button with a similar name, modify the wrong file, or submit a form before the page has finished loading.

Computer-use evaluations should therefore include at least the following metrics:

• Task completion rate: Whether the actual objective was achieved, rather than whether the expected actions were merely attempted.
• Completion time: The end-to-end duration, including waiting, retries, and human intervention.
• Step accuracy: Whether the correct objects, fields, and actions were selected.
• Recovery capability: Whether the system can detect and correct problems caused by pop-ups, network delays, or interface changes.
• Safety and auditability: Whether high-risk actions can be restricted and whether the system creates records that can be reviewed.

In professional software, interface operation must also be combined with domain knowledge. In circuit design, for example, the ability to place components and connect wires is not sufficient. The resulting design still requires rule checks, simulation, manufacturing constraint reviews, and professional confirmation.

A demonstration video can show that a particular workflow functions under specific conditions. It cannot replace comprehensive engineering validation.

4. Moving from Benchmark Scores to Credible Conclusions

For evaluation results to support meaningful conclusions, they should meet at least four requirements.

4.1 Disclose the Test Conditions

Reports should specify the task scope, runtime environment, supporting tools, available permissions, and retry rules. For interactive tasks in particular, evaluators should separate the contribution of the underlying model from that of the external execution framework.

Without this information, it is difficult to reproduce the test or determine what the reported score actually measures.

4.2 Report Multiple Dimensions of Performance

Average scores should be accompanied by failure rates, task duration, error severity, the proportion of tasks requiring human intervention, and variation across repeated runs.

An average can conceal a small number of failures with extremely high costs. In operational settings, rare but severe errors may matter more than modest improvements in overall completion rates.

4.3 Conduct Independent Replication

Results from one team, one dataset, or a single demonstration are not enough to support broad claims. Cross-validation across different environments, task designs, and independent evaluators can reduce the influence of data leakage, benchmark-specific optimization, and chance.

Independent replication is especially important when claims extend beyond a narrow benchmark to general reliability or intelligence.

4.4 Evaluate Long-Term Stability

Real work is not a one-time test. An AI system must maintain a consistent objective across long workflows, respond to changing information, and request help when uncertainty becomes significant.

Short evaluations cannot fully reveal whether a system will preserve context, recover from compounding errors, or remain reliable over extended periods.

5. Enterprise Deployment Should Use Tiered Authorization

Organizations adopting AI systems should begin with low-risk, reversible tasks rather than immediately allowing the system to perform critical operations. Permissions can be divided according to the potential impact of each task.

• Read-only stage: The system retrieves documents, summarizes information, or recommends actions without directly changing data.
• Reversible stage: The system may create drafts, organize copies, or make changes that can be restored.
• Controlled execution stage: External communications, payments, deletions, and changes to production environments require explicit authorization and human confirmation.

Organizations should also establish operational logs, data access boundaries, anomaly alerts, and rollback procedures. For long-running tasks, the system can be required to report its current status, planned next step, and unresolved uncertainties at critical checkpoints.

This approach preserves the value of automation while reducing the risk that one mistake will propagate through an entire workflow.

6. Conclusion: Focus on Verifiable Capabilities, Not Grand Labels

Advances in reasoning, tool use, and computer interaction may significantly change knowledge work and the way people operate software. Whether those advances represent the arrival of AGI, however, cannot be determined by promotional language or a small set of benchmark results.

A more reliable assessment asks whether the system can adapt across diverse, unfamiliar, and changing tasks; communicate uncertainty accurately; and continue working toward its objectives within clearly defined safety boundaries.

For developers and enterprises, the most useful question is not simply, “Is this AGI?” It is: “Which tasks can this system perform reliably, what happens when it fails, and how can its behavior and risks be verified and controlled?”

Evaluation and governance mechanisms built around those questions are essential for turning stronger AI capabilities into sustainable, practical value.