From LLM Evolution to AGI: Capability Evaluation and Implementation Guide

1. How to Rationally View the “AGI Era”

With general-purpose models rapidly improving in mathematical reasoning, scientific QA, code generation, web operation, and complex task planning, whether Artificial General Intelligence (AGI) has been achieved has become a core debate in the tech industry.

However, from an engineering perspective, AGI is not a conclusion confirmed by a single model release, a few benchmark scores, or an executive’s statement. It requires judgment across at least three dimensions:

1. Capability Breadth: Can the system handle a sufficiently wide variety of task types, rather than just excelling on a few test sets?
2. Capability Depth: Can it remain stable when facing long-chain, open-ended, and highly uncertain problems?
3. Autonomy Level: Can it continuously perceive its environment, formulate plans, invoke tools, verify results, and adjust strategies upon failure?

Therefore, even if a model approaches a perfect score on multiple benchmarks, it only proves strong adaptability to specific task distributions. To determine whether it has reached a broader level of general intelligence, we must also examine real-world generalization, long-term stability, error recovery, safety boundaries, and economic viability.

For enterprises, a more valuable question than “Is it AGI?” is:

• Which business tasks can it reliably complete?
• What are the probability and cost of failure?
• Does it offer observability, auditability, and rollback capabilities?
• How do you balance cost, latency, and accuracy?
• Which steps must retain human approval?

Only by clearly answering these questions can model capabilities be transformed into stable productivity.

2. What Does Benchmark Saturation Mean?

When advanced models continuously approach the upper limits of traditional test sets, the industry often refers to this as “benchmark saturation.” This does not mean intelligence has reached its endpoint; rather, it suggests that existing tests are no longer sufficient to differentiate between the strongest models.

2.1 Saturation Scores Do Not Equal Comprehensive Capability

A test set typically covers only a limited range of question types, knowledge scopes, and scoring criteria. Even if a model achieves top scores, the following issues may still exist:

• The test tasks are too closely aligned with the training data distribution.
• Questions lean heavily toward closed-ended problems with definitive answers.
• Scoring focuses only on the final result without examining the reasoning process.
• Sample sizes are insufficient to reflect low-probability but high-risk errors.
• The system’s continuous execution capability in dynamic environments cannot be assessed.

For instance, a model might excel in a single-turn scientific QA task. However, in a real-world research task, it also needs to perform literature screening, hypothesis construction, data verification, experimental design, and conclusion review. Any unverified content in these steps can render the final result untrustworthy.

2.2 Evaluation Must Shift from Static Q&A to Dynamic Tasks

A next-generation evaluation system should simultaneously include the following four types of tasks:

• Static Capability Tests: Knowledge QA, mathematical calculations, code comprehension, and logical reasoning.
• Interactive Tasks: The model must continuously adjust its operations based on environmental feedback.
• Long-Horizon Tasks: Tasks may involve dozens of steps and span a significant duration.
• Adversarial Tasks: Proactively introduce ambiguous instructions, misinformation, permission constraints, and exception states.

More importantly, evaluation metrics cannot be limited to the success rate. Enterprises should also record:

• Average completion time.
• Number of tool calls.
• Human intervention rate.
• Resource consumption per task.
• Error recovery success rate.
• High-risk action interception rate.
• Reproducibility of output results.

Only by combining these metrics can we determine whether a model is truly production-ready.

3. Why Computer Operation Capability is a Key Watershed

The primary value of traditional language models is text generation. However, an AI agent equipped with computer operation capabilities can further read interface states, invoke business tools, modify files, fill out forms, and execute workflows. This signifies a shift from “providing suggestions” to “directly completing tasks.”

A relatively complete agent execution loop typically includes:

1. Receiving goals and constraints.
2. Reading the current environment state.
3. Decomposing the goal into multiple subtasks.
4. Selecting appropriate tools or operation entry points.
5. Executing actions and obtaining feedback.
6. Determining whether the result meets expectations.
7. Retrying, switching approaches, or requesting human assistance upon failure.
8. Outputting the result, evidence, and complete operation log.

This process can be described using the following pseudocode:

task_status = initialize(user_goal, permission_boundary, cost_limit)
loop until task is completed:
    environment_info = read_current_state()
    next_step_plan = generate_plan(task_status, environment_info)
    if next_step_plan involves high-risk operations:
        approval_result = request_human_confirmation(next_step_plan)
        if approval_result is denied:
            terminate task and record reason
    execution_result = execute_in_isolated_environment(next_step_plan)
    validation_result = validate(execution_result, goal_constraints)
    write_audit_log(next_step_plan, execution_result, validation_result)
    if validation_result is success:
        update_task_status()
    else:
        perform limited recovery or degradation
return final_result, evidence, and audit_record

The real technical challenge is not making the model “click faster,” but ensuring it makes correct judgments under complex interfaces, network fluctuations, permission changes, and exceptional feedback.

Therefore, computer operation in production environments must adhere to three principles:

• Least Privilege: The model should only access the resources required to complete the current task.
• Reversible Actions: Operations like deletion, publishing, fund transfers, and permission modifications must have undo or approval mechanisms.
• Verifiable Results: Success cannot be judged solely based on the model’s self-reporting; the actual state must be read for confirmation.

4. How to Evaluate Software Engineering Capabilities

Code generation is only one part of software engineering. A model with engineering value also needs to understand requirements, locate bugs, modify multi-file code, execute tests, analyze logs, and control the scope of changes.

A more rational software engineering evaluation should cover the following processes.

4.1 Requirement Understanding

The model needs to identify functional goals, non-functional constraints, and acceptance criteria. If requirements are ambiguous, it should ask clarifying questions rather than directly generating an implementation.

4.2 Codebase Analysis

Real-world projects typically contain multiple modules, configuration files, and dependencies. The model must understand call chains, data flows, and exception handling logic to avoid fixing only surface-level issues.

4.3 Minimal Modification

An excellent engineering change is not about making as many modifications as possible, but rather minimizing the impact scope while meeting the objective. Evaluations should focus on the number of changed files, compatibility risks, and regression issues.

4.4 Automated Verification

Code generated by the model needs to pass static checks, unit tests, integration tests, and security scans. If a test fails, the system should feed structured error information back to the model, but infinite retries must be restricted to prevent resource exhaustion.

4.5 Human Review

Modifications involving core business logic, data structure changes, authentication and authorization, and production configurations should not be directly published by the model. The model can generate change summaries, risk explanations, and test evidence, leaving the final approval to the developer.

Key metrics for this workflow include first-time fix success rate, test pass rate, defect injection rate, human review time, and rollback ratio. Simply counting how many lines of code were generated does not reflect actual engineering value.

5. Cybersecurity Capabilities Require Dual Boundaries

Advanced models can help analyze vulnerabilities, identify anomalous code, generate remediation suggestions, and explain attack paths. However, they can equally be used to expand the automation of attacks. Therefore, stronger security capabilities necessitate stricter access controls and usage restrictions.

When deploying security-focused agents, enterprises should establish the following defense lines:

5.1 Environment Isolation

All validation behaviors should be completed in authorized testing environments or sandboxes, without defaulting to connections to live production networks. Test data should also be desensitized.

5.2 Permission Layering

The model may have permissions to read logs, analyze configurations, and generate suggestions, but active scanning, policy modifications, or executing high-risk commands require additional approval.

5.3 Operation Whitelisting

Allowed tools, parameter ranges, and target resources should be explicitly constrained. You cannot rely solely on natural language prompts to prevent unauthorized actions.

5.4 Full Audit Logging

The system needs to record input instructions, model plans, tool parameters, execution results, approvers, and timestamps for accountability and post-incident review.

5.5 Exception Circuit Breaking

When the model exhibits behaviors such as continuous failures, repeated calls, surging resource consumption, or accessing sensitive targets, the task should be immediately suspended.

In this domain, the model’s “upper limit of capability” is not the sole evaluation criterion. The ability to operate stably within authorized limits is the foundation for secure deployment.

6. What Changes When AI Trains AI

As model scale and training complexity increase, manual inspection of every training sample and output becomes impractical. Using relatively mature models to participate in data filtering, answer evaluation, error classification, and training supervision is becoming a standard technical direction.

This approach offers three main benefits:

• Improving the efficiency of training data preparation.
• Generating more complex tasks and edge-case samples.
• Conducting initial sorting and quality checks on candidate responses.

However, using models to supervise models also introduces new risks.

First, the supervisory model’s own biases may be inherited and amplified. Second, two models might form a consensus on the same error, creating a “false agreement on an incorrect outcome.” Third, if the training pipeline lacks traceable records, pinpointing the source of a problem becomes extremely difficult.

Therefore, AI-assisted training should not completely replace human effort; instead, it should employ multi-layer validation:

1. Models perform large-scale initial screening.
2. Human review is conducted for high-risk and high-uncertainty samples.
3. Independent evaluation sets are used to check capability and bias shifts.
4. Data sources, filtering rules, and version information are securely stored.
5. Red teaming and regression tests are executed on critical capabilities.

This human-in-the-loop collaborative approach is better suited for current high-risk model training than full automation.

7. Enterprises Should Adopt a Layered Agent Architecture

Enterprises should not connect large language models directly to all business systems. A more robust approach is to build a layered architecture that decouples model inference, tool invocation, permission control, and audit monitoring.

Layer 1: User and Business Entry

Responsible for identity authentication, request intake, file uploads, and result presentation. This layer should limit input length, file types, and sensitive data scope.

Layer 2: Task Orchestration

Responsible for decomposing tasks, selecting execution strategies, controlling retry limits, and managing context. Complex tasks can be broken down into retrieval, analysis, execution, and validation phases.

Layer 3: Model Service

Selects models of varying capabilities and cost tiers based on task difficulty. Simple classification or information extraction does not require the highest-tier model, while complex reasoning triggers an upgrade.

Layer 4: Tool Gateway

Centrally manages databases, object storage, message queues, search services, logging services, and internal business APIs. The model should not directly obtain underlying credentials but should access them through a controlled gateway.

Layer 5: Security and Governance

Responsible for permission validation, content moderation, sensitive data masking, action approvals, quota limits, and audit trails.

Layer 6: Observability System

Continuously records response latency, task success rates, resource consumption, error types, and human takeover instances, triggering alerts when anomalies are detected.

The advantage of this architecture is that even if the underlying model is replaced in the future, the business permissions, data governance, and tool ecosystem remain stable, avoiding vendor lock-in to a single model.

8. Cost is More Than Just Input/Output Pricing

The cost of using a model is rarely just the text processing price. A real-world agent task may also incur costs for retrieval, file parsing, tool invocation, repeated reasoning, failure retries, log storage, and human approval.

Enterprises can use the following general formula to estimate the total cost per task:

total_task_cost =
model_inference_cost
+ data_retrieval_cost
+ tool_execution_cost
+ storage_and_logging_cost
+ failure_retry_cost
+ human_review_cost

Common methods to reduce costs include:

• Model routing based on task difficulty.
• Using caching and retrieval for stable knowledge.
• Compressing invalid context to avoid repeatedly transmitting historical content.
• Setting budgets and maximum step counts for long-horizon tasks.
• Delegating deterministic steps to traditional programs.
• Classifying failure reasons to reduce meaningless retries.

The most powerful model is not necessarily the optimal choice for all tasks. What enterprises truly need is the lowest total cost that meets quality requirements.

9. Long-Term Metrics for Judging AGI

To rigorously determine whether a system is approaching general intelligence, a few high scores are far from sufficient. Continuous observation of the following metrics is required:

• Can it transfer to unseen tasks?
• Can it actively clarify when information is insufficient?
• Can it identify its own uncertainty?
• Can it complete tasks lasting hours or longer?
• Can it replan after environmental changes?
• Can it distinguish between facts, assumptions, and speculations?
• Can it act within permission and ethical boundaries?
• Can it reliably reproduce results at a reasonable cost?

Among these, “knowing what it does not know” is particularly crucial. An over-confident, highly capable system poses a greater risk in medical, financial, security, and industrial control scenarios than a slightly less capable system that can proactively refuse tasks and request human assistance.

10. Conclusion: From Chasing Labels to Building Reliable Systems

The capability leap of general-purpose models is indeed transforming software development, scientific research, knowledge work, and enterprise automation. Benchmark saturation, enhanced computer operation capabilities, improved software engineering skills, and AI participating in training are all technological trends worth noting.

However, there remains a significant gap between “high capability” and “safe to use.” Regardless of whether a model is labeled as AGI, enterprises must not bypass independent evaluation, permission isolation, human approval, result verification, and audit tracing.

For developers, future core competitiveness will not merely be writing prompts, but understanding task boundaries, designing reliable workflows, and integrating models, data, tools, permissions, and monitoring into a sustainably operating system.

The true milestone is not a leaderboard showing a near-perfect score for the first time. It is when an intelligent system can operate stably over the long term under real-world constraints: providing evidence when completing tasks, recovering when errors occur, stopping when risks are encountered, and ensuring accountability when impacts are made. This is the critical step from model capability evolution to the practical application of general intelligence.