Next-Generation General Models: Capability Boundaries and Engineering Evaluation

In recent years, the capability boundaries of large models have continued to expand. They have moved from text generation to multimodal understanding, from answering questions to invoking external capabilities, and further toward operating graphical interfaces and executing cross-application workflows. At the same time, statements such as “test scores are approaching the ceiling,” “agents can complete any work on a computer,” and “models are participating in training the next generation of models” have become increasingly common.

These developments deserve attention. From an engineering perspective, however, high scores, compute scale, and demo performance cannot independently prove that general intelligence has been achieved. The question enterprises really need to answer is not “how strong does the model look?” but whether it can complete tasks reliably in real business environments, handle exceptions, make errors discoverable and recoverable, control costs, enforce permissions securely, and support auditing of the entire process.

Therefore, judging the value of next-generation general models requires moving from news-style narratives to systematic evaluation.

1. Why Benchmark Saturation Does Not Mean There Is No Capability Ceiling

Benchmark saturation usually means that a model’s score on a fixed test set is already close to perfect, making it difficult for the test to continue distinguishing among different models. It shows that the model performs strongly on that type of task, but it cannot be directly used to infer that the model already has comprehensive and stable general capabilities.

1.1 Fixed Test Sets Have Coverage Limits

Any test set can cover only a limited range of question types, input distributions, and scoring rules. A model achieving a high score on fixed questions may mean that it has mastered relevant knowledge and problem-solving patterns, or it may mean that there is similarity between the training data and the test content.

Tasks in real environments are more complex. For example, a task to “organize sales data and generate a report” often includes the following steps:

1. Log in to the business system and retrieve data;
2. Identify the meaning of fields and statistical definitions;
3. Handle null values, duplicate values, and outliers;
4. Aggregate by department or time range;
5. Generate charts and written conclusions;
6. Write the results into a specified document;
7. Send the results to relevant people according to approval requirements.

This type of task is not a closed question, but a workflow with permission restrictions, data quality problems, and dynamic page changes. Strong single-point question-answering capability does not necessarily guarantee successful end-to-end execution.

1.2 Average Scores Can Hide Critical Failures

For enterprises, whether a model occasionally produces severe errors is often more important than average accuracy. Suppose an agent succeeds in ninety-nine out of one hundred operations, but in the remaining operation it mistakenly deletes production data. The average success rate may appear high, but the actual risk is unacceptable.

Therefore, in addition to average scores, organizations should focus on recording:

• Performance in the worst-case scenarios;
• Incidence of severe errors;
• Proportion of unsupported outputs;
• Number of unauthorized operations;
• Recovery capability after exceptions;
• Frequency of human intervention;
• Consistency when the same task is executed repeatedly.

1.3 Test Tasks May Lack Real-World Noise

Experimental environments usually have clear inputs, fixed interfaces, and well-defined goals. Production environments often experience page delays, pop-up interference, field renames, network interruptions, missing data, and expired permissions.

A more credible evaluation system should actively introduce these disturbances. For example:

• Randomly adjust button positions;
• Change page themes and window sizes;
• Insert unrelated notifications and pop-ups;
• Simulate API timeouts or empty data returns;
• Provide vague or even conflicting business instructions;
• Revoke a certain permission midway through a task;
• Require the model to resume execution from a checkpoint.

Only when the model can still maintain a high success rate under these conditions can it be said to have some degree of environmental adaptability.

2. “Operating a Computer” Is a System Capability, Not a Single-Model Capability

Enabling a model to complete tasks on a computer usually requires vision understanding, language reasoning, task planning, tool invocation, and state management to work together. The final result depends not only on the base model but also on the design of the entire agent system.

2.1 Perception Layer: Understanding the Current Interface State

The perception layer is responsible for identifying text, buttons, input fields, tables, menus, and prompt information on the page. The main difficulties include:

• Similar controls are easy to confuse;
• Page zoom changes element positions;
• Dynamic content may load with delay;
• Icon meanings depend on context;
• Pop-ups and floating layers may obscure the target of an operation;
• Multi-window environments can easily cause focus misalignment.

A system that relies only on screen-coordinate clicks will be very fragile. A more robust approach is to combine visual information, page structure information, and application state, and to locate targets semantically wherever possible rather than assuming that a button will always be in a fixed position.

2.2 Planning Layer: Decomposing Goals into Executable Steps

Users usually provide high-level goals, such as “organize this month’s contracts and generate a risk list.” The model must decompose the goal into a series of operations while clarifying dependencies and completion conditions.

A general task plan can be represented as follows:

Task goal: Generate a contract risk list
Step 1: Confirm the contract directory and time range
Precondition: The user has authorized reading of the target directory
Completion condition: A file list matching the time criteria is obtained
Step 2: Extract key contract fields
Precondition: The file formats can be parsed
Completion condition: Data records in a unified structure are formed
Step 3: Run risk rule checks
Precondition: The risk rule version has been confirmed
Completion condition: Each record includes a risk level and the basis for the judgment
Step 4: Generate a summary report
Precondition: Abnormal files have been marked separately
Completion condition: The report includes statistical results, risk details, and items requiring manual confirmation
Step 5: Submit for human review
Precondition: The report must not be sent externally automatically
Completion condition: The reviewer confirms the report content

The key to this type of plan is not the number of steps, but that each step has clear preconditions, completion conditions, and failure-handling methods.

2.3 Execution Layer: Invoking Capabilities and Verifying Results

The execution layer cannot be responsible only for “clicking.” It must also confirm whether the click produced the expected result. For example, after submitting a form, the model should check whether a success message appears, whether the data was actually saved, and whether the page has moved to the correct state.

A reliable execution loop usually includes the following process:

Loop until the task is complete:
    Retrieve the current environment state
    Compare the current state with the target state
    Generate candidate next actions
    Execute the action
    Check the action result
    If the result matches expectations:
        Write to the execution log and continue
    Otherwise, if recovery is possible:
        Roll back to the most recent checkpoint and retry
    Otherwise:
        Pause the task and request human handling

The most important part here is “verification after execution.” Without verification, an error in an earlier step may continue to propagate and eventually create a chain of failures that is difficult to detect.

2.4 Memory Layer: Preserving Task Context and Key Evidence

Long-running tasks often take several minutes or longer, and the model must remember which files have been processed, which filter conditions have been used, the reasons for failures, and user constraints.

Memory should not simply be an unlimited append-only conversation history. Instead, it should use structured records, such as:

• Task identifier;
• Current stage;
• Completed steps;
• Pending items;
• Summaries of key inputs and outputs;
• Permission status;
• Number of failures;
• Most recent checkpoint;
• Location of evidence files;
• Records of human confirmation.

Structured memory supports task recovery, troubleshooting, and auditing, and it can also reduce inference costs caused by irrelevant context.

3. How to Understand “AI Participating in Training AI”

In the training of next-generation models, earlier models may participate in data generation, answer screening, reasoning verification, or training feedback. This approach can improve the efficiency of data production, but it should not be simplistically understood as models being able to complete self-evolution fully autonomously.

The common approaches mainly include the following categories.

3.1 Synthetic Training Data

Existing models can generate questions, answers, explanation steps, and counterexamples according to task templates, thereby supplementing data in scarce domains. The advantages are fast generation speed and broad coverage, but the approach can also easily inherit the biases of the original model.

Without quality control, synthetic data may contain:

• Factual errors copied at scale;
• Highly similar expression patterns;
• Unreasonable difficulty distribution;
• Reasoning processes that appear complete but lead to incorrect conclusions;
• Small-probability biases amplified in large-scale data.

Therefore, synthetic data usually needs to go through rule validation, cross-validation, deduplication, difficulty stratification, and manual spot checks.

3.2 Model Evaluation and Feedback

One model can score another model’s responses and check whether they meet format, logic, and safety requirements. This method can reduce the cost of human evaluation, but the evaluating model itself may also make incorrect judgments.

In high-risk tasks, a single model should not serve as both the generator and the final judge. More reasonable mechanisms include:

• Separating the responsibilities of the generation model and the evaluation model;
• Using multiple independent judgments for cross-validation;
• Applying rule-based validation to high-risk results;
• Randomly sampling results for human review;
• Periodically evaluating the consistency between the evaluation model and human standards.

3.3 Automatically Generating Training Tasks

Models can also identify areas where current capabilities are weak and generate more targeted training tasks. This is close to automated curriculum design, but whether the tasks are reasonable still needs to be constrained by external goals and evaluation mechanisms.

If evaluation metrics are poorly designed, the model may tend to generate tasks that are easy to score highly on rather than tasks that improve real capabilities. This is a typical case of “optimization objectives diverging from business objectives.”

3.4 Human Oversight Remains Essential

AI participating in training does not mean that humans have completely exited the process. Humans still need to take responsibility for goal definition, data governance, risk classification, evaluation criteria design, and final accountability.

Especially in high-risk domains such as healthcare, finance, government services, and industrial control, training oversight needs to clarify:

• Which data can be used for training;
• Which data must be de-identified;
• Which results must be manually reviewed;
• Which errors are unacceptable events;
• How training data and model versions are tracked;
• How to roll back after a systemic bias is discovered.

4. What Growing Training Compute Means

Large-scale training usually requires substantial computing resources, but “using more computing hardware” does not automatically mean “capability improves in the same proportion.”

Model capability is influenced by multiple factors, including data quality, model architecture, training objectives, optimization methods, context organization, and inference strategies. Compute is only one important variable among them.

4.1 Scale Expansion Has Diminishing Returns

When other conditions remain unchanged, increasing training resources usually improves model capability, but the gains may gradually decline. Continuing to expand scale also increases costs for power, networking, storage, and operations.

Technical teams should focus not only on total compute, but also on:

• Capability gain per unit of computing resources;
• Effective utilization of training data;
• Communication overhead in distributed training;
• Efficiency of training failure recovery;
• Degree of capability retained after model compression;
• Unit task cost during inference.

4.2 Training Cost and Usage Cost Are Different

A model with high training investment does not necessarily mean that it will be expensive for enterprises to use. Conversely, a model with limited training investment may still incur high usage costs because it requires too many inference steps.

When evaluating usage cost, the full task chain needs to be considered:

Single-task cost =
input processing cost
+ model inference cost
+ multi-round retry cost
+ external capability invocation cost
+ data storage and logging cost
+ human review cost
+ error repair and compensation cost

Comparing only the listed price per unit of input or output can easily overlook the additional consumption caused by repeated planning, reading long contexts, and failure retries in agent systems.

5. General Intelligence Cannot Be Declared by a Single Test

Whether we have entered the era of general intelligence is first of all a question of definition. If general intelligence is understood as the ability to learn, transfer, plan, and execute stably across a wide range of tasks, then it must be judged comprehensively from multiple dimensions.

5.1 Breadth

Whether the model can handle tasks across multiple domains rather than performing well only on one type of problem.

5.2 Transfer Ability

Whether the model can quickly adapt to unseen rules, interfaces, and tasks through a small amount of instruction, rather than relying on fixed templates.

5.3 Long-Horizon Stability

Whether the model can execute tasks containing dozens or even more steps and maintain goal consistency when exceptions occur midway.

5.4 Self-Verification

Whether the model can recognize uncertainty, check results, detect contradictions, and proactively request help when information is missing.

5.5 Causal Understanding and Environment Modeling

Whether the model understands the relationship between actions and results, rather than merely guessing the next step based on surface patterns.

5.6 Safety and Controllability

Whether the model respects permission boundaries, can refuse dangerous instructions, and allows humans to pause, roll back, and assign accountability.

Even if a model achieves high scores on multiple tests, as long as it still exhibits unpredictable unauthorized behavior in real environments, execution errors that cannot be recovered from reliably, or an inability to determine whether it has made an error, it should not be regarded as a mature general intelligence system based solely on leaderboard results.

6. What Kind of Evaluation System Enterprises Should Build

Enterprises can divide model evaluation into four stages: offline testing, sandbox testing, shadow running, and limited production.

Phase 1: Offline Capability Testing

Without connecting to real business systems, test the model’s knowledge, reasoning, instruction following, and structured output capabilities.

Recommended metrics include:

• Task accuracy;
• Output format compliance rate;
• Factual consistency;
• Completeness of cited evidence;
• Quality of uncertainty expression;
• Consistency of repeated answers to the same question;
• Sensitive information recognition rate.

Phase 2: Sandbox Environment Testing

Replicate business processes in an isolated environment, allowing the agent to perform operations without touching real production data.

Faults and interference should be deliberately injected:

• Page elements change;
• Input data is missing;
• Network latency is simulated;
• Insufficient permissions are simulated;
• Duplicate files are created;
• Conflicting instructions are inserted;
• Tasks are terminated midway and then resumed.

Phase 3: Shadow Running

The agent reads real tasks but does not directly execute the final operations. It only generates suggested actions and compares them with the results of human operations.

Shadow running can answer three key questions:

1. How large is the difference between the agent’s decisions and human decisions?
2. Which tasks can be safely automated?
3. Which steps must retain human approval?

Phase 4: Limited Production

Select low-risk, reversible tasks for production and limit the scope of operations. For example, allow the agent only to read data, generate drafts, or update temporary records, while prohibiting direct deletion of data, modification of permissions, or external publication.

After the system becomes stable, gradually expand the task scope and permissions rather than opening all capabilities at once.

7. Production-Grade Agents Must Have Safety Boundaries

The stronger the computer-operation capability, the greater the potential risk. Production systems should have at least the following safeguards.

7.1 Least Privilege

An agent should receive only the minimum permissions required to complete the current task. Read, write, delete, approval, and external-send permissions should be separated from one another.

7.2 Confirmation of High-Risk Actions

For operations such as transfers, deletions, bulk modifications, permission changes, and external sending, human confirmation should be required. The confirmation page must show the action object, scope of impact, and key parameters, rather than providing only a vague “continue or not” option.

7.3 Complete Auditing

The system should record the model version, input summaries, planned steps, tool invocations, execution results, human approvals, and exception information. Logs need to be tamper-resistant, with access permissions set according to data sensitivity levels.

7.4 Interruptibility and Rollback

The system should support immediate pause during operations. For reversible operations, transaction, versioning, or snapshot mechanisms should be designed. For irreversible operations, the approval level should be raised before execution.

7.5 Data Isolation

Production data, test data, and training data should be clearly separated. Before sensitive information enters the model, de-identification, field trimming, or access control should be applied according to business requirements to avoid exposing irrelevant data to the task context.

8. Cost Evaluation Should Center on Successful Tasks, Not API Calls

One of the most common mistakes enterprises make when purchasing or deploying model capabilities is comparing only the price per unit of input and output. The truly meaningful metric should be “the comprehensive cost of completing one qualified business task.”

The following calculation can be used:

Effective task cost =
all model and infrastructure costs in a period
÷
number of tasks that pass quality acceptance in the same period

At the same time, record:

End-to-end success rate = successful tasks without human remediation ÷ total tasks
Human intervention rate = tasks requiring human handling ÷ total tasks
Severe error rate = errors causing high-risk impact ÷ total tasks
Cost per unit of value = effective task cost ÷ business value generated per task

If a highly capable model can significantly reduce retries and human review, then even if its per-inference price is higher, its final cost may still be lower. Conversely, a lower-priced solution that fails frequently may bring higher hidden costs.

9. System Architecture Recommendations for Next-Generation Models

To avoid tightly coupling business systems to the capabilities of a specific model, organizations can establish an independent orchestration and governance layer between applications and models. Its core modules include:

• Unified task entry point: receives natural-language goals and structured parameters;
• Planning module: decomposes complex goals into steps;
• Capability adaptation layer: uniformly encapsulates model calls and business interfaces;
• Policy engine: determines whether an action is allowed to execute;
• State storage: saves task progress and checkpoints;
• Quality validation: uses rules or independent models to check results;
• Human approval: handles high-risk actions and exception tasks;
• Monitoring and alerting: tracks latency, cost, failure rate, and unauthorized behavior;
• Audit center: records evidence across the entire task chain.

The advantage of this architecture is that when the underlying model is replaced in the future, business permissions, security policies, and process controls can remain stable. When evaluating models, enterprises can also conduct side-by-side validation under the same tasks, permissions, and data conditions.

10. Conclusion: Move from Chasing High Scores to Validating Real Value

Progress by next-generation large models in interactive tasks, complex reasoning, and computer operation may indeed change the way software is used. In the past, people had to learn the menus and operating procedures of each application. In the future, users may only need to describe a goal, and an agent may be responsible for planning and execution.

However, between “capability demonstrations” and “production readiness” there remains a series of engineering issues involving stability, security, cost, data governance, and responsibility boundaries. Benchmark tests approaching full marks indicate that old evaluation methods may need to be upgraded. Models participating in training indicate that data production and oversight methods are changing. Large-scale compute investment indicates that capability competition is moving toward higher resource density. These phenomena are all important, but they cannot independently constitute sufficient evidence that general intelligence has been achieved.

For developers and enterprise users, a more pragmatic strategy is to build their own task sets and acceptance criteria, validate capabilities in a sandbox, observe risks in shadow mode, and then gradually enter production with limited permissions. The system ultimately worth adopting is not the one with the highest promotional numbers, but the one that can continuously complete tasks within clear boundaries, proactively expose uncertainty, recover after errors occur, and have both cost and risk quantified.