The Evolution of Large Language Models: From Ultra-Large Sparse Architectures to Agentic Capabilities

1. The “Scale” of Parameters: Total vs. Activated Parameters

When discussing LLM capabilities, “parameter count” is one of the most easily misinterpreted metrics. Recently, a batch of models with total parameters reaching the trillions has emerged. While this number sounds astonishing, failing to distinguish between “total parameters” and “activated parameters per inference” can easily lead to false conclusions.

1.1 The Basic Principle of Mixture of Experts (MoE)

These models generally adopt the Mixture of Experts (MoE) architecture. The core idea is to divide the model into multiple relatively independent “expert” sub-networks, paired with a router network. During each forward inference, the router dynamically selects a small number of experts to participate in the computation based on the input, while the remaining experts remain dormant.

Here is a simplified illustration:

# Pseudo-code: Forward pass of an MoE layer
def moe_forward(x, experts, router, top_k=2):
    # router outputs the weights for each expert
    gate_scores = router(x)                  # shape: [num_experts]
    topk_weights, topk_indices = topk(gate_scores, k=top_k)
    topk_weights = softmax(topk_weights)
    output = 0
    for w, idx in zip(topk_weights, topk_indices):
        output += w * experts[idx](x)        # Only invoke the selected experts
    return output

The key takeaway is: Total parameters determine the upper limit of the model’s “knowledge capacity,” while activated parameters determine the “computational cost” of a single inference. A model with trillions of total parameters but only activating hundreds of billions per inference has a per-token inference cost roughly equivalent to a dense model with hundreds of billions of parameters, far lower than a traditional dense architecture with the same total parameter count.

1.2 Why This Forms the Basis for Lower Costs

In traditional dense models, the inference of every token must pass through all parameters, meaning the computational load is directly proportional to the total parameter count. MoE flattens this curve: total parameters can continue to scale up to improve knowledge coverage and generalization, but the FLOPs during inference are dictated by the activated parameters.

This creates a highly developer-friendly combination:

• Capacity dimension: Total parameter scale brings stronger knowledge storage and pattern fitting capabilities.
• Cost dimension: Activated parameters keep the computational consumption of a single call under control.
• Scaling dimension: Increasing the number of experts is easier to parallelize and shard than increasing the width of dense layers.

Understanding this layer makes it clear why “leading parameter scale” and “significantly lower price” can coexist—they are not contradictory, but rather two sides of the same architectural choice.

1.3 Long Context and Multimodality: From “Usable” to “Reliable”

Beyond parameter structure, the context window has been a key focus of recent model iterations. What does a context window of over a million tokens mean? Take a medium-sized code repository as an example: thousands of source files, configuration files, and dependency manifests combined usually fall within this magnitude. The model can read the entire repository at once and perform cross-file reasoning without complex retrieval-augmented generation (RAG) sharding.

These capabilities directly impact the following scenarios:

• Cross-file refactoring: Requires simultaneously understanding interface definitions, callers, and test cases.
• Dependency conflict resolution: Requires comparing multiple configuration and lock files.
• Codebase Q&A: Requires building a global index rather than local snippet matching.

The addition of native visual understanding capabilities allows models to directly process inputs like screenshots, architecture diagrams, and UI designs, reducing the intermediate loss of “converting to text before feeding it to the model.”

1.4 Anonymous Blind Testing: A New Release Paradigm

A noteworthy industry phenomenon is that some models undergo anonymous blind testing by developers on third-party evaluation platforms under codenames before their official release. They only “claim their identity” after accumulating a certain reputation.

This process is the exact reverse of the traditional “release technical report first, then launch product” sequence. Its value lies in:

1. Reducing brand bias: Evaluators do not know which company the model is from, leading to scores that better reflect true capabilities.
2. Early exposure of issues: The use-case distribution of real developers is far broader than standard benchmarks.
3. Building credibility: Letting the data speak first before discussing product positioning.

From an engineering culture perspective, this is an approach closer to the ethos of the open-source community.

2. Three Dimensions of Capability Evaluation

Evaluating whether a model is suitable for a production environment cannot rely on a single score. Here are three dimensions of recent evaluation focuses worth noting.

2.1 Coding Agents: From Completion to Autonomous Execution

Coding Agents do not merely test “writing the next line,” but rather “autonomously completing multi-step operations in a real terminal environment.” Typical tasks include: reading error messages, locating files, modifying code, running tests, and making further modifications based on failed results.

Key metrics for this type of evaluation usually include:

Dimension	Description
Task Completion Rate	Whether it ultimately passes the test cases
Step Efficiency	The number of interaction rounds required to complete the task
Error Recovery	Whether it can self-correct after a failed first attempt
Environment Understanding	Whether it correctly uses terminal commands and the file system

Based on public data, the task completion rate of top open-weight models in real terminal environments has entered the 60%+ range, narrowing the gap with top proprietary closed-source solutions to single-digit percentages. A year ago, this gap was in the double digits.

2.2 Long-Horizon Tasks: The Ability to “Stay on Track”

Long-horizon software engineering tasks (such as cross-day, multi-commit refactoring or migration) are fundamentally different from short tasks. They do not test single-point reasoning quality, but rather:

• Goal maintenance: Whether the original requirements are remembered after dozens of interaction rounds.
• State management: Whether it can correctly maintain a “completed/to-do” list.
• Context compression: How it trades off information when the context window is nearly full.
• Consistency: Not introducing contradictions across multiple rounds of modifications.

Interestingly, on these types of evaluations, some open-weight models have actually outperformed certain closed-source models that lead in short tasks. This indicates that long-horizon capability is not entirely correlated with single-step capability; it relies more heavily on task orchestration strategies and context management mechanisms during training.

The practical implication for developers is: If your use case involves multi-step automation (like batch migration or continuous integration fixes), you should prioritize long-horizon task scores during model selection, rather than general conversational scores.

2.3 Multimodality and Spatial Reasoning: Still Climbing

On benchmarks requiring spatial reasoning, such as 3D reconstruction and CAD understanding, model scores are generally high but lack differentiation, with the gap between top solutions usually within 3 percentage points. This suggests:

• Basic visual understanding is relatively mature.
• The real challenge lies in restoring 3D structures from 2D inputs and understanding engineering constraints.
• There is still significant room for improvement in this direction, especially regarding integration with professional toolchains.

3. The Significance of Third-Party Evaluation and Open Weights

3.1 The Role of Independent Evaluation Institutions

The value of independent evaluation institutions lies in providing cross-model horizontal comparisons. Taking composite intelligence index scores as an example, they are usually weighted across multiple benchmarks covering reasoning, coding, knowledge, and agentic dimensions.

A recent noteworthy signal is that a certain generation of open-weight models saw an overall score increase of about 11 points compared to the previous generation, entering the top ranks globally. More importantly, the judgment given by the evaluation institution states: The overall gap between open-weight and closed-source solutions is narrowing from about 12 points to about 3 points.

If this trend continues, the impact on developers is substantial:

1. Private deployment capability: Open weights mean models can be deployed on your own infrastructure.
2. Data sovereignty: Sensitive code and business data do not need to be transmitted externally.
3. Fine-tuning capability: Continued training can be applied for vertical domains.
4. Cost control: No rigid constraints of per-token billing.

3.2 Pragmatic Advice for Model Selection

Faced with a rapidly iterating model ecosystem, it is recommended to use the following framework for decision-making rather than chasing a single leaderboard:

Step 1: Define the Task Type

• Short interactive Q&A → Look at general reasoning and knowledge scores.
• Code generation and bug fixing → Look at coding agent scores.
• Multi-step automation → Look at long-horizon task scores.
• Mixed image-text input → Look at multimodal scores.

Step 2: Assess Deployment Constraints

• Is external data transmission allowed?
• Is there a need for private deployment?
• Peak concurrency and latency requirements.

Step 3: Conduct Small-Scale A/B Testing
Run comparisons using a set of real tasks (50–200 items) from your own business; this is more reliable than any public leaderboard.

Step 4: Focus on Cost Structure
Do not just look at the unit price. Consider:

• The number of tokens required to achieve the same result.
• Additional consumption caused by retries and error correction.
• The impact of context length on the cost per call.

4. Conclusion

In this round of model iteration, the most noteworthy aspect is not the refreshing of a single benchmark score, but three structural changes:

First, sparse architecture makes “large capacity” and “low cost” no longer mutually exclusive. The decoupling of total and activated parameters allows models to continuously expand their knowledge capacity while keeping inference overhead controllable.

Second, agentic capability has become the new competitive focus. Moving from “able to write code” to “able to autonomously complete tasks,” the evaluation focus is shifting from language quality to task completion.

Third, open weights are narrowing the capability gap with closed-source models. When the gap narrows to single-digit percentages, deployment flexibility, data compliance, and customizability become the deciding factors in model selection.

For developers, this means more choices, but it also requires a clearer understanding of their own needs. Clarifying the task type, deployment constraints, and cost structure, and then validating with real business data, is far more effective than chasing any leaderboard.