From Parameter Races to System Competition: Open-Weight Models Enter a New Phase

For the past two years, discussion around large language models has revolved around parameter counts, context lengths, and benchmark rankings. More parameters, longer context, and higher math and coding scores seemed to translate directly into stronger technical capability. But as models start taking on long-horizon, multi-step tasks, this one-dimensional evaluation framework is quickly losing its meaning.

Recent industry reports describe an open-weight model with a total parameter scale in the trillions (with rumors placing it around 2.5 trillion), native support for million-token contexts, built-in vision understanding, and standout performance on frontend coding and agent-style tasks. The significance of this signal lies not in another parameter record, but in what it reveals about a deeper trend: open-weight models from China are closing in on the capability boundary of the strongest international closed-source models, and the center of competition is shifting from “the model itself” to the end-to-end delivery capability of “model plus execution system.”

In other words, open weights are just the entry point. The real business moat is the complete execution framework built around the model. That judgment deserves serious attention from every developer and technical decision-maker.

Trillion-Scale MoE: Balancing Capability and Cost Through Sparse Activation

The trillion-parameter scale is not achieved by stacking dense layers. It relies on a Mixture-of-Experts (MoE) architecture. The core idea: the model contains a large number of expert subnetworks, but each forward pass activates only a small subset of them. This keeps per-inference compute within an acceptable range even as total parameters grow enormous.

From an engineering standpoint, MoE brings three major implications:

1. Capacity and efficiency are decoupled. Total parameters define the model’s “knowledge capacity” ceiling, while activated parameters determine the cost of each inference. A trillion-scale total parameter count paired with a moderate activation ratio expands the model’s capability boundary without significantly raising inference costs.

2. Routing strategy becomes critical. The quality of expert routing directly determines model performance. Uneven routing leaves some experts overloaded and others idle, hurting training stability and inference throughput. In practice, mechanisms such as load-balancing losses and expert capacity factors are typically introduced to constrain this.

3. Distributed inference gets more complex. When experts are spread across multiple compute nodes, expert parallelism introduces cross-node communication overhead. Balancing communication against computation is one of the core challenges in deploying trillion-scale models.

Here is a simplified illustration of expert routing to show how the mechanism works:

# Simplified MoE routing illustration (not a real implementation)
def moe_forward(x, experts, router, top_k=2):
    # The router scores each token against every expert
    scores = router(x)                    # [num_tokens, num_experts]
    topk_scores, topk_idx = top_k_select(scores, top_k)
    # Only the top_k experts are activated; the rest are skipped
    outputs = dispatch_and_combine(x, experts, topk_idx, topk_scores)
    return outputs

It is worth emphasizing that MoE is not a free lunch: it raises the bar for training data quality, routing stability, and distributed communication. Whether trillion-scale parameters actually translate into task capability depends on whether these engineering details are handled well.

Million-Token Context: Long-Context Capability Is More Than a “Bigger Window”

Native support for million-token contexts means the model can process very long documents, large codebases, or multi-turn conversation histories in a single pass. From an engineering perspective, though, long context introduces challenges at three levels:

First, the cost of attention computation. Standard attention scales quadratically with sequence length. Without optimization, a million-token sequence would be prohibitively expensive to run. Common industry approaches include sparse attention, chunked attention, and positional encoding extrapolation—all aimed at preserving long-range dependencies while keeping compute under control.

Second, how effectively the model uses the information in context. A longer window does not mean the model makes full use of everything inside it. In practice, “lost in the middle” is a well-known failure mode—content located in the middle of a long context gets ignored. Long-context capability therefore has to be paired with retrieval-augmented generation, segmented summarization, and placing key information up front.

Third, state management and recovery. When an agent task spans multiple context windows, saving and restoring task state becomes critical. This is precisely what the execution harness needs to solve, covered in detail below.

For developers, the pragmatic approach is: don’t assume “a long enough window means you can stuff everything in.” Treat long context as a capability resource, and combine it with retrieval and summarization strategies to organize inputs dynamically based on the task.

Native Vision: Multimodality as the Foundational Perception Layer for Agents

Native vision support means the model can directly interpret images, charts, UI screenshots, and similar inputs without relying on bolted-on OCR or image captioning modules. This matters enormously for agent applications: many real-world tasks—UI operation, document understanding, chart analysis—naturally contain visual information, and whether the model can perceive it directly has a direct impact on feasibility and accuracy.

Architecturally, native multimodality usually means the vision encoder and the language model were jointly optimized during training rather than simply “stitched together.” The benefit is fuller cross-modal semantic alignment and more coherent understanding of mixed image-and-text inputs.

One practical caveat: multimodal inputs typically consume far more tokens than plain text. A single high-resolution image can take up hundreds or even thousands of tokens. When building agent systems, you need sensible resolution control and cropping strategies for visual inputs, or images will quickly fill up your context window.

The Real Divide: A Model Alone Is Not an Agent—Model Plus Harness Is

Users never interact with a bare model. They interact with a complete execution system: the model handles understanding and reasoning, while the execution harness supplies context, calls tools, persists state, verifies results, and pulls tasks back on track after failures.

Industry consensus on agent evaluation is fairly clear by now: evaluating an agent means evaluating the combined result of the model and its harness. In frontier coding agents, context curation, task state persistence, tool feedback handling, result verification, and cross-context recovery all directly affect final performance. Multiple benchmarks targeting agent harnesses have reached similar conclusions: the same model can show dramatically different task success rates under different harnesses.

This explains why “open weights” does not mean “free capability.” The model can be downloaded, deployed, and plugged into different software, but making it perform reliably on long-horizon agent tasks requires developers to handle the following correctly:

1. Organizing the reasoning history. An agent’s reasoning process needs to be stored in a structured way, not simply piled onto the conversation history. The right approach is to manage thinking, tool calls, and observations in separate layers to keep the context from bloating chaotically.

2. Managing context state. When a task spans multiple windows, historical information needs to be compressed, summarized, and prioritized so that critical state is never lost.

3. Parsing and validating tool feedback. Tool outputs may contain errors, malformed formats, or irrelevant information. The harness must clean and validate them before passing useful information back to the model.

4. Task recovery. After an interruption or failure, the framework should be able to restart the task from saved state rather than from scratch.

Here is a simplified agent execution loop showing how these pieces work together:

# Agent execution loop illustration (not a real implementation)
def run_agent(task, model, harness):
    state = harness.init_state(task)
    while not state.done:
        context = harness.build_context(state)      # Organize context and reasoning history
        action = model.reason(context)              # Model reasoning
        result = harness.execute(action)            # Invoke tools
        feedback = harness.validate(result)         # Validate tool feedback
        state = harness.update_state(state, feedback)  # Persist and restore state
        if harness.should_checkpoint(state):
            harness.save_checkpoint(state)          # Checkpoint for cross-context recovery
    return state.output

As you can see, the model is just one link in this loop. The harness does much of the unglamorous heavy lifting—and it is precisely that work which determines whether an agent can deliver reliably.

A Two-Layer Competition: Model Capability and Product Delivery Must Evolve Together

Based on the analysis above, competition among software companies will play out on two levels simultaneously:

The model layer: model capability sets the ceiling for what tasks can be accomplished. Trillion-scale MoE, long context, and native multimodality are all ways to raise that ceiling. But no matter how strong the model is, if the product cannot deliver reliably, you may end up with nothing more than impressive lab metrics.

The product layer: product experience determines user retention. But if the experience is polished while the model cannot complete complex tasks, the product easily degrades into a chat box bolted onto traditional software—seemingly intelligent, yet incapable of real complex work.

The truly competitive form is deep synergy between the two layers: model capability fully unleashed by the product, with product requirements in turn driving targeted optimization of the model and the harness. That is the underlying logic behind the judgment that “weights are the entry point; the harness is the moat.”

For engineering teams, the practical recommendations are:

• Don’t equate integration with calling an API. Building your own context management, tool orchestration, and state recovery mechanisms around the model is what constitutes a long-term asset.
• Build your own evaluation system. Generic leaderboards are only a reference. Agent-level evaluations grounded in your own business scenarios are the only way to genuinely reflect system capability.
• Invest in harness observability. Logging every task’s state transitions, tool calls, and failure reasons is the prerequisite for continuously improving agent stability.
• Leverage general-purpose cloud infrastructure. Object storage for task snapshots and artifacts, message queues to decouple tool calls, and logging and monitoring services to trace task chains—these commodity capabilities are the foundation for keeping long-horizon agents running stably.

Conclusion

Trillion-scale parameters, million-token context, and native vision do mark a capability leap for Chinese open-weight models. But what deserves even more attention is the shift in where competition happens: from “how big is the model” to “how stable is the system.”

Open weights lower the barrier to accessing models, but they raise the bar for engineering execution. Whoever builds deep synergy between the model and the harness will be the one to convert model capability into real product value. For developers and enterprises, that is both a challenge and an opportunity to build a durable technical moat.