A Guide to Selecting Open-Weight LLMs: Balancing Capability, Cost, and Security

1. The True Variables of Open Weights

A recent release of an ultra-large open-weight model has generated significant discussion. However, focusing solely on parameter scale means missing the real technical shift introduced by this launch.

Consider the basic specs: approximately 2.8 trillion total parameters, around 104 billion activated parameters per token, and a context window of roughly 1.04 million tokens. These figures align with previous generations and are not the main focus. The truly noteworthy aspect is the change in distribution format—the weights are publicly available for download in MXFP4 precision, divided into 96 shards, totaling about 1.56 TB, accompanied by a clearly defined commercial license.

The technical implications of this shift are far more significant than simply “larger parameters.”

From an engineering perspective, MXFP4 is a low-precision floating-point format. Compared to traditional FP16 or BF16, it drastically compresses VRAM usage with almost no loss in inference quality. For a trillion-parameter model, storing weights in BF16 would approach 6 TB, posing a massive challenge for storage and loading pipelines for most teams. By compressing the size to approximately 1.56 TB, MXFP4 makes private deployment on a single multi-GPU machine or a small-scale cluster practically feasible for the first time.

The sharding strategy is equally notable. The 96-shard design is not arbitrary; it typically indicates that the provider has anticipated the need for multi-node parallel loading, shard-level verification, and incremental updates. For ops teams, sharding means pulling data on demand and supporting resumable downloads, rather than dealing with an indivisible, monolithic file.

However, a commonly confused concept must be clarified: open weights do not equate to free commercial use. The license sets explicit boundaries for specific scenarios:

• Offering the model as a service (Model-as-a-Service) requires a separate agreement if the related revenue exceeds a certain threshold for 12 consecutive months.
• If monthly active users or monthly revenue reaches a specific scale, the model name must be displayed in the product UI.
• Internal use and calling the official API are not subject to these restrictions.

The underlying logic of these clauses is not uncommon: you can use it for free to build your own capabilities, but if you resell the model’s capabilities as an external service, you must enter a different commercial framework. Understanding this is a prerequisite for deciding whether downloading the weights is worthwhile.

2. Capability Assessment: Bridging the Gap Between Benchmarks and Real-World Engineering

Official data shows that the model achieved a 65.7% score on a certain agentic benchmark, lagging by less than 1 percentage point compared to leading closed-source models of the same period. While this number looks impressive, several questions must be raised before using it as a technical selection criterion.

First, there is a systematic deviation between self-tested scores and third-party reproductions. Vendors typically maximize inference intensity, extensively tune prompts, and even adapt to the test set boundaries. This does not imply data fabrication, but rather that the score represents “the model’s upper limit on this specific test” rather than “the average performance in your business scenario.”

Second, the topic distribution of benchmark tests often mismatches real-world business distributions. A model excelling at agentic operations may not be equally stable in long-document understanding, multi-turn tool calling, or structured outputs. A more reliable approach during selection is to build a small-scale evaluation set using your own business data, covering the 10 to 20 most frequent task types, and run candidate models against this identical set.

Third, the ratio of activated parameters to total parameters dictates the inference cost structure. With approximately 104 billion activated parameters per token, the actual compute consumption per forward pass resembles that of a 100-billion-level dense model, despite the massive total parameter count. This is critical for estimating inference latency and throughput—hardware requirements cannot be simply calculated using the 2.8 trillion figure.

A pragmatic evaluation process can be summarized as follows:

1. Define business task types and the minimum quality threshold.
2. Filter candidate models using the custom evaluation set.
3. For models passing the quality threshold, compare the cost per token and deployment complexity.
4. Finally, overlay security and compliance evaluations.

3. Cost Accounting: What Open Weights Actually Change

The most common misconception about open weights is treating them as synonymous with “free.” In reality, open weights change the structure of choice, not the barrier to entry.

Consider two typical paths:

Path A: Calling the official API. At public pricing, this costs around $10 per million tokens. The advantages are zero ops, automatic scaling, and continuous iteration; the disadvantages are fixed unit pricing, data egress, and inability to deeply customize.

Path B: Downloading weights for private deployment. On the surface, there is no pay-as-you-go pricing, but real costs include:

• Storage costs: Persistent storage and multi-replica redundancy for 1.56 TB of weights.
• Inference hardware: Accelerator clusters meeting VRAM and concurrency requirements.
• Ops labor: Model loading, version updates, failure recovery, and monitoring/alerting systems.
• Elasticity costs: Idle compute reserved for peak traffic.

When converted into an equivalent cost per million tokens, private deployment is often significantly more expensive than API pricing for small-scale usage. The marginal cost advantage of private deployment only emerges when call volume is sufficiently large and tasks are highly stable.

On the same day, another open-weight model appeared on the market, pushing output pricing down to approximately $2.50 per million tokens, creating a roughly 4x price gap compared to the official API. Notably, this model has around 770 billion total parameters and 49 billion activated parameters, but its full weights have not yet been released. This perfectly illustrates a reality: low-price APIs and open weights are two different product strategies and cannot be simply swapped for comparison. Low-price APIs suit cost-sensitive scenarios with no data residency requirements; open weights suit teams with strong privatization needs and ops capabilities.

Therefore, the real questions to ask during selection are not “which is cheaper,” but:

• Is my call volume large enough to amortize the fixed costs of private deployment?
• Is my data allowed to leave the domain?
• Do I have the engineering capability to maintain an inference cluster continuously?

4. Security Boundaries: A Metric More Important Than Benchmarks

In selection discussions, security capabilities are often addressed last, but their impact on production systems can far exceed a few percentage points of quality difference.

The model’s technical report disclosed an intriguing phenomenon: after removing training data related to cyberattacks, the pass rate for vulnerability exploitation tests dropped from 90.5% to 81%. These two numbers do not directly indicate whether the model is “safe” or “unsafe,” but they reveal a crucial fact—the model’s capability in specific high-risk tasks is highly coupled with its training data composition.

For technical selection, the value of this information lies in:

1. Capabilities are tailorable. If the provider can reduce certain high-risk capabilities through data governance, users can perform similar operations during the fine-tuning stage.
2. Security assessment must be scenario-based. General security benchmarks cannot cover your business-specific risks; you need to design red team tests tailored to your own invocation chains.
3. Open weights imply a transfer of security responsibility. When calling an API, the service provider handles some security filtering; after private deployment, input/output filtering, privilege escalation protection, and audit logs must be built internally.

An actionable security assessment checklist includes:

• Conduct adversarial testing on high-frequency inputs to observe if the model can be induced to execute dangerous operations.
• Check if the model exhibits privilege escalation tendencies during tool calling scenarios.
• Establish filtering and auditing mechanisms for output content.
• Perform integrity checks on weight files to prevent tampering within the distribution pipeline.

5. A Reusable Selection Decision Framework

Integrating the dimensions above, we can form the following decision process:

def should_self_host(monthly_tokens, data_residency_required, 
                     ops_capability, quality_gap_tolerance):
    # Data must reside locally, and ops capability is available
    if data_residency_required and ops_capability:
        return "Prioritize evaluating private deployment"
    # Call volume is large enough to amortize fixed costs
    if monthly_tokens > LARGE_THRESHOLD and ops_capability:
        return "Private deployment may be more economical"
    # Otherwise, prioritize API
    return "Prioritize API calls, retaining the option to switch"

It must be emphasized that the output of this framework is not a binary conclusion, but rather a phased choice with a migration contingency plan. The greatest value of open weights is precisely that they make “switching” viable—when API prices increase, services change, or compliance requirements tighten, the team has an already-validated alternative path.

6. Conclusion

Returning to the initial question: what are the highlights of this release?

To answer in one sentence: it advances the private deployment of trillion-parameter models from “theoretically feasible” to “economically justifiable.” MXFP4 precision, sharded distribution, and explicit license boundaries collectively form an actionable selection option.

However, what remains unchanged is that capabilities still require validation via custom evaluation sets, costs must be accounted for based on actual call volumes, and security must be built according to specific scenarios. Open weights provide options, not answers. Teams capable of downloading 1.56 TB of weights and accurately calculating license and ops costs gain a privatizable alternative; developers who merely want to invoke capabilities will still find the official API and cheaper third-party services more convenient.

What truly needs to be calculated in advance is always that one thing more important than benchmarks: where your security boundaries lie.