Open Weights and Inference Cost: The New Divide in LLM Selection

I. What Open Weights Actually Change

For a long time, competition among large language models centered on parameter count, benchmark scores, and inference speed. But when model parameters reach the trillion-scale range and the cost of a single training or inference run becomes significant enough to influence business decisions, the openness of weights—what format they come in, and under what license—has emerged as a more practical dividing line.

The release at the center of this discussion is not defined by another leap in parameter size. The 2.8 trillion total parameters, roughly 104 billion active parameters per token, and approximately 1.04 million tokens of context length largely continue the specifications of the previous generation. What genuinely stands out is the delivery format: the weights are publicly downloadable in MXFP4 compression across 96 shards, totaling about 1.56 TB, accompanied by a license that explicitly defines commercial boundaries.

This shifts the model from an API-only black box to a gray box with some degree of downloadability. But for developers, the distance between “downloadable” and “freely usable” remains substantial.

II. How to Read Benchmark Scores Correctly

The vendor reports a score of 65.7% on Terminal-Bench 4.0. For reference, two leading models on the same benchmark score roughly 66.4% and 58.2%. On the surface, a 0.7 percentage point gap from the top score suggests the model has entered the first tier.

But several details warrant close attention.

First, this is a self-reported result. Vendors can control inference intensity, sampling strategies, prompt formats, tool-calling behavior, and even context truncation policies within their own evaluation pipelines. These factors often impact task scores more than the raw capability difference between models. Without independent third-party reproduction, self-reported scores are better treated as an upper-bound estimate rather than a direct predictor of production performance.

Second, inference intensity is a key variable. The same model can lose significant points in latency-sensitive or budget-constrained scenarios if self-correction rounds or tool-call depth are reduced. High benchmark scores typically correspond to near-maximum compute budgets, which real-world projects rarely allow without restriction.

Third, weighted averages and task distribution matter. A single aggregate score obscures uneven performance across task categories. In agentic or terminal-operation benchmarks, some sub-tasks may far exceed the average while others fall well below. Model selection should be based on a breakdown of sub-task scores, not the headline number alone.

A safer interpretation: this model’s performance on terminal-operation benchmarks approaches the upper range of the current first tier, but actual project-level differences need to be re-validated against task distribution, inference budget, and deployment environment.

III. Open Does Not Mean Free: Three Key Licensing Thresholds

Publicly downloadable weights create the impression that the model can be used freely, but licenses often draw clear commercial boundaries. The license accompanying this release distinguishes two scenarios: internal use and providing model-as-a-service to third parties.

Specifically, no additional constraints are triggered for internal enterprise use, research and development testing, or integration into internal systems accessed through the official API. However, if the downloaded weights are deployed as a third-party-facing service and revenue exceeds $20 million over a consecutive 12-month period, a separate commercial agreement is required. Additionally, when monthly active users exceed 100 million or monthly revenue exceeds $20 million, the model name must be clearly displayed in the user interface.

The logic behind these thresholds is straightforward: small and medium-scale applications retain considerable flexibility. Once revenues reach a certain scale, negotiations move to a separate track. For most engineering teams, these thresholds are unlikely to be reached in the short term, but growth paths still need early assessment. If a product is expected to cross the $20 million revenue line within 12 months, the potential cost of a commercial agreement should be factored into total cost of ownership from day one.

Beyond licensing, the MXFP4 format introduces another barrier. A 1.56 TB weight file requires substantial memory even if storage space is sufficient. At roughly 80 GB per accelerator, holding just the weights requires around 20 cards—before accounting for KV cache, activations, and compute throughput. Organizations capable of privately deploying this system already possess significant engineering and operational infrastructure.

IV. Choosing Between API Price Gaps

Alongside open weights, API inference pricing has become a point of divergence. The official API from the release vendor is priced at approximately $10 per million output tokens. On the same day, another cloud provider released a model priced at roughly $2.5 per million output tokens—but with smaller total and active parameter counts, and full weights not yet publicly available.

This price gap requires careful decomposition. First, the two models are not in the same class. Comparing 2.8 trillion total parameters with 104 billion active against 770 billion total with 49 billion active reveals distinct differences in per-token compute, memory footprint, and latency characteristics. The price difference partially reflects different cost structures rather than purely competitive pricing strategy.

Second, API pricing competition is shifting from per-token rates to a combined assessment of quality and cost. For developers, saving $8 per million tokens only matters if quality differences are minimal. If a cheaper model requires more inference rounds to achieve equivalent results on complex tasks, total cost may actually be higher.

A more practical approach is a hybrid strategy: route high-difficulty tasks to a stronger model, route routine standardized tasks to a cheaper model, and use a unified gateway for traffic control. This leverages the respective cost and capability advantages of both models without being locked into a single option.

V. Safety Disclosure Data: Closer to Real-World Metrics Than Benchmarks

Safety evaluation data is often relegated to appendices or separate pages in technical documentation and release notes, and it is easy to overlook. But this release disclosed a number with direct engineering relevance: after removing network-attack-related training data, the exploit-test pass rate dropped from 90.5% to 81%.

This figure reveals two things. First, cybersecurity-related tasks are highly sensitive to the source and coverage of training data. Vulnerability analysis, attack path construction, and penetration testing reports in the training corpus directly influence a model’s performance on security benchmarks. When this data is removed from training, pass rates drop noticeably, indicating that security capability is not simply an extension of general reasoning ability.

Second, for teams evaluating models for security defense, code auditing, or vulnerability scanning, this kind of targeted disclosure carries more decision value than generic benchmarks. Security performance is not a one-dimensional “the higher, the better” question. If a model can be easily induced to generate attack payloads or bypass security checks, its usefulness in defensive scenarios is limited. Evaluating whether a model suits security-related tasks requires examining both its positive capability and its abuse potential.

From an engineering perspective, the 9.5 percentage point drop suggests a practical testing approach: if you plan to use an open-weights model for security tasks, build your own exploit-test set in a controlled environment rather than relying on general code generation or reasoning benchmarks.

VI. Practical Guidance for Model Selection

Based on the analysis above, the following decision paths may be useful for technical teams in different situations.

If you only need to call model capabilities. The official API remains the most direct path. While per-call cost may not be the lowest, it eliminates the hidden costs of weight download, distributed loading, inference engine adaptation, and ongoing operations. For workloads with fluctuating demand, pay-as-you-go elasticity also beats fixed hardware investment.

If you have a clear private deployment requirement. Scenarios such as data compliance rules prohibiting external API transmission, deployment on specific hardware environments, or deep customization and fine-tuning make open weights a genuine option. But the prerequisite is hardware resources at the level of roughly 20 accelerator cards, the ability to load and run MXFP4-format weights, and legal support to handle license compliance.

If you focus on cost optimization. Build an evaluation framework based on your actual business pipeline: measure first-attempt success rate, average correction rounds, total tokens per task, and end-to-end latency across typical tasks. Combine these four metrics into a “unit effective output cost,” and use that to measure the efficiency of every dollar spent, rather than comparing per-million-token list prices alone.

If you prioritize security scenarios. Look first for whether the vendor provides targeted safety capability disclosures. For models without such data, construct internal test sets covering vulnerability analysis, attack path construction, and defense strategy generation, and let measured results—not general reasoning benchmarks—drive the deployment decision.

Open weights do not provide unlimited free use; they provide choice. For some organizations, they are an entry point to technical autonomy and data localization. For others, they are merely an information asset that requires substantial operational cost to consume. The starting point for any decision is a clear definition of your own requirements and technical capacity.