Rethinking Model Selection After Open-Weight Releases: Licensing, Evaluation, and Total Cost

Open Weights Are Not an Automatic Capability Leap

The recent release of a large model with open weights has drawn attention less for its parameter count and more for the shift in how the model is distributed. The model in question has roughly 2.8 trillion total parameters, activates about 104 billion per token, and supports a context window around 1.04 million tokens. These figures are largely consistent with the previous generation—there is no order-of-magnitude jump.

What has changed is the distribution mechanism: the weights are publicly available in MXFP4 format across 96 shards, totaling approximately 1.56 TB, alongside a license that explicitly defines commercial use boundaries. This signals that the intent behind the open release is not to demonstrate raw capability gains but to offer an additional deployment option.

For developers, it is important to understand what this “openness” actually means. Being able to download weights does not imply the same freedom as typical open-source software—modification, redistribution, and commercial use are all constrained by the license. The license specifies commercial usage thresholds, user-scale limits, and even UI labeling requirements. When evaluating such a model, separate your assessment of “how much has capability improved” from “do our usage conditions comply” rather than getting distracted by the act of release itself.

Benchmark Scores Must Be Read in Context

In some vendor-published benchmark results, the model scores 65.7% on Terminal-Bench 4.0, very close to leading closed-source models—within one percentage point. But such numbers need proper framing.

First, these results come from the vendor’s own test setup with maximum inference effort. They are not independently reproduced by a third party on identical prompts and procedures. Different inference strategies, sampling parameters, and prompt engineering choices can all shift the final score. A strong benchmark showing can be treated as a signal of potential, not as proof that the model will perform at the same level in a production environment.

Second, a single benchmark captures only one slice of model capability. Terminal operations, code execution, and tool use each have their own nuances. A high score in one area does not automatically transfer to others. For selection purposes, the more reliable approach is to build an internal evaluation set based on your own business tasks and observe the model’s consistency and failure modes. Public benchmark numbers alone are not sufficient grounds for a decision.

Licenses Define Commercial Boundaries, Not a Free-for-All

The most common misunderstanding with open weights is equating “downloadable” with “free for commercial use.” In this case, the license sets explicit conditions:

• If you provide the model as a service (MaaS) and generate revenue above a certain threshold over 12 consecutive months, a separate agreement is required.
• If your monthly active users or monthly revenue exceed specified levels, you must clearly label the model name in your product interface.

Internal use and calls through the official API are not subject to these restrictions. This means for most small-to-medium teams and internal R&D scenarios, the open-weight release has limited practical impact. But for companies that plan to package the model into an external service and scale revenue, the license terms must be part of early technical and legal review.

The value of open weights is not that it lowers all barriers. It offers an additional option for privatization to organizations with sufficient engineering and compliance capacity. Jumping into production without fully understanding the license constraints can create downstream legal and operational risk.

Cost Comparisons Belong in a Total Cost of Ownership Framework

Cost is unavoidable in model selection. The official API for this model lists output pricing at about $10 per million tokens. Meanwhile, some existing models on the market have pushed output prices down to around $2.5 per million tokens—roughly a fourfold difference. However, those cheaper models have notably smaller total and active parameter counts, and their full weights are not publicly available.

Unit price is only one part of the cost structure. If you choose the official API, the main expense is per-call inference fees. If you download the weights and deploy privately, you need to account for the following:

1. Hardware and infrastructure: The weights are about 1.56 TB. Inference places heavy demands on GPU memory, memory bandwidth, and compute. Multi-card deployment and cluster operations are almost unavoidable.
2. Engineering and labor: Sharded weight loading, inference optimization, version updates, monitoring, alerting, and container orchestration all require a dedicated engineering team on an ongoing basis.
3. Stability and disaster recovery: Private deployment means you are responsible for service availability, load balancing, scaling, and failure recovery.
4. Compliance and security audits: License interpretation, data leak protection, and vulnerability response all need to be budgeted over the long term.

Only teams with high call volumes large enough to amortize infrastructure and labor costs—and with strong internal platform engineering—are likely to find private deployment superior to API calls in total cost of ownership. For most developers, using the official API or a lower-cost third-party service remains the path of least resistance for experimentation.

Security Boundaries Deserve More Attention Than Benchmarks

Beyond capability, security performance is an often-overlooked but critical selection dimension. According to vendor disclosures, after removing training data related to cyberattacks, the model’s exploit-test pass rate dropped from 90.5% to 81%. This suggests that the model’s capability in cybersecurity domains does not fully derive from general reasoning but depends partly on domain-specific training data.

This has direct implications for enterprise users:

• If your use cases involve security testing, penetration testing, or vulnerability analysis, pay close attention to how the model performs under safety guardrails, not just its general reasoning scores.
• The source of a model’s capability affects its adjustability. If certain abilities depend on specific training data, capabilities may fluctuate significantly with future updates or changing data compliance requirements.
• Security evaluation should be a standalone phase in model selection, running alongside functional and performance tests—not an afterthought.

From an engineering perspective, teams can build a minimal security evaluation baseline covering common risk types such as prompt injection, data leakage, unauthorized instruction execution, and dangerous code generation. Run this baseline continuously on each candidate model version and keep a traceable evaluation record.

Open Weights Transfer Choice—But Choice Requires Capability

The core change in this open-weight release is that it adds a privatization option to the market. It does not redefine model capability or lower usage barriers.

For technical teams, the decision can be framed as three questions:

1. Do we actually need privatization? Only if data compliance rules prevent external API use, or if you already operate large-scale inference infrastructure, does private deployment warrant serious consideration.
2. Can we support privatization? From downloading weights to production readiness, the path involves shard loading, inference optimization, service wrapping, and continuous monitoring. Teams without platform engineering experience often underestimate this chain of work.
3. Do the license and cost structure allow it? Your business model, user scale, and revenue levels all affect how the license applies. Confirm this with your legal team early.

If any of these answers are unclear, continuing with the official API or a more economical third-party service remains the more practical choice.

Open weights do not automatically lower the barrier to use. They transfer the choice from the model provider to the user. And the value of that choice depends entirely on your team’s ability to absorb and manage the freedom it brings.