1. The Real Story Isn’t the Parameter Count—It’s That the Weights Are Downloadable

The open-source community recently saw a typical 3T-class open-weight model release: roughly 2.8 trillion total parameters, a mixture-of-experts architecture activating about 104 billion parameters per token, a context window of around 1 million tokens, weights released in MXFP4 quantized format across 96 shards totaling approximately 1.56TB, and a license with explicit terms defining commercial boundaries.

On specs alone, none of these numbers are novel—sparsely activated large-scale MoE and ultra-long context are continuations of existing industry technical directions. What genuinely deserves discussion is three things:

1. The shift in choice: open weights return the decision of “whether and how to use it” to the user;
2. The infrastructure threshold: a 1.56TB payload delivered as shards imposes concrete requirements on download, verification, storage, and networking;
3. The compliance gap: license terms determine that “can download” and “can use commercially” are not the same thing.

This article uses these three points to turn a typical open-weight release into an actionable reference for model selection and deployment.

2. Reading the Key Specs: The Engineering Meaning Behind Three Numbers

2.1 2.8T Total Parameters, 104B Activated: The MoE Trade-off

The core of MoE (Mixture of Experts) architecture is replacing the feed-forward layers of a dense model with multiple “expert” sub-networks, where a router selects only a small number of experts per token for computation. This creates a key decoupling relationship:

• Storage and VRAM requirements ∝ total parameters: all expert weights must reside in VRAM or memory;
• Per-inference compute ∝ activated parameters: the compute cost per token is equivalent to a hundred-billion-class dense model.

The 2.8T/104B combination means you must be able to “afford” the storage and bandwidth for over two trillion parameters, while the compute budget is calculated at the hundred-billion-parameter level. This is also the root cause of why such models “look cheap but easily blow the budget in practice”—compute attracts attention, while storage, bandwidth, and networking costs are often underestimated.

2.2 Million-Token Context: Capability Ceiling and VRAM Cost

A million-token context is a genuine capability gain for repository-level code understanding, long-document analysis, and multi-source information aggregation, but KV cache VRAM usage grows linearly with sequence length. The commonly used engineering estimate is:

KV Cache ≈ 2 × layers × KV heads × head dimension × sequence length × batch size × bytes per element

Taking a model with hundred-billion-level activated parameters as an example, if general-purpose KV-compression designs such as grouped-query attention are not adopted, the KV cache for a single long sequence can reach tens of GB. Production environments therefore typically need to pair it with:

• Paged KV cache management to reduce VRAM fragmentation and improve throughput;
• Prefix cache reuse so multi-turn sessions hitting common prefixes avoid redundant computation;
• Request-level length budgets and truncation policies: do not interpret “supports 1M context” as a commitment that “every request can be filled to the brim”—set input budgets based on actual business distribution.

2.3 MXFP4 and 96 Shards: How to Land 1.56TB

MXFP4 is a microscaling 4-bit floating-point format that preserves better numerical dynamic range than naive integer 4-bit quantization, with controllable quality loss in most scenarios. The direct implication for users: weight size is significantly reduced compared with 16-bit precision, and the inference side can carry the same parameter count with fewer VRAM nodes.

Facing a release delivered as 96 shards totaling about 1.56TB, several things must be done in engineering:

# Integrity verification after shard download (general workflow)
sha256sum -c weights_checksums.txt   # Verify hash per shard; re-download immediately on any failure
# After verification passes, merge in manifest order, or hand shards directly to the inference framework for loading

Also plan ahead for:

1. Download channels and resumable transfer: 1.56TB may take days on an ordinary office network; use tools that support resumable downloads over stable links;
2. Verification is not optional: anomalies caused by corrupted shards often only surface mid-training or mid-inference, and backtracking is extremely costly;
3. Version management: establish an object storage directory structure keyed by “model name-version-quantization format,” retain the original checksum manifest, and avoid mixing multiple versions.

3. Open Weights ≠ Free Commercial Use: Three Key License Clauses

Using the typical structure of such licenses as an example, at least three points should be checked clause by clause before selection:

1. Service clause: if the model is packaged as a model-as-a-service offering to external users and the company’s revenue exceeds the annual revenue threshold set by the license, a separate commercial license must be signed with the copyright holder;
2. Attribution clause: when a product’s monthly active users or monthly revenue reach a certain scale, the source must be labeled in the user interface in a form such as “powered by XX model”;
3. Exemption scope: internal enterprise use and calls through the official API are typically not subject to the above restrictions.

Accordingly, the compliance checklist should answer three questions: Does our business constitute “providing model services externally”? How far are revenue and user scale from the thresholds? Will attribution obligations affect the product interface form? These questions should take priority over “can the weights be downloaded” itself.

4. How to Read Benchmark Scores: Self-Testing, Reproduction, and Real Projects

New model releases typically come with coding and agent benchmark results—for example, scores on terminal-operation benchmarks reporting a gap narrowed to within 1 percentage point of leading closed-source models. Interpreting such numbers requires three clear-eyed recognitions:

1. Vendor self-testing vs. third-party reproduction: self-tested scores are often obtained with inference intensity maxed out (longer chains of thought, larger sampling budgets); when third parties reproduce the same test set with different inference configurations, scores may drop noticeably;
2. Public test sets vs. private test sets: community feedback commonly reports “good public benchmark performance, marked decline on private test sets,” suggesting possible benchmark data contamination or overfitting to leaderboards—a leaderboard rank does not directly equal real-project performance;
3. Capability leaderboards vs. safety data: safety disclosures are often more informative. Industry observations show that removing cyber offense-and-defense-related corpora from training data causes model pass rates on exploit-class red-team tests to drop by roughly ten percentage points—indicating that capability and safety data are highly coupled, and safety-sensitive teams should include such data in their selection reports.

The recommended approach is to build your own two-dimensional evaluation:

# Minimal evaluation framework sketch: ability score + safety score + cost in one
cases = load_cases(["coding_private", "agent_terminal", "safety_redteam"])
for endpoint in candidates:
    ability = run_eval(endpoint, cases.ability, n=3)        # Multiple samples, take the mean
    safety  = run_eval(endpoint, cases.safety, judge="rule+llm")
    cost    = measure_cost(endpoint)                         # Record per-case cost
    report.record(endpoint, ability, safety, cost)

Evaluation dimensions should cover at least a subset of real business tasks, boundary and adversarial inputs, and record per-call cost—only then can you reach a three-dimensional conclusion of “capability—safety—cost.”

5. The Cost Ledger: How to Calculate Three Routes

The same open-weight model typically has three usage routes, with completely different cost structures:

Route	Explicit costs	Hidden costs	Suitable scenarios
Private self-hosted	GPU purchase/rental, storage and bandwidth	Model operations staffing, inference optimization, incident handling	Data must not leave the domain, high stable QPS
Official API	Higher unit price	No ops burden, but subject to rate limits and availability constraints	Rapid validation, moderate traffic
Third-party hosting	Unit price may be several times lower	Stability and data compliance must be assessed independently	Cost-sensitive, non-sensitive data

Two easy pitfalls:

1. A price gap is not equal to net benefit. With open weights available, third-party hosting prices may be significantly lower than the official API, but you need to verify the host’s quantization version (whether a lower-precision version is being resold), concurrency stability, and service-level commitments—the price gap may be eaten by retry rates and failure rates;
2. Private deployment requires looking at total cost. An inference cluster carrying TB-scale weights typically requires multiple multi-GPU accelerator servers forming tensor parallelism plus expert parallelism, layered with KV cache headroom and redundant nodes. Hardware depreciation, power, and dedicated operations staffing should all be amortized into per-call cost—do not just see “free weights.”

6. Selection Decision Framework: One Checklist to Determine the Route

Teams can self-check item by item in the following order:

1. Must data stay within the domain? Yes → only private deployment or dedicated instances can be considered;
2. Do you have model operations capability (inference framework tuning, capacity planning, failure drills)? No → prioritize API;
3. What are the average daily call volume and per-request length distribution? Long context with high concurrency will significantly raise private deployment’s VRAM and bandwidth budgets;
4. Might revenue or user scale touch the license thresholds? Yes → include commercial licensing and attribution obligations in the total ledger;
5. Do you need fine-tuning or deep customization? Yes → open weights only then make sense; pure API mode offers limited customization space.

A pragmatic rollout path: first use the official API to complete business validation while using a private evaluation set to establish capability and safety baselines; after traffic stabilizes and compliance requirements are clear, evaluate migrating high-frequency stable links to private deployment, forming a hybrid “API + private deployment” architecture.

7. Conclusion

The real value of open weights is not that the parameter scale has been refreshed again, but that choice is returned to the user: teams willing to invest in infrastructure and compliance costs gain a privately deployable, auditable, customizable option; teams that only want to consume capability continue to enjoy the low barrier of hosted APIs. What determines success is clear-headed accounting during selection—of license clauses, real evaluation data, safety boundaries, and full cost structure. That homework matters far more than chasing leaderboard ranks.