Why Open Weights Matter: From API Calls to Auditable Assets

When a large model is available only through an API, developers are limited to making calls and tuning prompts. Only when the full weights are released for download do engineering teams gain three genuinely new capabilities: private deployment, domain-specific fine-tuning, and model auditing. Recently, a mixture-of-experts (MoE) model with a total parameter count in the trillions published its complete weights, delivering on its earlier promise to open up by a set date. For mid-sized and large teams that need data compliance, predictable costs, or deep customization, this matters far more than simply having one more model to pick from.

From an engineering perspective, open weights mean a team can run the model inside its own VPC and connect it directly to internal knowledge bases, toolchains, and permission systems; perform continued pretraining or instruction fine-tuning on proprietary corpora; and carry out white-box analysis of weight distributions, quantization error, and routing behavior. None of these are possible with a pure API offering.

Model Specifications: 2.8T Total Parameters, 104B Active

The model’s core specifications can be summarized as follows:

• Total parameters: approximately 2.78 trillion (measured by weight files), with the repository occupying about 1.56TB;
• Active parameters: roughly 104B activated per token — a classic sparse MoE design;
• Context length: up to 1 million tokens;
• Input modalities: text and images, with a vision encoder of roughly 400 million parameters;
• Layer structure: 93 layers in total, of which 69 use a linear attention variant and 24 use gated sparse attention.

The essence of MoE is “large total parameters, small active parameters.” The 2.8T total determines the model’s representational capacity, while the 104B active count determines the per-token compute and memory bandwidth cost. The goal of this design is to maximize knowledge capacity while keeping inference costs under control.

The Sparse Attention Trade-off

The previous generation replaced full attention layers with sparse attention; this generation goes further by replacing the gated attention layers within full attention with gated sparse attention. The direct benefit is cheaper long-context modeling: at 1 million tokens, standard full attention scales quadratically with sequence length, while sparse approaches can substantially reduce effective computation. The trade-off is that the model needs stronger routing and gating mechanisms to avoid losing information — which is precisely why the gating design exists.

Expert Routing Configuration

The expert layers contain 896 experts, of which 16 are selected per token, plus 2 shared experts. Shared experts are always active and carry general knowledge; routed experts are chosen dynamically per token and carry domain-specific knowledge. The number of selected experts determines the actual compute per token and directly affects the communication overhead of expert parallelism (EP).

Quantization Strategy: MXFP4 Weights and MXFP8 Activations

Weights are stored in MXFP4 format and activations use MXFP8, with the team stating that quantization-aware training (QAT) was applied. These choices are worth unpacking:

• 4-bit microscaling floats for storage: this compresses the 1.56TB of weights into a size that is practical to download and load. Without quantization, the storage and memory requirements of trillion-scale parameters would multiply.
• 8-bit microscaling floats for activations: activation values are more sensitive to precision, and 8-bit strikes a balance between accuracy and bandwidth.
• QAT rather than post-training quantization: quantization-aware training simulates quantization error during training, allowing the weight distribution to adapt to low-precision representations. This is generally more stable than post-training quantization (PTQ) on complex reasoning tasks.

It is worth stressing that the core idea of the Microscaling format is block-shared exponents: elements are grouped together and share a single scaling factor, preserving dynamic range at extremely low bit widths. This is one of the mainstream technical routes for low-precision inference today.

Estimating the Hardware Barrier for Local Deployment

The hardware barrier for running such a model locally is high. One feasible configuration estimated by third parties calls for 8 high-memory accelerator cards with roughly 2.3TB of total memory, at an estimated hardware cost of about $550,000 to $650,000 for the machine. Note that this is a third-party estimate; no official hardware guidance has been published.

From an engineering standpoint, three factors deserve close attention during deployment:

1. Memory capacity: weights + KV cache + activation buffers. The KV cache overhead of a 1-million-token context is substantial and typically calls for paged attention and KV cache quantization.
2. Expert parallelism communication: 896 experts cannot fit on a single card, so expert parallelism must span multiple cards or even multiple nodes, making interconnect bandwidth the bottleneck.
3. Throughput-latency balance: batch size, expert load balancing, and continuous batching strategy all affect real-world QPS.

Inference Framework Support

Major inference frameworks have published deployment recipes, typically including weight loading scripts, parallelism configuration templates, and service startup parameters. The general workflow in practice looks like this:

# Illustrative workflow; refer to the framework documentation for exact parameters
# 1. Download the weights to local high-speed storage
# 2. Configure the tensor parallelism / expert parallelism degrees
# 3. Start the inference server and expose an OpenAI-compatible endpoint
# 4. Benchmark first-token latency and throughput, then tune batching parameters

Teams that prefer not to build their own cluster can use a hosted API instead. The model is generally compatible with mainstream API protocols and offers multiple reasoning-effort tiers as well as high-speed variants.

Agent Long-Horizon Session Evaluation: How to Read the Net Improvement Metric

The evaluation of this model introduces a dimension worth paying attention to: rankings based on real user agent sessions. The data comes from real long-horizon sessions, from which evaluators extract several types of signals:

• Whether users explicitly confirm that a task succeeded;
• Praise and complaints about the model;
• Response quality after user corrections;
• Whether shell commands executed successfully.

From these signals, a net improvement score is computed. In an evaluation built on more than 17,500 real long-horizon sessions, the model scored +5.1% net improvement, ranking first among open-weight models; the previous generation scored +0.91%. By subcategory, command execution reliability improved markedly (+7.9%), whereas the previous generation was negative on that item.

The value of this methodology is that it measures real performance in multi-turn, long-horizon, tool-calling scenarios rather than single-turn Q&A benchmark scores. For teams building agent applications, metrics of this kind are often more informative than static leaderboards.

Engineering Details of Multi-Turn Tool Calling

When running multi-turn tool calling on this class of models, one detail is easy to get wrong: the previous turn’s reasoning content must be passed back together with the tool call results. Otherwise, the model loses its chain of thought within the context. In addition, reasoning effort typically offers several tiers, and the default tier is set fairly high — production environments should adjust it according to their latency budget.

Deployment Recommendations

Taken together, this type of trillion-parameter open-weight MoE model is a good fit for teams that:

• Have private deployment and data compliance requirements;
• Need to fine-tune on domain-specific corpora;
• Need to audit model behavior in a white-box fashion;
• Run long-horizon agent workloads and are sensitive to tool-calling reliability.

Teams without cluster resources, or those that only need general-purpose Q&A, are better served by a hosted API, freeing them to focus on prompt engineering and toolchain integration.

Whichever path you choose, start with a small-scale validation: build an evaluation set from real business sessions and watch three things closely — information retention over long contexts, tool-calling success rates, and recovery after corrections. These three metrics often determine the end-user experience more than parameter count does.