Deploying the Largest Open-Weights Model Yet: Memory, Licensing, and Engineering Constraints

    A newly released 2.8T-parameter multimodal agent model has 104B active parameters, 1.56TB of MXFP4 weights, a 1M-token context window, and a custom license that is not open source. This article breaks down its architecture, quantization constraints, benchmark profile, licensing obligations, and the memory, parallelism, and operational requirements needed to deploy it.

    1. What happened: the largest open-weights release to date

    On November 4, a research organization released the full weights of its flagship multimodal model, following through on an earlier commitment. The repository was closed in the morning and became publicly available in the evening, with no application required.

    The release contains 112 files and 96 tensor shards, totaling about 1.56TB. At the time of release, it was the largest open-weights model to date, and it is not in the same scale class as earlier open models.

    The timeline is also notable. On October 24, the model first became available through a cloud API and web interface, with a clear commitment that the full weights would be released by November 4. The release happened on schedule. This “service first, weights later” pattern is becoming common for leading models: collect feedback under real production load first, then publish the weights, avoiding repeated restructuring of weight files.

    For engineering teams, the important information is not only “what was released,” but three engineering facts that become visible once the weights land:

    1. What structure is inside the checkpoint?
    2. What does the license actually allow?
    3. Can your infrastructure run it?

    The rest of this article examines each point in turn.

    2. Model structure: 2.8T total parameters and 104B active parameters

    According to the official model card, the model is a natively multimodal agent model, continued from the previous-generation base model. Its key structural parameters are:

    Dimension Value Engineering implication
    Total parameters 2.8T Determines weight size and minimum memory capacity
    Active parameters per token 104B Determines compute required per forward pass
    MoE experts 896, with 16 selected per token Determines routing and expert-parallel partitioning
    Shared experts 2 Every token passes through them; they carry general capabilities
    Layers 93 Affects KV-cache growth and pipeline parallelism
    Context length 1M tokens Long context creates memory and scheduling pressure
    Vision encoder Proprietary multimodal encoder Image inputs require extra compute and preprocessing

    Compared with the previous generation, there is only one structural change: in 93 layers, full attention in 24 layers has been replaced by sparse attention, referred to as DSA.

    This is not a performance-tuning tweak. It is a compute design for million-token context. Full attention scales quadratically with sequence length. At a 1M-token context, KV-cache reads can become the dominant bottleneck. Sparse attention reduces the effective computation range in those layers to local windows or compressed representations, lowering the marginal cost of long context.

    There is a counterintuitive point worth emphasizing:

    With 2.8T total parameters and 104B active parameters, this model is “memory-heavy, compute-light.”

    MoE routing means each token passes through only 16 experts, so per-token compute is roughly comparable to a dense model in the 100B class. However, the weights of all 896 experts must remain resident in high-bandwidth memory, or be swappable on demand. Otherwise, every routing decision would require reloading weights.

    This determines the deployment strategy: compute can be purchased somewhat more conservatively, but memory capacity and memory bandwidth cannot be reduced in the same way.

    3. Quantization format: why there is only one MXFP4 version

    The weights are stored in 4-bit micro-scaled floating point, MXFP4, while activations use 8-bit floating point, MXFP8. The model is the result of quantization-aware training, not post-training quantization. The organization released only this version. There is no BF16 version and no FP8 version.

    These three facts need to be read together:

    • What QAT means: The model “knew” during training that it would run in 4-bit. Quantization error has already been absorbed into the weights during training. Do not expect re-quantizing to 8-bit to “make it more accurate.” You do not have a high-precision master copy. Dequantizing upward only recovers the original precision and provides no real benefit.

    • MX formats are block-level micro-scaling formats: A group of low-bit mantissas shares a scaling factor. MX is a public quantization format standard, and hardware support is expanding, but implementation maturity varies across accelerators.

    • The only realistic option is hardware adaptation: Since there is no BF16 fallback, “can it run?” is almost equivalent to “does your accelerator natively support the compute path for MXFP4 weights and MXFP8 activations?” Hardware without support can only rely on simulated dequantization, which can reduce throughput by an order of magnitude and make the deployment effectively unusable.

    4. How to read the vendor performance claims

    The official comparisons are vendor-reported, and the model wins some categories while losing others:

    • Coding and terminal tasks: On terminal-style benchmarks, it scores 65.7%, slightly below a closed flagship model at 66.4% and above another closed flagship at 58.2%. On a long-horizon software engineering benchmark, it scores 51.7%, clearly above the comparison model’s 46.3%.

    • Clear weaknesses: On a scientific terminal benchmark, it scores only 38.1%, compared with 63.3% for the comparison model. On a difficult subject-reasoning benchmark, it scores 39.4%, compared with 60.9%.

    • The organization says that 16 of 37 benchmarks exceed the comparison model, meaning that most benchmarks do not show a lead. A third-party evaluation reportedly gives it a composite intelligence score of about 55, ranking fourth globally.

    The correct way to read this is as a capability profile, not a leaderboard:

    Strong at agents and code execution; weak at scientific knowledge and rigorous reasoning chains.

    For business use, this suggests suitability for automated operations, code generation and modification, tool-calling orchestration, and long-document processing. It is not a good fit for scientific question answering or high-accuracy knowledge engines without additional validation.

    More importantly, real self-hosted performance depends on sampling parameters, prompt construction, context assembly, and tool-calling implementation. Vendor numbers cannot replace your own evaluation. A practical approach is to build 100–300 representative use cases that match your business, run them first, and then decide.

    5. License: open weights does not mean open source

    This is one of the most important points to examine separately. The repository uses a custom license. The hosting platform labels the license as “other/custom,” not MIT, Apache, or another standard open-source license.

    Notably, the license text is word-for-word identical to the previous generation’s license, with only the name changed.

    The main grant is close to an MIT-style license: it allows free use, copying, modification, distribution, fine-tuning, and commercial use. However, two additional restrictions apply to large-scale external businesses:

    1. Entities that provide the model as a service and have cumulative revenue above USD 20 million over a 12-month period must sign a separate agreement before commercial use.
    2. Commercial products trigger additional obligations when monthly active users exceed a certain threshold. The exact number must be checked against the official license text, not secondhand summaries.

    From an engineering and compliance perspective, three concepts need to be distinguished:

    • Open Source: Requires a license that satisfies the Open Source Definition, typically including patent grants and no discrimination against fields of use. A custom license with revenue and user-scale thresholds does not satisfy the OSI requirement against restricting fields of use, so it is not open source.

    • Open Weights: Weights can be downloaded, run locally, and fine-tuned, but usage is constrained by the license. This is the true position of most “open” models.

    • Open data / open training code: Neither is included in this release.

    Practical compliance advice:

    • Include the license text in your artifact approval process.
    • If your product is MaaS and approaches the revenue threshold, begin commercial discussions early.
    • Record the monthly-active-user and revenue thresholds in a compliance register, and review them periodically as the business grows rather than confirming them only once.

    6. Can you run it locally? The memory math behind 1.56TB

    Start with a verifiable estimate. With 2.8T total parameters and 4-bit weights, roughly 0.5 bytes per parameter, the theoretical weight size is about 1.4TB. The difference between 1.4TB and the actual 1.56TB comes from shard alignment, a small number of unquantized tensors such as embeddings and normalization parameters, and metadata.

    In other words, almost all of the weights must reside in the accelerator’s high-bandwidth memory.

    1. def estimate_memory(total_params=2.8e12, bits=4,
    2. layers=93, kv_bytes_per_token=24*1024,
    3. seq_len=131072, concurrency=8, overhead=0.08):
    4. weight_bytes = total_params * bits / 8
    5. kv_cache = kv_bytes_per_token * seq_len * concurrency
    6. total = (weight_bytes + kv_cache) * (1 + overhead)
    7. return {"weights_GB": weight_bytes / 1e9,
    8. "kv_cache_GB": kv_cache / 1e9,
    9. "total_GB": total / 1e9,
    10. "min_cards_192G": total / (192e9) + 1}

    kv_bytes_per_token is an order-of-magnitude estimate. The real value depends on the attention implementation. If the model uses a compressed latent representation, it may need only a few hundred bytes per token per layer. If it uses ordinary KV cache, the requirement can be one or two orders of magnitude larger. Treat this value as a placeholder that must be replaced by profiling with your real inference framework.

    The resulting conclusions are hard constraints:

    • With more than 1.4TB of weights, accelerators at the 192GB-per-card level need at least eight cards just to hold the weights, with no headroom. Eight cards are a memory floor, not a recommended production configuration. Production planning should reserve 10%–20% for KV cache, communication buffers, fragmentation, and activations.

    • A 1M-token context is not a free feature. Even under optimistic assumptions for compressed attention, KV cache for long sequences can reach tens of gigabytes. You must trade off throughput against context length.

    • Memory bandwidth determines throughput. During decoding, each generated token requires reading the weights corresponding to the activated experts. Total bandwidth demand is much higher than for a dense small model. As a result, single-user decoding speed is naturally lower than that of a same-generation dense model. Its advantage is in batch scenarios, not low-latency interactive use.

    7. Engineering deployment checklist

    7.1 Parallelism strategy

    • Tensor parallelism, or TP, is the main path within eight cards, but excessively high TP degree amplifies communication overhead. In MoE settings, TP is often combined with expert parallelism, or EP.
    • Expert parallelism must distribute the 896 experts evenly across cards. The two shared experts are hotspots and are often replicated across cards rather than split.
    • If the cluster spans multiple nodes, keep the TP group for a single request within one node whenever possible. Leave cross-node EP communication to all-to-all operations, avoiding extra hops.
    1. # MoE routing and expert parallelism (logical sketch)
    2. logits = gate(x) # [tokens, 896]
    3. topk = top_k(logits, 16) # select 16 experts per token
    4. weights = softmax(topk.logits) # routing weights
    5. tokens_per_expert = bincount(topk.idx, minlength=896)
    6. tokens_per_rank = all_to_all_scatter(tokens_per_expert, ep_size)
    7. y = sum(weights * experts[owner(topk.idx)](x)) + shared_experts(x)

    7.2 Weight loading and validation

    Ninety-six shards mean that downloading and loading require a resumable and verifiable process. At minimum, you should ensure:

    • The shard manifest matches the actual files one-to-one.
    • Downloads support resuming after interruption.
    • After loading, sample tensors are numerically validated.

    This helps avoid the hardest failure mode to diagnose: the model starts successfully but produces incorrect results.

    1. import json, os
    2. def check_shards(repo_dir: str, index_file: str = "index.json") -> None:
    3. index = json.load(open(os.path.join(repo_dir, index_file)))
    4. shards = sorted(set(index["weight_map"].values()))
    5. missing = [s for s in shards if not os.path.exists(os.path.join(repo_dir, s))]
    6. zero = [s for s in shards
    7. if os.path.exists(os.path.join(repo_dir, s))
    8. and os.path.getsize(os.path.join(repo_dir, s)) == 0]
    9. print(f"shards={len(shards)} missing={missing} empty={zero}")
    10. check_shards("/mnt/models/open-weights-2p8t")

    7.3 Common cloud deployment pattern

    On public clouds, this type of workload usually combines several vendor-neutral capabilities:

    • Object storage for weight persistence and distribution, often paired with CDN to reduce multi-region pull time.
    • Container platforms for orchestrating inference instances and elastic scaling.
    • Logging and monitoring services for throughput, time-to-first-token, and memory-utilization observability.
    • Message queues for queuing long-running requests and smoothing traffic spikes.

    Product names vary by provider. The key design principle is to decouple weight distribution from inference autoscaling. Preheat weights into a local cache before shifting traffic. Otherwise, a sudden scale-out event can be overwhelmed by the need to pull 1.5TB of weights.

    7.4 Cost perspective

    When comparing a self-hosted eight-card cluster with calling a cloud API, calculate three costs:

    1. Utilization: In low-concurrency interactive scenarios, expensive memory may sit idle most of the time.
    2. Operations: Expert-parallel tuning and long-context OOM debugging require dedicated engineering effort.
    3. Licensing: Crossing revenue or user-scale thresholds can introduce commercial costs.

    A practical rule of thumb: self-hosting is usually more cost-effective for stable, high-concurrency batch workloads. Cloud services are more economical for low-concurrency scenarios or workloads that frequently switch between models.

    8. Conclusion and action items

    • This is not “open source”; it is “open weights.” Run legal review before launch: check the custom license text and confirm whether the revenue and monthly-active-user thresholds are triggered.

    • Architecturally, this is a memory-heavy, compute-light MoE model: 896 experts and 104B active parameters. Procurement budgets should prioritize memory capacity and memory bandwidth rather than peak compute.

    • The 1.56TB weight size makes eight cards a memory floor, not a comfortable production baseline. Plan for 10%–20% headroom, and run separate capacity tests for long context.

    • Four-bit quantization with no high-precision fallback makes hardware compatibility a hard gate. Validate single-card compatibility before attempting large-scale deployment.

    • The capability profile favors code and agent workflows over scientific reasoning. Before selection, run regression tests with your own business evaluation set instead of relying on vendor comparison tables.