2.8T Open-Weight Model Goes Live: The 8-GPU Floor and Where the License Ends
An open-weight multimodal agent model with roughly 2.8 trillion total parameters (about 104 billion activated per token) has shipped with downloadable weights and a 1.56TB file set. This breakdown covers what the release actually contains, how the MoE architecture (896 experts, 93 layers, 1M-token context, sparse attention, built-in vision encoder) changes deployment math, why the only published version is 4-bit quantized, how coding and agentic strengths compare with weaker science and reasoning results, and why the custom license makes large-scale commercial use a legal question rather than a purely technical one. It also maps out the enterprise service architecture, a sample distributed inference configuration, and a selection framework that weighs scenario fit, total cost, license compliance, and operational control instead of parameter count alone.
1. Open Weights Enter the “Very Large Model” Tier
A multimodal agent model with roughly 2.8 trillion total parameters and about 104 billion activated parameters per inference pass has been released with open weights. Unlike models marketed as “fully open source,” this one sits closer to “open weights with limited constraints”: anyone can download the weight files and self-host, but the license terms, commercial boundaries, and hardware floor are all noticeably higher.
From an engineering standpoint, the core messages of this release come down to three points:
- A new scale class. Around 2.8 trillion total parameters, with roughly 1.56TB of weight files — far beyond a typical open-weight release.
- A materially higher deployment floor. Even with 4-bit quantization, local execution starts at a minimum of 8 high-memory accelerator cards.
- Not a standard open source license. The main grant is permissive, but large-scale commercial services and very large user-facing products carry additional constraints.
In other words, this is no longer just a “reproducible research model.” It behaves much more like a production candidate that requires engineering evaluation, cost modeling, and legal review before adoption.
2. What the Weight Release Looks Like: File Size and Acquisition
The weights are published in a public repository and can be downloaded without an application process. The repository contains roughly 112 files, of which about 96 are weight shards, totaling around 1.56TB. For an individual user, that volume is no longer something a single drive, a single GPU, or a single machine handles comfortably — it is closer to an enterprise-grade model asset distribution exercise.
Two factors drive the size:
- The sheer parameter count. Even after 4-bit quantization, 2.8 trillion parameters still produce a very large static weight footprint.
- Multimodal plus long-context structure. A vision encoder, long-context cache, expert routing, and shared experts all add deployment complexity.
On the acquisition side, a public repository is not the same as an easy transfer. Practical downloading and verification require a stable connection, high-bandwidth links, checksum validation, and resumable downloads. Users on constrained networks are better served by mirror syncing, shard-by-shard downloads, or offline physical copies.
3. Model Architecture: MoE, Multimodality, and Long Context
Per the model card, the model is defined as a natively multimodal agent model, continued from the previous generation. Its core structural traits:
3.1 Total Parameters Separated from Activated Parameters
Total parameters are about 2.8 trillion, but each token activates only about 104 billion during inference. This is a classic mixture-of-experts (MoE) design.
The goal is not to run every parameter on every token, but to use a routing mechanism to dynamically select a subset of experts for each computation. Capacity stays high while per-token compute drops.
3.2 896 Experts Plus Shared Experts
The model contains 896 experts. Each token selects 16 of them, plus 2 shared experts. The split works like this:
- Routed experts handle fine-grained capability across different tasks and semantic spaces.
- Shared experts carry more general, foundational capability, reducing fragmentation between experts.
The upside is large capacity and strong trainability. The cost is much heavier demands on expert scheduling, memory management, communication overhead, and cache strategy during inference.
3.3 93 Layers and Million-Token Context
The model has 93 layers and supports up to 1,000,000 tokens of context. Long context improves document analysis, codebase comprehension, and complex task planning, but it introduces two practical problems:
- KV cache growth. The longer the context, the more memory the inference cache consumes.
- Sharp memory pressure. Long context amplifies memory requirements, and the effect is more pronounced under batched requests.
So million-token context is not free to use. In practice it needs to be paired with paged caching, cache quantization, context truncation, sliding windows, and asynchronous scheduling.
3.4 Vision Encoder and Multimodal Capability
A built-in vision encoder lets the model accept image input. For enterprise use, multimodality widens the addressable surface, for example:
- Image understanding;
- Document layout recognition;
- Mixed table and image input;
- Agentic tasks driven by visual information.
But multimodality also raises memory consumption and preprocessing complexity, including image tiling, encoding, feature injection, and context assembly.
3.5 Attention Optimization
One of the main structural changes versus the previous generation is replacing some full-attention layers with sparse attention, aiming to cut compute cost in long-context scenarios. The value of sparse attention:
- Reduces the quadratic cost of full-sequence attention;
- Improves throughput on long text, long code, and long documents;
- Makes million-token context engineering-viable.
Sparse attention also raises the bar on implementation quality. Whether the inference framework supports it correctly, how the cache is organized, and how stable long sequences remain all affect final results.
4. Quantization: 4-Bit Only, No Higher-Precision Version
The weights are stored at 4-bit with 8-bit activation compute, and they are the product of quantization-aware training. The official release provides only this one version; there is no higher-precision weight set.
For users this means:
- Memory-friendly. Compared with original high-precision weights, the 4-bit version is far more suitable for local or private deployment.
- Accuracy tradeoff needs measurement. Quantization may reduce capability, and the effect is more sensitive in math, scientific reasoning, and long-chain code tasks.
- Fine-tuning needs care. Anyone planning to continue training on the 4-bit weights should assess gradient stability, calibration data, and training framework compatibility.
- Not for extreme-accuracy use cases. If a workload demands very high output stability, run comparison tests up front.
For most inference applications, 4-bit quantization is a reasonable way to cut deployment cost. But if the goal is a high-reliability production system, “it boots” is not the bar — output consistency, long-context stability, and complex-task completion rates are.
5. Capability Profile: Strong at Coding and Agents, Weaker at Science and Reasoning
On the vendor’s self-reported numbers, the model performs strongly on coding and agentic tasks:
- Roughly 65.7% on coding-execution tasks;
- Roughly 51.7% on long-horizon software engineering tasks;
- Around 38.1% on scientific reasoning tasks;
- Around 39.4% on general knowledge and reasoning tasks.
These figures say it is not an across-the-board leader. The task skew is clear:
- Strengths: code generation, terminal tasks, tool calling, agentic tasks, software engineering workflows;
- Weaknesses: scientific computation, complex reasoning, and some general knowledge tasks.
Third-party independent evaluation puts its overall intelligence index at roughly 55, near the global front. Keep in mind, though, that self-reported scores, independent scores, and real business outcomes can diverge. Enterprises should validate against their own business samples rather than lean on leaderboards alone.
A practical reframing: instead of asking “is it smart enough,” ask:
- Is it stable on the target task?
- Does it forget or hallucinate under long context?
- Does multimodal input degrade text reasoning?
- Does throughput under high concurrency fit the cost model?
- Does fine-tuning preserve the original behavior?
6. License Terms: Open Weights Is Not Fully Open Source
The license deserves its own section. The model ships under a custom license, not a standard permissive open source license.
The core logic:
- Permissive base grant. Free use, copying, modification, distribution, fine-tuning, and commercial use are allowed.
- Large-scale commercial services are restricted. If you offer the model as a service to third parties and cumulative revenue exceeds $20 million within 12 months, a separate agreement with the rights holder is required before commercial use.
- Very large products carry an attribution obligation. If a commercial product exceeds 100 million monthly active users, or exceeds $20 million in monthly revenue, it must prominently attribute the model’s source or model designation in the interface.
The practical reading: “open weights, but not fully unconstrained open source.” The differences from traditional open source licenses:
- Open weights does not mean open training data and training code;
- Commercial use allowed does not mean unrestricted large-scale commercial redistribution;
- Friendly to individuals and small teams; very large businesses need extra compliance review.
Enterprise users should confirm the following before deployment:
- Will it be classified as “offering the model as a service to third parties”?
- Will 12-month cumulative revenue cross the threshold?
- Is the product a commercial application above the MAU or revenue limit?
- Is model attribution required?
- Will weights be redistributed after fine-tuning?
- Is it embedded in a third-party platform or agent service?
- Does it involve resale, API wrapping, or subscription products?
If the model is headed into a commercial product — especially a scaled SaaS offering, an agent platform, a coding assistant, or an enterprise knowledge base — the license review should come before technical selection.
7. Local Deployment: Why 8 GPUs Are Only the Starting Point
Total parameters reach 2.8 trillion, but with only about 104 billion activated, per-token compute is not calculated against the full 2.8 trillion. The real deployment bottleneck comes from weight storage, memory allocation, KV cache, communication overhead, and the runtime framework.
7.1 The Weight Volume Alone Is Already High
The weight files come to roughly 1.56TB. Spread across 8 accelerators, that is about 195GB of weight shards per card on average — before accounting for:
- KV cache;
- Activations;
- Temporary buffers;
- The vision encoder;
- Inference framework overhead;
- Parallel communication buffers;
- Memory reserved by the system.
Eight cards is therefore the minimum entry point, not a guarantee that “eight ordinary cards will run it comfortably.” With common 80GB-class accelerators, total memory is 640GB — clearly insufficient to hold 1.56TB of weights, so more cards, higher aggregate memory, or higher per-card memory are required.
7.2 High Memory and High Bandwidth Are Hard Requirements
Hardware demands go beyond memory capacity:
- Inter-card interconnect bandwidth. Expert parallelism and tensor parallelism communicate constantly.
- Intra-node memory bandwidth. Weight loading and cache swap-in/swap-out are bandwidth-bound.
- Storage throughput. Loading 1.56TB of weights needs high-throughput storage.
- Network latency. In multi-node, multi-GPU setups, network latency shows up directly in inference latency.
If inter-card interconnect is weak, the model may load fine and still deliver poor throughput, high latency, and weak concurrency.
7.3 Long Context Amplifies Memory Pressure Further
A 1,000,000-token context sounds impressive, but running long contexts causes the KV cache to balloon quickly — especially in multi-turn dialogue, long-document QA, and codebase understanding, where cache management becomes critical.
Common optimization directions:
- KV cache quantization;
- Paged caching;
- Context sliding windows;
- Long-text segmented summarization;
- Request-level cache reuse;
- Inference batching and scheduling optimization.
7.4 Multimodal Input Adds Deployment Complexity
A vision encoder introduces image preprocessing, feature extraction, and cross-modal alignment. Compared with a text-only model, a multimodal model typically also needs:
- Image tiling and scaling;
- Image encoding cache;
- Multimodal context assembly;
- Interleaved visual and text ordering;
- Input validation and error handling.
All of this adds memory consumption and system complexity.
8. Deployment Architecture: From Single-Machine Inference to Enterprise Service
If you plan to self-host this class of model, do not treat it as a single inference program. Treat it as a model serving system. A reasonably complete enterprise architecture usually includes:
8.1 Inference Compute Layer
A cluster of high-memory accelerators supporting:
- Tensor parallelism;
- Expert parallelism;
- Pipeline parallelism;
- Dynamic batching;
- Request-priority scheduling.
8.2 Cache and Storage Layer
Given the weight volume:
- High-throughput parallel storage;
- Local cache acceleration;
- Weight shard verification;
- Multi-version canary loading;
- Model file integrity checks.
8.3 Service Gateway Layer
Responsible for:
- Authentication;
- Rate limiting;
- Quotas;
- Logging and audit;
- Request routing;
- Sensitive content filtering;
- Multi-model routing.
8.4 Monitoring and Operations Layer
Metrics to watch:
- GPU memory utilization;
- Inter-card communication latency;
- KV cache hit rate;
- Request queue time;
- Token throughput;
- Time to first token;
- Output completion rate;
- Error codes;
- Hardware indicators such as temperature, power draw, and card dropouts.
8.5 Security and Compliance Layer
Enterprise environments should also include:
- Data isolation;
- Prompt auditing;
- Output content moderation;
- Model invocation permission control;
- Commercial license compliance checks;
- Version change records.
9. Pseudocode: Distributed Inference Configuration
The pseudocode below shows the key parameter structure for deploying this class of model. Actual implementation depends on the specific framework and hardware, but the configuration logic is broadly transferable:
# Pseudocode: ultra-large-scale MoE multimodal model inference configmodel_config = {"total_parameters": "2.8T","activated_parameters": "104B","architecture": "MoE","expert_count": 896,"active_experts_per_token": 16,"shared_experts": 2,"layers": 93,"max_context_length": 1_000_000,"weight_storage": "4bit","activation_compute": "8bit","vision_support": True,"sparse_attention": True}deployment_config = {"gpu_count": 8,"memory_per_gpu": "high_memory_tier","parallel_strategy": ["tensor_parallel","expert_parallel","pipeline_parallel"],"kv_cache_policy": {"dtype": "low_precision","paged_cache": True,"max_cached_tokens": 200_000},"batching": {"continuous_batching": True,"max_batch_size": 32,"timeout_seconds": 30},"monitoring": ["gpu_memory","interconnect_latency","kv_cache_hit_rate","tokens_per_second","request_latency"]}
The point is not any specific implementation. It is that deploying a model at this scale forces several dimensions to be considered at the same time: weight volume, parallel strategy, cache management, batching capability, and observability.
10. Can Individual Developers Run It?
For the vast majority of individual developers, running it locally is not realistic:
- Memory cost is prohibitive. 1.56TB of weights requires very high aggregate GPU memory.
- Interconnect demands are strict. Multi-card communication noticeably affects usability and stability.
- Long context consumes heavily. Million-token context pushes costs up further.
- Tuning skill required. Quantization, parallelism, caching, and batching all demand engineering experience.
- The payoff may not justify the investment. Many tasks are already served by smaller models or hosted endpoints.
More sensible options:
- Use hosted inference services;
- Use managed model APIs;
- Use model routing, sending simple tasks to small models;
- Reserve the ultra-large model for genuinely complex tasks;
- Distill or fine-tune smaller models for high-frequency workloads.
11. Enterprise Selection: Parameter Count Is Not the Decision
When evaluating this class of model, build the decision framework on four dimensions.
11.1 Scenario Fit
Validate first:
- Code comprehension;
- Long-document analysis;
- Tool calling;
- Multimodal understanding;
- Agentic task orchestration.
If the business depends mainly on scientific reasoning or high-precision computation, evaluate cautiously.
11.2 Cost Model
Look past GPU purchase price to:
- Weight load time;
- Concurrent throughput;
- Additional cost of long context;
- Cache hit rate;
- Operational headcount;
- Failure recovery cost;
- Model update frequency.
11.3 License Compliance
Pay particular attention to:
- Whether you are providing a service to third parties;
- Whether you exceed revenue or MAU thresholds;
- Whether attribution is required;
- Whether commercial resale is permitted;
- Whether redistribution after fine-tuning is permitted.
11.4 Engineering Controllability
Including:
- Whether mature inference frameworks support it;
- Whether long context runs stably;
- Whether monitoring can be integrated;
- Whether canary rollout and rollback are feasible;
- Whether output risk can be controlled.
12. Closing: The Larger the Open Weights, the More Engineering Judgment Required
The significance of this release is not just a model with more parameters. It pushes ultra-large-scale models into real engineering deployment. 2.8 trillion parameters, roughly 1.56TB of weights, 4-bit quantization, million-token context, MoE expert routing, and multimodal input combine into a meaningful candidate for coding, agentic, and long-context workloads.
At the same time, the 8-GPU floor, the commercial boundaries of a custom license, the accuracy loss from quantization, and the cache pressure of long context all make the same point: open weights is neither zero-cost usage nor full open source.
For developers, the key question is not how to run the largest possible model, but whether it fits the specific task. For enterprises, the key is not parameter count or leaderboard position, but a combined assessment of effectiveness, cost, license, and operational capability. The value of ultra-large-scale open-weight models ultimately has to be proven inside real business systems.
