Open Weights and Hosted Agents: The Dual-Track Evolution of LLM Deployment

A leading AI model provider recently released the complete weights of a 3T-class open-weight model, alongside a Hosted Agents service tailored for enterprises and developers. This dual release sends two clear signals: first, model accessibility is shifting from API-only calls to weight-level access, enabling entities with sufficient compute to deploy and deeply customize models; second, the engineering complexity of AI agents is being encapsulated into services, freeing enterprises from building runtime frameworks from scratch. This article analyzes this evolution across four dimensions: model architecture, open-weight boundaries, hosted services, and deployment practices.

1. Model Architecture: Technical Implications of 3T-Class MoE and Million-Token Context

According to public disclosures, this 3T-class open-weight model utilizes a Mixture-of-Experts (MoE) architecture with a total parameter scale of 2.8 trillion and approximately 104 billion active parameters per inference. It features a 1-million-token context window and natively supports multimodal visual understanding. For developers, these parameters carry several layers of technical significance.

1. Separation of Total and Active Parameters

The core of the MoE architecture lies in “sparse activation.” The model possesses a massive total parameter count to carry broad knowledge and capabilities, but each forward pass activates only a small subset of expert networks. The combination of 2.8 trillion total parameters and ~104 billion active parameters means the computational overhead of a single inference is roughly comparable to a dense model in the 100-billion-parameter range, while its knowledge capacity is significantly larger. For deployment teams, this directly dictates the lower limit of VRAM usage—loading the weights requires accommodating all parameters, while the compute peak depends on the active parameter scale. This is the practical starting point for discussing quantization and caching optimizations.

2. Million-Token Context and VRAM Pressure

A 1-million-token context window allows the model to process lengthy documents, massive codebases, or complex multi-turn conversation histories in a single pass. However, long context is not “free”: the computation of attention mechanisms and the storage of the Key-Value (KV) Cache expand rapidly with sequence length. In ultra-long context scenarios, the KV Cache often becomes the primary bottleneck for VRAM usage. In engineering practice, techniques such as PagedAttention, KV cache quantization, and prefix cache reuse are typically required to control overhead. This is particularly critical in on-premises deployment scenarios.

3. Native Support for Multimodal Visual Understanding

Native multimodality means visual encoding capabilities are jointly optimized with language capabilities during the training phase, rather than being attached post-hoc. The business value lies in processing mixed text-and-image inputs (such as invoices, reports, UI screenshots, and quality inspection images) within a single model for understanding and reasoning, reducing the information loss and engineering complexity associated with chaining multiple models together.

4. Capability Iteration Path: Joint RL Over Base Retraining

Official statements indicate the model is not a base trained from scratch, but rather iterated upon an existing base through large-scale, cross-domain joint reinforcement learning (RL). The engineering significance of this path is that capability improvements derive mostly from post-training optimizations rather than base reconstruction. This ensures relatively friendly compatibility with existing adaptation tools and chains, keeping migration costs manageable.

2. Boundaries of Open Weights: Coexistence of Accessibility and Constraints

“Open weight” is a term that requires precise understanding. Developers receive downloadable model weights, enabling them to deploy, run, optimize, and customize the model independently. However, this is not equivalent to traditional open-source as defined by the OSI—training code and the original training datasets are not disclosed, and commercial use must adhere to the vendor’s custom license agreements.

For enterprises, this boundary implies a coexistence of “capability accessibility” and “compliance caution”:

• Accessibility Benefits: Data does not leave the local domain; inference can be completed within an intranet environment. Secondary fine-tuning can be conducted based on industry-specific data. Inference optimization and distillation can be tailored for specific tasks.
• Practical Constraints: The hardware threshold for a 2.8T-class MoE model is high. Full deployment requires substantial accelerator clusters and VRAM capacity. License agreements typically include specific provisions regarding commercial use and redistribution, necessitating a compliance review before implementation.

Therefore, open weights are more suited for enterprises with existing compute infrastructure and engineering teams, rather than lightweight “download-and-run” scenarios. For most organizations, open weights and hosted services form a complementary relationship: the former addresses deep customization and data compliance needs, while the latter alleviates engineering complexity and operational overhead.

3. Hosted Agents: Turning the Harness Framework into a Cloud Service

The Hosted Agent service launched alongside the weight release targets the “last mile” engineering challenges in agent deployment. In traditional development models, maintaining stable agent operations requires enterprises to build their own code execution environments from scratch, resolving issues like context management, tool calling, permission control, and task interruption recovery one by one. The vendor noted that building a reliable Harness infrastructure could take months, with ongoing maintenance costs.

Hosted Agents encapsulate these capabilities into a service: enterprises only need to define tasks, configure agents, and connect business tools, while the service provider maintains and upgrades the underlying runtime environment. Publicized features include 24/7 hosted operation, support for long-duration complex task execution, and multi-agent collaboration.

What Problems Must a Reliable Runtime Framework Solve?

From an engineering perspective, a robust agent runtime framework must include at least the following components:

# Pseudocode: Key components of an agent runtime framework
class AgentHarness:
    def __init__(self, model_client, tools, policy):
        self.model = model_client        # Model inference integration
        self.tools = tools               # Tool registration and invocation
        self.policy = policy             # Permission and security policies
        self.memory = ContextManager()   # Context and state management
    def run(self, task):
        plan = self.model.plan(task)                 # Task planning
        while not plan.done:
            action = self.model.next_action(plan)    # Decision making
            self.policy.check(action)                # Permission validation
            result = self.tools.execute(action)      # Tool execution
            self.memory.append(action, result)       # State persistence
            plan = self.model.reflect(plan, result)  # Reflection and replanning
        return plan.result

Each of these steps introduces additional complexity in a production environment: how to compress and retrieve context, how to retry failed tool calls, how to recover from checkpoints after long-task interruptions, how to orchestrate and facilitate communication between multiple agents, and how to audit sensitive operations. These are precisely the burdens the hosted service aims to absorb for the enterprise.

Two Typical Application Scenarios

• Internal System Integration: Undertaking business monitoring and IT operations response. For example, routing alert events to an agent for initial root cause analysis, executing standardized remediation actions based on predefined playbooks, and retaining comprehensive operation logs for auditing.
• User-Facing Product Embedding: Providing capabilities like investment research assistants or intelligent customer service. These scenarios demand higher standards for response latency, concurrent elasticity, and content safety, making the elasticity of the hosted runtime environment critical.

For enterprises, the core value of the hosted model is shortening the cycle from “prototype to production.” It transforms uncertain engineering investments into predictable service usage, allowing teams to focus on task definition, tool integration, and business logic.

4. Deployment and Adaptation Practices: Hardware Ecosystem and Optimization

After weights are open-sourced, the ability to quickly run inference on mainstream accelerator hardware is critical to actual usability. According to public information, hardware vendors completed 0-day inference adaptations on the day of the weight release. Leveraging mainstream inference frameworks, deployment verification was conducted on their accelerators. Addressing the characteristics of million-token long context and massive MoE architectures, the adaptation solutions mitigate VRAM pressure through caching optimization and quantization, providing hardware ecosystem support for on-premises deployment.

For teams planning self-deployment, the following practical path offers universal reference value:

1. Validate First, Scale Later

Prioritize verifying the inference pipeline on a single machine or small cluster. Confirm that weight loading, tokenizer, and inference framework versions match, and that multimodal input paths are functional. Then, evaluate whether throughput and time-to-first-token (TTFT) meet business requirements.

2. Prioritization of VRAM Optimization

• Quantization: Evaluate quantization strategies for weights and KV cache separately. Significantly reduce VRAM usage under the premise of acceptable accuracy loss.
• KV Cache Management: Enable PagedAttention and cache reuse; benefits are particularly evident in long-context scenarios.
• Expert Parallelism: MoE models can leverage expert parallelism strategies to distribute VRAM and compute pressure.
• Prefix Caching: Reuse prefixes for fixed system prompts and repeated contexts to reduce redundant computation.

3. Differentiate “Loading” vs. “Inference” in Capacity Planning

Weight loading consumes static VRAM, while inference peaks correlate strongly with concurrency and sequence length. Capacity planning should measure these separately to avoid linearly extrapolating single-request VRAM usage to high-concurrency scenarios.

4. Compliance and Licensing First

Complete license agreement reviews before commercial use and redistribution. Clarify usage boundaries for secondary fine-tuned products, data compliance requirements, and auditing obligations to avoid discovering compliance blockers late in the engineering process.

5. Selection Guide: Open Weights vs. Hosted Services

These two capabilities are not mutually exclusive; they correspond to different organizational stages and needs:

• Choose Open Weights If: You have strict data residency requirements, possess accelerator hardware and inference engineering teams, require deep industry fine-tuning or distillation, and demand complete control over the inference pipeline.
• Choose Hosted Agents If: You want to quickly validate the business value of AI agents, lack the personnel to maintain underlying runtime frameworks, deal with long task chains requiring reliable recovery mechanisms, or need multi-agent orchestration.
• Combined Usage: Use hosted services to carry user-facing online agents, and use locally deployed open-weight models for offline fine-tuning and internal inference of sensitive data. This forms a dual-track architecture of “online elasticity + offline control.”

Regardless of the chosen path, teams should first clarify three questions: Can the data leave the local domain? What are the peak concurrency and latency targets? Do we have the personnel to continuously maintain the inference infrastructure? Answering these three questions will largely determine the direction of your technical selection.

Conclusion

Open weights transform model capabilities from “service calls” into “held assets,” while hosted agents turn agent runtime frameworks from “self-built engineering” into “consumable services.” Both point to the same trend: the competitive focus of LLM deployment is shifting from singular model capabilities to a comprehensive system of “model accessibility + engineering controllability.” For developers and enterprises, understanding the physical implications of architectural parameters, clarifying the boundaries of open licenses, and evaluating the cost structures of hosting versus self-building provide more long-term value than merely chasing parameter numbers.