The Technical Path Behind a New Generation of Cost-Effective Multimodal Models

    As large language models shift from showcasing capabilities to powering production workloads, enterprises increasingly care less about benchmark leaderboard scores and more about whether a model can reliably handle real tasks at a controllable cost. This article examines the design choices behind a new-generation multimodal model—224B total parameters with only ~12B activated per inference, support for multimodal input, and a context window of up to ~1M tokens—and explains how sparse activation, hybrid attention, KV cache compression, self-distillation, reinforcement learning, quantization, and inference kernel optimization work together to balance quality, speed, cost, and operational complexity.

    As large language models move from “capability demos” toward “scaled deployment,” enterprise users no longer focus primarily on top leaderboard scores. Instead, they ask whether a model can consistently complete real tasks at a controllable cost. In scenarios such as document editing, presentation building, spreadsheet processing, visual understanding, and multi-step automation, a model’s practical value is typically determined by four factors together: task completion quality, response speed, per-call cost, and deployment complexity.

    A recently released new-generation multimodal model was designed around exactly this goal. It has roughly 224B total parameters, including about 64B enhanced parameters, but only ~12B parameters are actually activated per inference. It also supports multimodal input and a context window of up to ~1M tokens. Its core philosophy is not simply stacking parameters, but achieving a better balance between model quality and operating cost through sparse computation, cache compression, training strategy adjustments, and inference kernel optimization.

    1. From “Parameter Scale” to “Effective Compute”

    Traditional models increase total parameters to boost representational capacity, but in real inference not all parameters need to participate simultaneously. A sparse Mixture-of-Experts (MoE) architecture divides the model into multiple expert modules and routes each input to only a subset of them, giving the model large total capacity while keeping per-inference compute under control.

    This model has 224B total parameters but activates only ~12B per request, meaning the parameters actually involved in a single inference are far fewer than the total. This design delivers three benefits:

    1. Lower compute overhead: Fewer matrix operations per token reduce single-token inference cost.
    2. Reduced memory pressure: While the full model still requires sensible parameter management, runtime activation state and compute resource demands are easier to control.
    3. High expressiveness maintained: Different experts can specialize in code, long text, structured tasks, visual understanding, or tool calling, giving the model greater knowledge capacity for complex tasks.

    Note that sparse activation is not the same as simply trimming parameters. Router stability, expert load balancing, expert collapse during training, and capability transfer across tasks all directly affect final results. Training sparse models therefore usually involves many experiments at increasing scale to progressively validate the relationships between model capacity, data volume, and compute.

    2. Hybrid Attention and Long-Context Design

    Supporting million-token context lengths means the model must not only process more input but also address the rapid growth of attention computation and the KV cache.

    Standard full attention is computationally and memory intensive on long sequences. To address this, the model introduces a hybrid attention mechanism that combines different attention structures: strong attention is retained for local content requiring precise associations, while information that is distant but does not require token-by-token interaction uses more efficient compressed or low-rank representations.

    The key value of this architecture is that it:

    • Preserves key semantics across long documents;
    • Reduces attention computation over long contexts;
    • Shrinks the KV cache that must be kept during generation;
    • Improves stability for long-document QA, cross-page information extraction, and multi-turn task execution.

    During autoregressive generation, the KV cache is often a major driver of memory usage. According to the release information, this model’s KV cache is about 1/24 the size of the previous generation’s and 1/6 the size of a larger model’s. With a smaller cache, the same hardware can host more concurrent requests and is less likely to suffer throughput degradation caused by memory exhaustion.

    However, long context does not mean lossless understanding of all input. In engineering practice, context filtering, hierarchical summarization, key-segment recall, and task state management are still needed. For enterprise applications, the more sensible approach is usually “long-context capability plus retrieval and compression,” not stuffing all historical content into the prompt indiscriminately.

    3. Improving Effective Output Quality Through Post-Training

    Pretraining alone can rarely guarantee strong performance on real work tasks. Enterprise scenarios care about whether the model correctly understands objectives, calls tools, generates deliverable outputs, and corrects errors when they occur. Post-training therefore needs to optimize around closed task loops.

    This model uses self-distillation to extract knowledge, reasoning patterns, and task-handling experience from a stronger teacher model, then further optimizes outputs with reinforcement learning. The advantage of self-distillation is that it transfers part of a large model’s capability to a cheaper inference model while reducing full dependence on human annotation.

    Reinforcement learning focuses on end-to-end outcomes. For example, in a task involving multiple rounds of operations, the model must not only produce a correct answer but also complete file reading, content analysis, tool calls, result verification, and final output. If the model can complete the task via a shorter path, it simultaneously reduces token consumption, tool call counts, and user waiting time.

    According to the release information, after reinforcement learning the output length of some end-to-end tasks dropped by roughly 50%. This is not about simply compressing answers; it comes from cutting ineffective reasoning, repeated explanations, and redundant steps so the model reaches a usable result faster. For applications billed by input/output tokens, shorter outputs translate directly into lower costs.

    4. Evaluation Should Shift From Leaderboard Scores to Task Outcomes

    Public benchmarks reflect a model’s performance on specific capability categories but cannot fully represent the real experience of enterprise users. Office automation scenarios carry many implicit requirements: whether formatting is preserved, whether tables break, whether presentations are well structured, whether audio and video content is accurately understood, and whether the model delivers within a reasonable time.

    The developer therefore built an internal evaluation set targeting typical work tasks, covering:

    • Presentation creation and modification;
    • Document editing and formatting;
    • Spreadsheet repair and data processing;
    • Collaboration on structured content such as Markdown;
    • Audio and video content analysis;
    • Visual understanding of images and pages;
    • Multi-step tool calling and automated operations.

    Such evaluations should not merely record whether answers are correct; they should also assess three metrics:

    1. Artifact quality: Whether the generated files, spreadsheets, or documents are directly usable.
    2. End-to-end duration: Total time from task submission to final delivery.
    3. Cost per task: Including input tokens, output tokens, cache hits, tool calls, and infrastructure consumption.

    In practice, a model with slightly lower leaderboard scores is often more commercially valuable if it completes tasks at lower cost and in less time.

    5. Inference-Side Optimization Determines Large-Scale Adoption

    Once training is complete, the inference system largely determines the actual user experience. The model applies multiple inference optimizations, including efficient fused operators, low-bit quantization, and scheduling optimizations for distributed inference.

    4-bit quantization can significantly reduce weight storage and memory footprint. A sound quantization scheme must balance compression ratio against precision—it cannot simply chase the lowest bit count. Sensitive layers, attention layers, and output layers typically require differentiated precision strategies to limit the impact of quantization error on generation quality.

    Fused kernels reduce intermediate tensor reads/writes and kernel launches by merging multiple compute steps. For generative models, the decoding phase generates only a few tokens at a time, so memory access and scheduling overhead are often prominent—efficient kernels directly improve tokens generated per second.

    Release data shows the model decodes at over 150 tokens per second. Higher speed not only shortens user wait times but also lets a single server serve more concurrent requests, amortizing hardware costs. For enterprise deployment, single-machine performance still needs to be combined with dynamic batching, request prioritization, cache reuse, failover, and elastic scaling to become stable service capability.

    6. Cost Advantages Come From Full-Stack Optimization

    The published pricing is roughly ¥0.5 per million input tokens and ¥2 per million output tokens; with cache hits, relevant input costs drop further to about ¥0.02 per million tokens. Compared with larger models, the core advantage is not just a discount on the price sheet but cost reduction across the entire pipeline—training, inference, and task execution.

    In internal comprehensive measurements, the model’s cost per task is about 1/13 of a larger preview model, 1/20 of another high-spec model, and 1/8 of a certain premium-tier model. Importantly, per-task cost cannot be inferred from token price alone; you should also consider:

    • Whether multiple retries are needed;
    • Whether excessively long intermediate outputs are produced;
    • Whether external tools are called frequently;
    • Whether oversized contexts invalidate the cache;
    • Whether human intervention is needed for corrections;
    • Whether a usable deliverable can be produced in one pass.

    Effective cost evaluation should therefore use the yardstick of “how much does it cost to complete one task,” not just per-million-token pricing.

    7. Real-User Blind Tests Approximate Application Value

    In a blind test covering 85 internal users, evaluation spanned 27 categories and 259 real tasks. Results showed user satisfaction approaching that of a lightweight high-performance model, while per-task cost was significantly lower and completion times shorter.

    Blind tests are valuable because they complement automated benchmarks. Users typically care about:

    • Whether the model understands natural-language objectives;
    • Whether prompts need repeated rewording;
    • Whether it generates files that meet specifications;
    • Whether it handles exceptions gracefully;
    • Whether they would keep using it in daily work.

    For enterprise-oriented model evaluation, combining automated testing, expert scoring, and real-user blind testing is recommended. Automated tests ensure repeatability, expert scoring ensures professionalism, and real users reveal how usable the model actually is within complex workflows.

    8. Use Cases Best Suited for Early Adoption

    Based on its technical characteristics, this type of model is particularly well suited to:

    1. Long-document processing: Summarizing, comparing, and Q&A over contracts, policies, reports, and project materials.
    2. Office file generation: Creating documents, spreadsheets, and presentations from natural language.
    3. Multimodal content understanding: Extracting key information from images, pages, audio, or video.
    4. Enterprise knowledge assistants: Answering internal knowledge questions combined with retrieval systems.
    5. Automated agent tasks: Executing multi-step operations and producing final deliverables.
    6. High-concurrency online services: Leveraging low activated parameters and fast decoding to control service costs.

    During deployment, model size should still be matched to the task type. Use lighter models for simple classification, summarization, and short-text rewriting; for complex reasoning, multimodal understanding, and long-running workflows, focus on evaluating context capability, tool-calling reliability, and error recovery.

    Conclusion

    The core of a cost-effective model is neither chasing the lowest price nor blindly compressing parameters, but establishing a systemic balance among model capacity, activated compute, context management, training quality, inference speed, and task completion quality.

    This new model combines sparse activation, hybrid attention, cache compression, self-distillation, reinforcement learning, quantization, and inference kernel optimization to achieve strong overall capability at low usage cost. For developers and enterprise users, evaluating such models should shift from “model scores” to “task outcomes, completion time, and cost per unit of output.”

    Going forward, truly competitive models will not simply be the ones with more parameters or higher leaderboard rankings, but the ones that can reliably complete real work and run at scale at a sustainable cost.