Open-Source LLM Report: Sparse Attention and Asynchronous Reinforcement Learning

    This article reviews a technical report on an open-source large language model that combines a 744B-parameter MoE base with 40B active parameters, latent attention, per-head orthogonalized optimization, multi-token prediction, dynamic sparse attention, and fully asynchronous agent reinforcement learning. It explains the engineering trade-offs behind these choices and offers practical guidance for evaluation and deployment.

    Open-Source LLM Report: Sparse Attention and Asynchronous Reinforcement Learning

    The most important takeaway from this technical report is not any single benchmark score. It is how parameter scale, attention mechanisms, optimization strategy, training paradigms, and reinforcement learning infrastructure are evaluated within one engineering system. Looking at the report section by section makes it clearer how open-source models are approaching top-tier closed-source models.

    Model Base: Scaling and Efficiency Increase Together

    The base model in the report uses a mixture-of-experts architecture, or MoE. It has 744B total parameters but activates only 40B parameters per inference pass, with 256 experts and 80 layers. Compared with the previous generation open-source base model, total parameters increased from 355B to 744B, while activated parameters increased only from 32B to 40B. This indicates that most of the new parameters were placed in the expert pool rather than uniformly added to the computation path for every token.

    The core value of MoE is that it allows a large total parameter count while keeping per-inference computation manageable. It lets the model store more knowledge while maintaining relatively stable inference cost. But MoE is not just parameter stacking. It also requires solving expert load balancing, cross-device communication, memory footprint, and training stability. As the number of experts grows, communication pressure increases, so what really determines the system ceiling is often not model size alone, but whether the optimizer and parallelism strategy match the architecture.

    Training data is also increasing. The report mentions that pretraining tokens rose from 23T in the previous generation to 28.5T, with 27T used in the pretraining phase and another 1.5T used for mid-training. Mid-training usually handles long-context extension, domain adaptation, and instruction-distribution alignment, serving as a key buffer between large-scale pretraining and post-training. Compared with simply adding more pretraining data, mid-training can strengthen targeted capabilities at a lighter cost.

    On a third-party composite intelligence index, the open-source model scored 50, becoming the first open-source model to reach that score. In a public battle arena, it also ranked first among open-source models in both text and code tracks, and sits in the same tier as current leading closed-source models. One notable signal is that the strongest capabilities still belong to closed-source models, but the gap between open-source and closed-source models has narrowed significantly. In other words, open-source models are not just making occasional breakthroughs on narrow metrics; they have entered the same competitive range on overall capability.

    Architecture Change 1: Multi-Head Latent Attention and Split Optimizers

    Multi-head latent attention is a critical part of this architecture. Its basic idea is not to store the full key-value cache directly, but to compress key-value information into latent vectors through low-rank projection. This significantly reduces memory consumption in long-context scenarios and makes the KV cache more compact during inference. Theoretically, the longer the context, the closer the memory overhead of standard attention comes to quadratic growth, while this type of compressed attention can bring the cost down.

    But engineering implementation is often more complex than theory. The report notes that during training, when an optimizer based on matrix orthogonalization was used with multi-head latent attention, its effectiveness fell behind a simpler grouped-query attention baseline. The issue is that different heads in multi-head attention may focus on different subspaces, with inconsistent gradient scales, directions, and statistical properties. If the optimizer orthogonalizes the entire projection matrix uniformly, it can erase differences between heads and distort the optimization direction.

    The solution is a per-head split orthogonalized update. In simple terms, instead of treating the whole projection matrix as one unit, each attention head maintains its own update direction and performs orthogonalization within its own low-dimensional subspace. This preserves the training-stability benefits of the orthogonalized optimizer while avoiding interference between heads.

    An illustrative pseudocode example is below:

    1. # Illustrative pseudocode: per-head split orthogonalized update
    2. def muon_split_update(head_params, head_grads, momentum, lr, beta):
    3. for h in range(head_params.shape[0]):
    4. update = beta * momentum[h] + head_grads[h]
    5. update = orthogonalize(update) # approximately preserve a "rotation" update direction
    6. momentum[h] = update
    7. head_params[h] -= lr * update

    In actual implementations, the orthogonalization step is usually completed with approximate methods such as Newton-Schulz iteration, rather than expensive singular value decomposition for every matrix. More importantly, this split aligns the optimizer with the attention structure, making convergence easier in large-scale parallel training with many experts and heads.

    Architecture Change 2: Multi-Token Prediction

    Traditional language model training usually predicts only the next token. Multi-token prediction adds multiple output heads on top of a shared backbone, each predicting a number of tokens after the current token. During training, the model no longer looks only one step into the future; it learns multi-step future information at the same time.

    This brings several clear benefits.

    First, each training position produces more supervision signals and denser gradients, which helps improve sample efficiency. Second, the model is forced to learn longer-span dependencies in advance, which is especially important for tasks such as code completion, mathematical derivation, long-document understanding, and tool calling. Third, during inference, the multi-token prediction heads can be used as draft models with speculative decoding to reduce the latency of step-by-step generation.

    However, multi-token prediction also has costs. Multiple output heads add memory and computation overhead; the prediction tasks need reasonable weighting, or the model may optimize only one head while others degrade. It also requires consistency between training and inference implementations; otherwise, the model may struggle to use the extra prediction capability during generation. Whether this design can be deployed therefore depends on training compute, model scale, and the supporting inference framework.

    Architecture Change 3: Dynamic Sparse Attention

    Long context is one of the core competitive areas for current large models. The computational complexity of standard attention rises quickly with sequence length, so directly extending context often means high training and inference costs. Dynamic sparse attention attempts to break this limitation.

    Its core idea is that for each query vector, a lightweight indexer first computes matching scores against key-value blocks, and then only a small number of the highest-scoring blocks participate in subsequent attention computation. Different queries may focus on different blocks, so this is not a fixed window or a simple sliding window, but data-dependent sparse selection. The direct effect is that attention computation changes from “look at all positions” to “focus only on the positions that truly matter.”

    The comparison data in the report is worth attention: on long-text tasks, sparse attention approached the performance of dense attention trained with nearly 1T tokens while using only about a 20B-token related training budget. If this result can be independently reproduced, it means long-context capability no longer has to be achieved by filling the entire pretraining corpus; instead, a smarter attention selection mechanism can deliver similar benefits at lower cost.

    Of course, sparse attention also introduces new engineering problems. Sparse selection itself requires extra computation, and indexer quality directly affects final results; incorrect sparse selection may cause the model to miss key information and reduce recall. Dynamic sparse attention is therefore not simply “computing less,” but a system-level trade-off among retrieval, compression, routing, and attention.

    Training Paradigm: Fully Asynchronous Agent Reinforcement Learning

    Model capability improvements come not only from architecture but also from training paradigms. The report focuses on a fully asynchronous agent reinforcement learning training framework.

    Traditional reinforcement learning pipelines are usually synchronous: collect a batch of trajectories, wait for all sampling nodes to return, perform gradient updates, and then continue sampling. This pattern gradually becomes a bottleneck in agent tasks because agent trajectories can be long and may include multiple tool calls, code executions, result judgments, and environment feedback. Each step may introduce uncertain latency, and synchronous waiting wastes a large amount of compute.

    A fully asynchronous framework decouples sampling, training, evaluation, and reward computation. Multiple sampling nodes continuously generate trajectories and write them to an experience pool, while the trainer consumes data from the experience pool without waiting for all nodes to finish the current batch. This design can significantly improve compute utilization and overall throughput.

    But asynchronous training also brings more engineering challenges. The trajectories received by the trainer may come from an old policy, creating distribution shift; if the shift is too large, the model can easily repeatedly update on incorrect samples, causing training instability. Frameworks therefore usually need replay strategies, importance weights, freshness control, or conservative update strategies to limit the impact of old data on the current policy. In addition, rewards in agent tasks are often sparse, delayed, and potentially noisy. Correctly computing returns, evaluating tool-call results, and judging whether a task is complete all require rigorous reward alignment design.

    The in-house reinforcement learning infrastructure mentioned in the report is essentially intended to engineer the above pipeline. It does not just support a single experiment; it enables agent reinforcement learning to run continuously like a production workload, including failure recovery, dynamic scaling, sample storage, monitoring and alerting, and version management. For open-source models, the importance of this infrastructure capability is no less than that of the model parameters themselves.

    Deployment and Practical Recommendations

    From a deployment perspective, 744B total parameters and 40B activated parameters mean the model still needs multiple GPUs or even multiple machines. Although activated parameters are lower, all expert parameters must reside in memory, so memory planning cannot rely only on activated parameters. The inference framework also needs to support MoE expert parallelism and cross-device communication, and ideally should support custom operators for multi-head latent attention and sparse attention.

    For enterprise users, it is advisable to evaluate from four dimensions:

    • Effective throughput in real business scenarios, not just single-request latency;
    • Accuracy and memory behavior in long-context scenarios;
    • Stability in agent tool calling and multi-turn tasks;
    • Whether inference cost can be effectively controlled as business scale grows.

    Evaluation should also not rely on a single leaderboard. Composite indexes and public arenas reflect overall capability, but real business scenarios often have their own data distributions. A safer approach is to place the open-source model into a continuous integration pipeline and build an internal regression evaluation set covering long documents, code, tool calls, and adversarial inputs. The stronger the model, the more the evaluation system must keep up; otherwise, it is difficult to tell whether an upgrade is a real capability improvement or merely a match with certain test-distribution preferences.

    Overall, the core signal from this technical report is that open-source models are advancing “larger, longer, and more automated training” simultaneously. Closed-source models still hold the highest capability point, but the open-source camp is clearly accelerating its catch-up. For developers, the key is not to chase a particular name, but to understand the costs, benefits, and boundaries behind these technical choices, and then decide whether to incorporate them into their own technology stack.