Open-Source LLMs: Sparse Attention and Asynchronous Agentic Reinforcement Learning Explained

    This article breaks down the key engineering ideas behind a recent open-source large model technical report, focusing on large-scale Mixture-of-Experts (MoE) design, low-rank latent-space attention, grouped orthogonalization, multi-token prediction, sparse attention, and a fully asynchronous agentic reinforcement learning framework. Written for developers and architects, it explains how these techniques balance model quality, cost, stability, and scalability, along with the practical trade-offs, common pitfalls, infrastructure requirements, and evaluation practices needed to adopt them. It does not reference any specific vendor or product, staying at the level of general technical principles and engineering experience.

    1. Open-Source LLM Competition Enters the “Efficiency and System Capability” Phase

    For a long time, competition in large model capability was often understood as a race over parameter count, training data volume, and benchmark scores. But as models enter the trillion-parameter era, long-context scenarios become widespread, and agentic tasks grow in number, simply scaling up parameters is no longer the only goal. What truly determines whether a model can ship and serve business workloads at low cost usually comes down to three engineering questions:

    1. Can model capacity grow while inference cost stays under control?
    2. Can long context and complex tasks reduce computational complexity through sparsification and modular design?
    3. Can training and alignment be supported by systems that deliver higher throughput, greater stability, and better observability?

    Looking at the technical direction of recent open-source models, a clear trend is emerging: models are no longer just “larger dense networks.” Instead, they increasingly combine approaches such as sparse activation, low-rank compression, sparse attention, asynchronous training, and agentic reinforcement learning. The core goal of these techniques is not to chase paper metrics alone, but to establish a better balance among quality, cost, stability, and scalability.

    Using an open-source large model technical report as background, this article neutrally dissects its key designs, focusing on the large-scale Mixture-of-Experts model, low-rank attention, grouped orthogonalization, multi-token prediction, sparse attention, and a fully asynchronous agentic reinforcement learning training framework. The article does not concern itself with any specific vendor or product; it analyzes from the perspective of general technical principles and engineering practice, so that developers and architects can draw on the ideas.


    2. Model Fundamentals: Combining Large Total Parameters with Small Active Parameters

    The model in this technical report adopts a typical large-scale sparse-activation architecture: about 744B total parameters, roughly 40B parameters activated per inference, 256 experts, and a network depth of about 80 layers. Compared with the previous generation, the total parameter count increased substantially, and the active parameter count grew somewhat as well. At the same time, the pretraining data scaled to about 28.5T tokens, of which roughly 27T tokens were used for pretraining and about 1.5T tokens for a mid-training or capability-strengthening phase.

    The key significance of this design is that it separates parameter capacity from per-inference computation.

    2.1 Large Total Parameters Provide Knowledge Capacity

    The larger the total parameter count, the more language patterns, code structures, knowledge relationships, and task priors the model can theoretically store. For complex reasoning, code generation, long-text understanding, and tool use, greater parameter capacity helps raise the ceiling.

    2.2 Small Active Parameters Control Inference Overhead

    If every inference activated all parameters, inference cost and latency would grow linearly with model size. The Mixture-of-Experts architecture uses a router to select only a subset of experts for the current input, so a single inference activates only about 40B parameters. This lets the model retain substantial knowledge capacity while avoiding inference costs that scale directly with total parameters.

    2.3 The Engineering Challenges of Sparse Activation

    However, sparse activation is not a “free lunch.” It introduces new engineering problems:

    • Whether routing stays balanced, or whether expert collapse occurs;
    • Whether communication overhead is manageable under expert parallelism;
    • Whether long-tail inputs cause load imbalance;
    • How to efficiently schedule experts during inference;
    • How to avoid capacity waste and token dropping during training.

    As a result, the quality of a Mixture-of-Experts model depends not only on the number of experts, but also on the routing strategy, load balancing, parallelism strategy, and system scheduling capability.


    3. Low-Rank Latent-Space Attention: Reducing Long-Text Cost with Compressed Representations

    Long context is an important indicator of model capability, but traditional attention mechanisms face clear computational and memory pressure when handling long sequences. For a sequence of length (L), the computational complexity of standard attention is typically (O(L^2)), which means that as context length increases, computation and cache overhead rise quickly.

    To mitigate this, the model adopts a low-rank latent-space attention approach. The core idea is not to store full key-value representations directly, but to compress key-value information into a low-dimensional latent space and then project it back to recover the needed representations.

    3.1 The Basic Logic of Low-Rank Attention

    You can think of it as a “compress–restore” structure:

    1. The input is mapped into a low-dimensional latent space through a projection matrix;
    2. The key information of historical context is stored in that low-dimensional space;
    3. When attention needs to be computed, an up-projection matrix restores the required dimensions;
    4. The attention-weighted output is finally produced.

    The advantages of this approach include:

    • Smaller cache size: no need to store full high-dimensional key-value caches;
    • Lower long-text inference overhead: the longer the sequence, the more pronounced the savings;
    • Higher throughput potential: a smaller cache favors serving larger batches;
    • Preserved expressiveness: low-rank mapping retains the main semantic information.

    3.2 The Value in Long-Text Scenarios

    In scenarios such as code-repository understanding, long-document question answering, multi-turn dialogue, and long-chain reasoning, context length often becomes the bottleneck. If the attention cache is too large, inference latency and memory usage climb rapidly. Low-rank latent-space attention lets the model process longer contexts under limited memory, making it an important direction for engineering long-text models.

    3.3 Engineering Considerations

    Low-rank attention does not deliver benefits just by compressing. In practice, it requires attention to:

    • The compression dimension should not be too low, or key information may be lost;
    • The projection matrix should be initialized reasonably to avoid training instability;
    • Representations at inference time should stay consistent with those at training time;
    • For long-dependency tasks, it should be paired with global tokens or hierarchical summarization mechanisms;
    • Cache hit rate and memory usage should be monitored, to avoid the case where compression is nominal but not effective in implementation.

    4. Grouped Orthogonalization: Solving Training Instability in Complex Projection Matrices

    During model training, the optimizer directly affects convergence speed, training stability, and final quality. The technical report notes that the team encountered a problem when using an orthogonalization-based optimizer: directly applying global orthogonalization to an entire projection matrix actually performed worse than the simpler grouped-query attention approach.

    This problem is quite representative.

    4.1 Why Global Orthogonalization May Fail

    Attention projection matrices usually contain multiple heads or subspaces. Different attention heads serve different functions—some attend to local syntax, some to long-range semantics, and some to positional relationships. Applying the same global orthogonalization constraint to all heads can cause two problems:

    1. It ignores differences between heads: the weight distributions of different heads are not identical, so forcing a unified constraint can disrupt their natural distributions;
    2. Optimization signals get averaged out: a global constraint can cause training gradients to conflict across heads, reducing effective learning.

    Therefore, simply applying a “more complex optimizer” to a “larger projection matrix” does not necessarily yield better results.

    4.2 The Idea of Per-Head Orthogonalization

    The solution described in the report can be summarized as “grouped orthogonalization” or “per-head optimization.” The basic idea is: instead of orthogonally constraining the entire projection matrix as one block, split the matrix by attention head, and apply constraints or updates to each head separately.

    The benefits of this approach include:

    • Preserving the independent representational capacity of different attention heads;
    • Reducing optimization conflicts between heads;
    • Making weight updates for each head more balanced;
    • Being more compatible with structured designs such as grouped-query attention;
    • Being easier to compute locally in distributed training.

    4.3 The Meaning for Training Stability

    In large model training, early instability, late-stage collapse, and gradient oscillation are all common problems. Grouped orthogonalization embodies an engineering principle: constraints should match structure. If the model structure is inherently grouped, the optimizer should also use grouped constraints rather than treating all parameters the same way.

    The lesson for developers: when introducing advanced optimizers, don’t focus only on the optimizer’s name—also consider how it couples with the model structure.


    5. Multi-Token Prediction: Increasing Training Signal Density

    Beyond attention and optimizers, the technical report also mentions multi-token prediction. A traditional language model usually predicts the next token at each position, whereas multi-token prediction lets the model, during training, predict not only the next token but also the content at several future positions.

    5.1 The Role of Multi-Token Prediction

    Ordinary autoregressive training provides only the supervision signal of “what the next token is.” Multi-token prediction, through auxiliary objectives, lets the model learn trends in future context earlier. Its potential benefits include:

    • Increasing training signal density;
    • Strengthening the ability to model long-range dependencies;
    • Improving performance on planning, reasoning, and code completion;
    • Letting the model absorb richer structural information during representation learning;
    • Providing optional draft or prediction capability for the later inference phase.

    5.2 Differences Between Training and Inference

    It is worth noting that multi-token prediction is usually used during training to strengthen representation learning, but inference does not necessarily have to predict multiple tokens at the same time. There are several engineering options:

    1. Use it only during training, keeping only the main prediction head at inference;
    2. Keep the auxiliary heads after training, for verification or parallel decoding;
    3. Distill the auxiliary heads’ capability into the main model;
    4. Selectively enable multi-step prediction for specific tasks.

    5.3 Illustrative Training Logic

    Below is a simplified pseudocode sketch:

    1. Input token sequence
    2. Obtain hidden representations through the backbone network
    3. Main prediction head predicts the next token
    4. Auxiliary prediction head predicts the next K tokens
    5. Compute main-task loss and auxiliary-task loss
    6. Total loss = main-task loss + λ * auxiliary-task loss
    7. Update model parameters

    Here, the auxiliary loss weight must be set carefully. Too high a weight may make the model over-focus on local future prediction and distort the main-task distribution; too low a weight may fail to provide effective regularization. So the practical effect of multi-token prediction depends on task type, data quality, model scale, and training-phase design.


    6. Sparse Attention: A Key Efficiency Lever in the Long-Context Era

    In this technical report, sparse attention is one of the most discussed techniques. An important conclusion in the report is that on some tasks, sparse attention can achieve results close to traditional full-attention approaches using far fewer training tokens. For example, the report notes that about 20B tokens of sparse-attention training can approach the performance that traditional approaches achieve with over 900B tokens.

    If this conclusion holds, its significance is substantial. It suggests that sparse attention is not only an inference-acceleration technique, but may also be a key method for improving training efficiency and lowering the cost of learning long context.

    6.1 Why Sparse Attention Is Needed

    Standard full attention requires computing the relationship between every position in the sequence and all other positions. For a sequence of length (L), the computational complexity is about (O(L^2)). As context expands from a few thousand tokens to tens or even hundreds of thousands, computational and memory pressure rises sharply.

    The core idea of sparse attention is: not all token pairs need equal-strength interaction.

    In language, code, and long-document tasks, the truly important relationships are often concentrated in local windows, key entities, global tokens, or a few highly relevant blocks. So it is possible to compute only a subset of attention connections, thereby reducing complexity.

    6.2 Common Sparse Patterns

    Common sparse-attention strategies in engineering include:

    Local Window Attention

    Each token attends only to a nearby span of context. Suitable for syntax, local semantics, and short-term dependencies.

    Global Token Attention

    Retain a small number of global tokens and let them interact with all positions, aggregating long-range information.

    Block-Level Sparse Selection

    Divide the sequence into multiple blocks, first compute correlations between blocks, then select the important blocks for attention computation.

    Dynamic Routing Attention

    Dynamically decide, based on the input content, which positions or blocks should participate in computation, so the sparse pattern changes with the task.

    Hierarchical Sparse Attention

    Use local attention at lower layers and global or block-level attention at upper layers, forming a hierarchical context compression.

    6.3 Consistency Between Sparse-Attention Training and Inference

    One of the biggest difficulties in deploying sparse attention is that training and inference must stay consistent. If training uses only local windows but inference suddenly enables full attention, model behavior may drift; conversely, if training uses full attention but inference switches to a sparse pattern for speed, quality may degrade.

    Therefore, engineering practice usually requires:

    • Introducing the target sparse pattern during training;
    • Reusing the same sparse strategy during inference;
    • Version-controlling the sparse patterns;
    • Verifying the difference between full attention and sparse attention on the evaluation set;
    • Designing dedicated regression tests for long-dependency tasks.

    6.4 Sparse-Attention Pseudocode Sketch

    1. Function sparse_attention(query, key, value):
    2. Divide the sequence into multiple blocks
    3. Retain a small number of global tokens
    4. Compute the match score of each block against the query block
    5. Select the few blocks with the highest scores
    6. Merge local blocks, global tokens, and the selected blocks
    7. Perform the attention-weighted computation
    8. Return the output

    This design can substantially reduce computation, but you must watch whether the ignored regions contain key information. For high-risk scenarios such as legal, financial, and code repositories, it is advisable to combine it with retrieval augmentation, global summaries, or hierarchical caching mechanisms.


    7. Fully Asynchronous Agentic Reinforcement Learning: From Synchronous to Pipelined Training

    Another focus of the technical report is the fully asynchronous agentic reinforcement learning training framework. Agentic reinforcement learning means letting the model execute tasks in an environment, call tools, receive feedback, and optimize its policy based on that feedback. Unlike traditional text alignment, agentic reinforcement learning usually involves multi-turn interaction, tool calls, code execution, retrieval, environment feedback, and long-trajectory generation.

    7.1 The Bottleneck of Synchronous Training

    In traditional synchronous training, the flow is usually:

    1. The model generates a batch of trajectories;
    2. Wait for all environments to finish their interactions;
    3. Compute rewards uniformly;
    4. Send all samples to the training nodes;
    5. Update the model;
    6. Enter the next round of generation.

    The problem with this pattern is that if even one long-tail task fails to complete, the entire batch has to wait. Agentic tasks inherently have a pronounced long-tail characteristic: a code-execution task that takes too long, a tool call that times out, or a multi-turn dialogue with too many rounds can all leave training nodes idle.

    The results are:

    • Lower resource utilization;
    • Limited training throughput;
    • Higher hardware cost;
    • Reduced model update frequency;
    • Difficulty supporting large-scale agentic training.

    7.2 The Core Idea of a Fully Asynchronous Framework

    A fully asynchronous framework decouples generation, reward, storage, and training so that different stages can run independently. A typical asynchronous pipeline can include:

    1. Task queue: continuously supplies tasks to be executed;
    2. Generation nodes: run the model in parallel to produce trajectories;
    3. Reward nodes: score or verify trajectories;
    4. Experience pool: stores samples waiting to be trained on;
    5. Training nodes: continuously sample from the experience pool and update the policy;
    6. Model release module: synchronizes new parameters to the generation nodes.

    In this way, generation does not need to wait for all tasks to finish, and training does not need to wait for all trajectories to be collected. The system can keep running like a pipeline.

    7.3 Asynchronous Training Pseudocode Sketch

    1. Task queue: holds agentic tasks waiting to be executed
    2. Trajectory queue: holds generation results
    3. Experience pool: holds training samples with rewards
    4. Parallel generation nodes:
    5. Fetch a task from the task queue
    6. Execute the task with the current model
    7. Write the trajectory into the trajectory queue
    8. Parallel reward nodes:
    9. Fetch a trajectory from the trajectory queue
    10. Invoke rules, verifiers, or a scoring model
    11. Write the rewarded sample into the experience pool
    12. Training nodes:
    13. Sample a batch of data from the experience pool
    14. Compute the policy update objective
    15. Update model parameters
    16. Publish the new model version

    7.4 The Key Challenge of Asynchronous Training: Sample Staleness

    While asynchronous training boosts throughput, it introduces a new problem: the samples used by the training nodes may have been generated by an older model. In other words, there is a gap between the current policy and the policy that generated the samples. If you train directly as in the synchronous setting, the optimization objective may drift.

    Common mitigation methods include:

    • Using importance sampling to correct sample weights;
    • Setting a maximum age limit for old samples;
    • Reducing the sampling probability of stale samples;
    • Using clipped objectives to limit the magnitude of policy updates;
    • Periodically clearing low-value or outdated experience;
    • Tagging samples by the model version that generated them.

    7.5 Reward Design for Agentic Tasks

    In agentic reinforcement learning, reward design is more complex than text alignment alone, because a task is judged not only by its final output but also by whether the process is reasonable. Common reward sources include:

    1. Task success rate: whether the tool call achieves the goal;
    2. Process correctness: whether the steps follow the constraints;
    3. Result verifiability: whether code passes tests, whether a query returns the correct result;
    4. Format compliance: whether the output meets the structured requirements;
    5. Safety: whether any risky action is triggered;
    6. Efficiency metrics: whether the number of calls, latency, and resource consumption are reasonable.

    If reward design is too simplistic, the model may learn to “game the system”—for example, repeatedly calling tools, producing superficially correct but unverifiable answers, or bypassing validation logic. Therefore, asynchronous reinforcement learning must be combined with strict verification mechanisms.


    8. Training Infrastructure: The System Capabilities Behind the Asynchronous Framework

    A fully asynchronous agentic reinforcement learning system is not a single-point algorithm but a complete systems-engineering effort. It requires scheduling, storage, queues, monitoring, fault tolerance, and elastic scaling to work together.

    8.1 Task Scheduling and Elastic Scaling

    Agentic tasks are bursty and long-tailed. The compute cost of different tasks varies widely, so a task queue and elastic resource scheduling are needed. The system should be able to allocate different resources by task type, for example:

    • Using high-throughput nodes for lightweight generation tasks;
    • Using isolated environments for heavy code-execution tasks;
    • Using large-memory resources for long-context tasks;
    • Using highly reliable execution environments for verification tasks.

    8.2 Object Storage and Trajectory Management

    During training, large numbers of trajectories, logs, checkpoints, and intermediate results are produced. These objects need to be stored reliably for retrospection, auditing, and reproduction. A general practice is to use object storage to hold:

    • Model checkpoints;
    • Task trajectories;
    • Reward details;
    • Training configurations;
    • Evaluation results;
    • Abnormal samples.

    8.3 Message Queues and Decoupling

    One of the core aspects of an asynchronous training framework is decoupling. Generation nodes, reward nodes, and training nodes should not be tightly coupled. Message queues can serve the following purposes:

    • Buffering generated trajectories;
    • Smoothing peaks and filling valleys;
    • Supporting concurrent processing by multiple consumers;
    • Isolating failed nodes;
    • Supporting retries and dead-letter handling.

    8.4 Logging Services and Monitoring Alerts

    Asynchronous systems are highly complex and must be observable. Metrics that need monitoring include:

    • Generation throughput;
    • Reward-computation latency;
    • Number of samples in the experience pool;
    • Sample staleness;
    • Training loss;
    • Policy update frequency;
    • Model-version synchronization latency;
    • Number of long-tail tasks;
    • Failure rate and retry rate.

    Logging services and monitoring alerts help teams quickly locate bottlenecks, such as an environment task blocked for a long time, abnormal reward computation for some category, or a node that keeps failing.

    8.5 A General Way to Deploy in Cloud Environments

    In cloud environments such as Baidu AI Cloud, enterprises can build training and inference platforms on general cloud capabilities, for example:

    • Using a container platform for task orchestration and elastic scheduling;
    • Using object storage to save training data and model checkpoints;
    • Using message queues to decouple the generation, reward, and training flows;
    • Using logging services to collect runtime logs;
    • Using monitoring and alerts to track throughput, latency, and resource utilization.

    These general infrastructure capabilities are not exclusive to any single model; they are a shared foundation for engineering large models in practice.


    Taken together, this technical report reflects not a single-point trick but a systematic line of thinking. It can be distilled into the following trends.

    9.1 Model Architecture Moving from Dense to Sparse

    Designs such as sparse activation, sparse attention, and grouped structures all aim to control computational cost while expanding capacity. Future large model design will increasingly resemble a “system optimization problem” rather than merely a “network architecture design problem.”

    9.2 Attention Mechanisms Moving from Full Connectivity to Structured Selection

    As long-context demand grows, full attention can no longer independently handle every scenario. Sparse strategies such as local, global, block-level, and dynamic routing will become important components of long-context models. The key is not to abandon global information entirely, but to let important information be seen at lower cost.

    9.3 Optimizers and Model Structure Must Be Co-Designed

    The grouped-orthogonalization case shows that an optimizer is not better just because it is more complex; it must match the model structure. For grouped attention, multi-head structures, and sparse expert networks, optimization strategies should also account for local structure and inter-head differences.

    9.4 Training Efficiency Depends on the System Pipeline

    Asynchronous agentic reinforcement learning shows that improvements in model capability increasingly depend on training-system throughput. Synchronous training is suitable for small-scale validation, but large-scale agentic training must adopt an asynchronous, decoupled, recoverable, and monitorable pipeline.

    9.5 Evaluation Must Move from Single-Point Scores to Comprehensive Verification

    Large model capability is judged not only by static question answering, but also by long text, code, tool use, multi-turn tasks, safety, and stability. Both sparse attention and asynchronous training change the model’s behavior distribution, so more fine-grained regression testing and online monitoring are needed.


    10. Engineering Recommendations

    For teams hoping to draw on this kind of technical direction, a few starting points are worth considering.

    10.1 First Define the Cost and Quality Boundaries

    Do not chase the largest parameters or the longest context from the outset. First clarify:

    • How much latency the business can tolerate;
    • What the per-request cost ceiling is;
    • What the real requirement for context length is;
    • What the task success-rate requirement is;
    • Whether tool calls and long-trajectory interaction are needed.

    10.2 Validate Sparse Attention in a Small Scope

    Sparse attention needs to be validated on target data rather than assumed effective. It is recommended to:

    • Evaluate separately on short, medium, and long contexts;
    • Compare the differences between full attention and sparse attention;
    • Focus on long-distance dependency tasks;
    • Check whether inference is consistent with training;
    • Build dedicated test sets for critical business.

    10.3 Pay Attention to Routing Balance in Mixture-of-Experts Models

    If you adopt a Mixture-of-Experts architecture, you need to continuously monitor:

    • Whether expert load is balanced;
    • Whether a few experts become overloaded;
    • Whether routing entropy is too low;
    • Whether expert collapse occurs;
    • Whether there are expert-cache-hit problems during inference.

    10.4 Start Asynchronous Reinforcement Learning with Observability

    An asynchronous framework should not be judged only by whether the training loop runs, but by whether the system is observable. It is recommended to prioritize building:

    • Task-queue monitoring;
    • Trajectory-generation-rate monitoring;
    • Reward-latency monitoring;
    • Sample-staleness statistics;
    • Failure and retry mechanisms;
    • Model-version release and rollback mechanisms.

    10.5 Embed the Evaluation Loop into the Training Pipeline

    Both asynchronous training and sparse attention change the model distribution. Teams should treat evaluation as part of the pipeline rather than an add-on after training. At a minimum, it should include:

    • Basic language-capability evaluation;
    • Long-context evaluation;
    • Code-capability evaluation;
    • Tool-use evaluation;
    • Safety and compliance evaluation;
    • Latency and throughput stress testing;
    • New-versus-old model comparison regression.

    11. Conclusion

    The core value demonstrated by this technical report is not simply that a certain model achieved higher scores, but that it offers a forward-looking methodology for large model engineering: use a large-scale sparse-activation model to balance capacity and cost, use low-rank latent-space attention to reduce long-text overhead, use grouped orthogonalization to improve training stability, use multi-token prediction to enrich the training signal, use sparse attention to improve long-context efficiency, and use a fully asynchronous agentic reinforcement learning framework to raise training throughput.

    For developers, what deserves the most attention is not any single term, but the shared principles behind these designs:

    1. Capacity can be large, but inference cost must stay controllable;
    2. Context can be long, but attention computation must be structured;
    3. Training can be complex, but the system pipeline must be efficient and stable;
    4. Capability can be improved, but evaluation and monitoring must form a closed loop.

    Going forward, competition among large models will be not only a competition of parameters, but a comprehensive competition of architecture design, training systems, inference services, and evaluation systems. Whoever can obtain more stable long-context capability, stronger agentic execution ability, and more reliable engineering observability at lower cost is more likely to win in real business scenarios.