Open-Source Multimodal LLM Training in Practice: From Staged Pretraining to Joint Reinforcement Learning
This article examines a practical training recipe for a new open-source multimodal large model, covering staged pretraining, short-cycle joint reinforcement learning, reasoning-trajectory distillation, MoE scaling, sparse-attention operator optimization, agentic fine-tuning, and safety-aware open-weight release. It explains how small and mid-sized teams can validate training recipes efficiently and why systematic pipeline design and engineering detail now matter as much as parameter count.
Background: Open-Source LLMs Enter a Dense Release Period
Open-source large models have been iterating at a visibly faster pace in recent years. Multiple research organizations have released open models with more than ten billion parameters within short intervals, and industry observers have even labeled one peak release month as an “open-source month.” Two forces are driving this phenomenon:
- Architecture dividend: The maturation of Mixture-of-Experts (MoE) architectures decouples total parameter scale from per-inference cost. Models can continue to raise their capacity ceiling by expanding the expert pool while keeping inference cost manageable.
- Convergence of training methods: Staged pretraining, distillation-style post-training, and unified reinforcement learning have gradually become industry consensus. The barrier for new teams to reproduce and surpass previous model generations has dropped significantly.
Against this backdrop, some small research teams release medium-sized models in a “small-step, fast-iteration” manner, validate the training recipe, and then extend it horizontally to more sizes. This has become a pragmatic product path: release the first rung of a model family, then gradually fill in different parameter tiers.
Model Positioning and Capability Boundaries
A typical recent generation of open-source multimodal models shows clear priorities in capability design:
- Input and output formats: It supports text and image input with text output, which is a standard vision-language model form. Image understanding is no longer an add-on capability but is deeply coupled with text reasoning.
- Adjustable thinking depth: The model allows developers to control the “thinking investment level,” meaning that under the same weights, adjusting the chain-of-thought length at inference time trades off response speed against answer quality. This is useful for cost-sensitive online services and quality-sensitive offline tasks, respectively.
- Long-horizon autonomous task execution: It focuses on supporting long periods of autonomous operation in agent scenarios, such as continuous coding, multi-step tool calls, and self-correction. This requires the model to maintain goal consistency over long contexts.
- Optimization for custom training: The model explicitly treats continued training with developers’ own data as a first-class objective. It also supports calling external tools within the training workflow to automate fine-tuning of other models.
Training Recipe: Four-Stage Pretraining + Short-Cycle Joint Reinforcement Training
The most notable part of the training recipe is the combination of “four-stage pretraining + 9-day joint reinforcement training.”
The Value of Staged Pretraining
Multi-stage pretraining usually advances progressively according to data mix and quality: early stages use large-scale general corpora to build foundational language and visual representations; middle stages introduce high-quality STEM, code, and reasoning data; later stages learn knowledge-intensive data; and the final stage uses a high-quality subset for annealing and convergence. Compared with “mix once and train to the end,” this approach has two major advantages:
- Each stage can be evaluated independently, so problems can be traced to a specific data stage.
- Annealing with high-quality data in the final stage can significantly improve the evaluation performance of the final checkpoint.
Short-Cycle Reinforcement Training
A 9-day joint reinforcement training cycle may look short, but it reflects a common industry trend: rather than running an extremely long single round of RL, it is better to perform short-cycle, high-intensity, multi-task joint reinforcement on a base model whose capabilities are already solid. Joint training mixes reward signals from mathematical reasoning, code generation, agent tool use, and visual understanding, avoiding the “capability seesaw” phenomenon caused by single-task RL.
Reasoning Trajectory Distillation
In the post-training stage, the model uses reasoning trajectories collected from another leading lab’s high-capability large model. This is a common and efficient practice in the open-source community: use a strong model to generate verified chain-of-thought and problem-solving processes as SFT or RL data sources, allowing a smaller model to inherit strong reasoning patterns at very low cost. Its predecessor also relied on synthetic data from third-party strong models for training, indicating that “distilled trajectories + in-house RL” has become a standard path for small and medium-sized models.
Architecture and Operator Optimization
MoE Scale Configuration
The model has approximately 594B total parameters and about 17B activated per token, compared with 276B total parameters and 12B activated in the predecessor. Between the two generations, total parameters doubled while activated parameters increased moderately. This reflects a typical scaling strategy in MoE design: expand expert capacity rather than thickening dense layers, thereby raising evaluation scores while keeping per-token computation growing at a near-linear, modest rate.
Sparse Attention Operator Acceleration
One engineering detail with strong reference value is that the team optimized the community-standard sparse attention forward and backward operators to about 6x the speed of the original implementation. Although sparse attention reduces the computational complexity of long contexts algorithmically, if the operator implementation is unfriendly—for example, with frequent index gather/scatter or inefficient kernel scheduling—the practical benefit can be largely consumed. Typical optimization directions include:
- Compiling sparse patterns into block-dense computation to improve GPU tensor core utilization.
- Fusing softmax with sparse index lookup to reduce memory round trips.
- Rewriting the automatic differentiation graph for backpropagation to avoid redundant intermediate storage.
For teams planning to build their own training frameworks, operator-level optimization is often more cost-effective than simply adding more GPUs.
Engineering Loop for Custom Fine-Tuning
When the model was released, it demonstrated a representative agentic fine-tuning case: the model autonomously completed evaluation set writing, data preparation, training plan comparison, and error correction. In a little over 4 hours, it fine-tuned a 4B-level base model, reducing that base model’s relative prediction error on unseen data by about 15%. This case shows:
- Evaluation first: The first step in an automated fine-tuning workflow is not running training, but building a trustworthy held-out evaluation set.
- Automated plan comparison: Letting an agent enumerate configurations such as learning rate, data mix, and training epochs and compare them side by side is more systematic than manual tuning.
- Error-driven iteration: Using validation error as a feedback signal for closed-loop correction avoids blindly increasing training volume.
In addition, the officially demonstrated visual creation capabilities—such as interactive city simulation, system-level desktop simulation, and 3D scene roaming—indicate that multimodal models are moving from “describing images” toward “generating interactive code from images.” This has direct practical value for front-end prototyping, teaching demonstrations, and similar scenarios.
Open-Source Release and Safety Process
The model is planned to be released with full weights under a permissive open-source license such as Apache 2.0 after external safety testing and red-team testing are completed. This process is worth borrowing for any team planning to open-source a large model:
- Evaluate before releasing: Complete third-party safety evaluation before opening weights, so that abuse risks do not spread irreversibly.
- Full weights rather than an API: Allowing the community to deploy locally and continue training is what truly builds an ecosystem.
- Tiered product line: Validate the recipe with a single size first, then release different parameter tiers as needed, reducing single-point risk.
Summary
The iteration path of the new generation of open-source multimodal models can be summarized as: an MoE scaling strategy of “large capacity, low activation” + a training recipe of multi-stage pretraining and short-cycle joint reinforcement learning + distillation of reasoning trajectories from strong models + operator-level engineering optimization + first-class design for developer custom training. For technical teams planning to build or adapt large models, these practices point to a common conclusion: in today’s mature open-source ecosystem, final model quality is determined not only by parameter count, but also by the systematization of the training pipeline and the depth of engineering detail.
