Open Source LLM Selection Guide: Sparse MoE, Dense Models, and World Models
This article compares three major open-source LLM architecture directions—sparse Mixture of Experts (MoE), dense models, and world models—and explains how to choose among them based on engineering goals rather than benchmark scores alone. It covers a model verification checklist, how MoE decouples capacity from per-token compute (and why it does not necessarily save GPU memory), the real conditions behind "single-GPU" dense deployment and quantization, acceptance criteria for world models in autonomous driving and robotics, a side-by-side comparison, and common FAQs and mistakes. Readers will find a practical, reproducible methodology for model verification, selection, evaluation, and deployment.
1. Quick Overview: Decide “Worth Choosing” Before Chasing Scores
Recent discussion around open-source LLMs clusters around three routes: sparse Mixture of Experts (MoE), dense models, and world models. These are not simple substitutes for one another; each maps to a different engineering goal. MoE pursues the capability ceiling, dense models pursue deployment determinism, and world models supply high-value simulation data for autonomous driving and robotics training.
A note on scope: this article does not fabricate a specific “weekly release list.” If you cannot verify each model’s name, weights, context length, license, and reproduction results one by one, confirm the information through official model cards, license texts, weight repositories, technical reports, and community reproductions. Treat the checklist below as a template for tracking new releases:
| Verification item | Information to record | Action if it fails |
|---|---|---|
| Model name and version | Official name, version number, whether weights are fully open | Treat as conceptual observation only; do not enter the deployment candidate set |
| Parameter scale | Total parameters, activated parameters, number of experts, routing strategy | Reporting only total parameters without activated parameters is incomplete information |
| Context capability | Official context length, actual effective length, KV Cache scheme | Without hands-on testing or reproduction, do not use it for long-context selection |
| License compliance | Commercial use allowed, redistribution allowed, attribution required | If the license is unclear, do not include it in the enterprise production pipeline |
| Official links | Model card, code repository, license, evaluation report | If it cannot be traced to a source, lower its credibility |
| Verification status | Reproduced by third parties, evaluation contamination concerns | Do not treat it as the basis for key conclusions before it is verified |
This article serves as a hotspot review and selection methodology; it does not replace any model’s official statements. When making a formal selection, re-verify the data cutoff and verification method, because open-source model licenses, weights, and evaluation results change over time.
2. Sparse MoE: Does Adding Parameters Really Save Compute?
The core idea of sparse MoE is to decouple “model capacity” from “per-token compute.” In a dense model, every token passes through all parameters. In MoE, a router selects a few experts for each token while the rest stay silent. As a result, a model with a very large total parameter count only activates part of those parameters per inference.
The logic simplifies to:
logits = router(x) # Gating network scores each experttopk_idx, topk_w = top_k_softmax(logits, k) # Each token activates only k expertsy = sum(topk_w[i] * experts[topk_idx[i]](x) for i in range(k))
This means MoE inference compute is usually closer to the activated parameters than to the total parameters. However, it does not necessarily save GPU memory, because all expert weights may still need to reside in device memory. In real deployment, the memory budget should include at least the following:
- Full expert weights: all experts must be loaded, otherwise routing is impossible.
- Activation cache and intermediate tensors: the forward pass produces intermediate activations, and the pressure grows with sequence length.
- KV Cache: long-context scenarios significantly amplify cache overhead.
- Communication buffers: cross-GPU All-to-All, All-Reduce, tensor parallelism, and expert parallelism all consume extra bandwidth and memory.
- Inference framework overhead: CUDA context, operator caches, the scheduler, and batch management all consume resources.
A simplified example: suppose an MoE model has 1.2 trillion total parameters and 18 billion activated parameters. Even if each inference only activates a small subset of experts, the weight memory must still cover all 1.2 trillion parameters. At FP16, the weights alone need roughly 2.4 TB of memory; at INT4, the weights are about 0.6 TB, which typically still requires multiple machines and GPUs. What is actually saved here is the per-token compute, not the weight storage pressure.
Therefore, whether MoE is worth choosing comes down to three questions: do you have enough memory and bandwidth; can you accept the cross-GPU communication overhead; and do you need to trade larger total parameters for a higher capability ceiling. For a small team deploying on a single GPU, MoE is usually not a low-barrier option.
3. Dense Models: How to Judge “Can It Run on a Single GPU?”
A dense model activates all parameters for every token, so inference latency is more predictable and the engineering pipeline is simpler, with no need to handle expert routing or load balancing. Its advantage is a “predictable cost-performance ratio”: a regular structure that is easy to distill, quantize, and deploy.
But “can it run on a single GPU” cannot be stated as a blanket claim; it has to become a conditional statement. A more rigorous phrasing is: under 4-bit or INT4 quantization, a short context length, a low batch size, a budgeted KV Cache, a well-adapted inference framework, and accuracy regression completed on the real business distribution, some dense models can serve inference on a single high-memory GPU. If the context is stretched to 32K or 128K, or if you need high-concurrency batching, multimodal inputs, tool calls, and long-chain reasoning, both memory and latency can rise sharply.
For quantized deployment, pay attention to:
- Quantization bits: INT4, INT8, FP8, and mixed precision all have different accuracy tradeoffs.
- Context length: longer context increases the KV Cache, and memory usage grows with sequence length and batch size.
- Batch size: batching improves throughput but also increases memory pressure.
- Inference framework: whether it supports the relevant quantization format, PagedAttention, KV Cache compression, and continuous batching.
- Accuracy regression: it must be validated on the real business distribution, not judged by general leaderboards alone.
License compliance should also become an explicit checklist:
- Commercial terms: whether commercial products, cloud services, internal inference, or external API calls are allowed.
- Attribution and redistribution: whether the license, NOTICE, and model provenance statements must be retained.
- Derivative model naming: whether quantized, distilled, or fine-tuned models must be labeled with the original name or follow specific naming requirements.
- Data usage and evaluation disclosure scope: whether the training data involves restricted sources and whether evaluation results must be published.
- Regional and legal restrictions: whether export controls, data compliance, or industry-specific regulatory limits apply.
- Output content responsibility: generative models can introduce content risks, which require accompanying review and logging mechanisms.
For enterprises, the real advantage of dense models is not “it can definitely run on a single GPU,” but that the deployment boundary is easier to quantify, verify, and control.
4. World Models: From “Looks Real” to “Acceptable”
The goal of a world model is to learn the dynamics of an environment and to generate or predict future states. In autonomous driving, it can generate multi-view video, LiDAR point clouds, and the behavior of traffic participants; in robotics, it can be used for simulated training of tasks such as dual-arm coordination, grasping, and mobile manipulation.
Its value lies in easing the high cost of collecting real data and the scarcity of long-tail scenarios. But a world model cannot be judged by visual quality alone; it must be measured against acceptance criteria:
- Physical consistency: whether speed, acceleration, collision, contact, and rigid-body constraints are reasonable, and whether clipping, teleporting, or floating occur.
- Temporal coherence: whether frames flicker, objects suddenly disappear, or trajectories drift.
- View consistency: whether multiple cameras are spatially consistent and whether images align with LiDAR point clouds.
- Semantic and task effectiveness: whether the generated data can support downstream tasks such as detection, segmentation, tracking, and planning.
- Sim-to-Real regression: transfer must be validated on a real test set, not self-proven only on the simulation set.
The proportion of synthetic data also needs threshold management. Non-safety-critical scenarios can use a higher ratio of synthetic data; safety-critical scenarios should rely primarily on real data, with synthetic data as a supplement, and should state the synthetic ratio, generation source, and acceptance results in the model card or data report. If a world model cannot prove transfer, it should not simply replace real data.
In addition, world models involve license, data authorization, privacy, and compliance issues. Autonomous driving road-test data, camera-captured data, and LiDAR point clouds in particular require confirming the authorization scope, anonymization methods, and redistribution restrictions. Only after the license, compliance, physical consistency, and Sim-to-Real regression all pass is a world model better suited to the data production pipeline.
5. Comparing the Three and Choosing
| Dimension | Sparse MoE | Dense Models | World Models |
|---|---|---|---|
| Capability ceiling | Usually higher; suited to flagship capability exploration | Medium-high; depends on training scale and data quality | Depends on generation quality and downstream task fit |
| Deployment cost | High memory and communication cost | Simple pipeline; controllable cost after quantization | High generation and verification cost; needs a simulation platform |
| Latency predictability | Affected by routing, load balancing, and cross-GPU communication | Relatively predictable | Depends on generation time and real-time requirements |
| Main risks | Insufficient memory, uneven routing, communication bottlenecks | Quantization accuracy drop, unclear license, limited context | Physical inconsistency, Sim-to-Real gap, data authorization |
| Use cases | Long context, multimodal, agents, capability-first | Single-GPU/small-cluster deployment, low-cost inference | Autonomous driving, robotics simulation, data augmentation |
When choosing, consider these questions first: If you prioritize capability, have ample compute, and need long-context and multimodal agent abilities, you can evaluate a sparse MoE flagship model, but you must account for memory, communication, and scheduling costs. If you prioritize deployment, require low latency and predictable cost, and a single GPU or small cluster is enough, a dense model with a quantization scheme is the preferred starting point. If you prioritize data production and are targeting embodied intelligence or autonomous driving simulation, you can bring in world models, but you must verify physical consistency, view consistency, and Sim-to-Real effectiveness.
6. FAQs and Common Mistakes
Q1: Can it actually run on a single GPU?
A: Don’t look at parameter count alone. You need to consider the quantization bits, context length, KV Cache, batch size, inference framework, and business accuracy regression results together.
Q2: Is MoE more memory-efficient?
A: Not necessarily. MoE usually saves per-token compute, but the total weights still have to cover all experts, so memory pressure is not necessarily lower.
Q3: How do you verify a quantization accuracy drop?
A: Build a fixed evaluation set on the real business distribution, compare key metrics before and after quantization, and record confidence intervals, failure samples, and scenario coverage.
Q4: Can world models replace real data?
A: Not simply. World models are suited to supplementing long-tail and simulation scenarios, but safety-critical tasks still require validation and calibration with real data.
Common mistakes also include: looking only at total parameters and not activated parameters; ignoring license redistribution restrictions; not running business-distribution regression after quantization; judging world models by visual effect alone; ignoring KV Cache and batch concurrency; and treating vendor-reported scores as the final conclusion.
7. Conclusion
Competition among open-source LLMs has shifted from single-point benchmark scores to a combined contest of architecture tradeoffs, deployment economics, license compliance, and scenario fit. MoE pushes the capability ceiling, dense models solidify the foundation for real deployment, and world models supply the data fuel for embodied intelligence and autonomous driving. For developers and enterprises, the truly effective approach is not to chase every release, but to build a sustainable methodology for model verification, selection, evaluation, and deployment.
