The Evolution of Open-Source LLMs: Low-Cost Inference and Multimodal Practices

1. Introduction: The Paradigm Shift from Compute to Efficiency

On January 20, 2025, the release of an open-source reasoning model marked a watershed moment in the AI industry. Public records indicate that its complete training cost was approximately $5.6 million—a fraction of the cost of comparable closed-source models—yet its performance rivaled the strongest international closed-source reasoning models. This triggered a systemic revaluation of the “compute is king” narrative, causing massive volatility in semiconductor stocks and wiping out nearly $600 billion in market capitalization from a leading chip company. Many practitioners liken this to the AI industry’s “Sputnik moment”: it didn’t invent an entirely new technological trajectory, but it proved through extreme engineering efficiency that top-tier intelligence does not necessarily require a top-tier budget.

For developers, the technical signal matters more than market fluctuations. The path to improving model capabilities is shifting from “stacking pre-training compute” to a multi-curve parallel approach encompassing “architectural innovation + reinforcement learning + inference-time scaling.” This article follows this main thread to review the critical transitions in open-source LLMs in 2025 and translates them into actionable engineering practices.

2. Low-Cost Reasoning Models: Convergence of Three Technical Threads

1. Reinforcement Learning for “Deep Thinking”

The core of the first generation of open-source reasoning models is the use of reinforcement learning (RL) to directly optimize “verifiable rewards”: whether a mathematical proof is correct, or whether code passes tests. These signals are objective, cheap, and require no manually annotated preference data. Unlike traditional RLHF, which relies on an independent reward model, this new class of algorithms uses “group-relative comparison.” It samples a group of responses in parallel for the same prompt and uses the group’s average score as a baseline. Responses better than the baseline are reinforced, while inferior ones are suppressed. By eliminating the value network, both memory footprint and training complexity are reduced, allowing mid-sized clusters to complete the full training process.

Even more valuable are the “emergent” behaviors during training: models spontaneously learn to self-verify, backtrack, correct, and start over within long chains-of-thought (CoT). This means “inference-time scaling” has become a second scaling curve alongside “pre-training scaling.” Giving the model more time to think yields more reliable answers, with marginal costs far lower than continuously stacking training compute.

2. MoE Architecture: Decoupling “Large” from “Expensive”

The capabilities of dense models are roughly linearly tied to parameter counts, causing inference costs to rise linearly as well. The Mixture-of-Experts (MoE) architecture breaks this binding through sparse activation. The total parameter count can reach hundreds of billions, but only a small fraction of experts are activated per token. This is paired with shared experts for general knowledge and balance strategies (with or without auxiliary loss) to prevent expert degradation. The result is that “total parameters determine the capability ceiling, while activated parameters determine inference cost”—the two can now be adjusted independently. After 2025, MoE effectively became the default choice for flagship models.

3. Engineering Cost Dividends

Low costs stem not only from algorithms but also from engineering:

• Low-Precision Training: FP8 mixed precision is applied throughout forward computation and most matrix multiplications. Combined with fine-grained quantization and high-precision accumulation, it reduces memory and bandwidth pressure with almost no loss in convergence quality.
• Parallelism and Scheduling Optimization: Bi-directional pipeline scheduling compresses pipeline bubbles, and overlapping cross-node communication with computation hides collective communication overhead.
• Multi-Token Prediction: Predicting multiple subsequent tokens simultaneously at one position increases training signal density and leaves room for speculative decoding on the inference side.
• Downstream Distillation: Using CoT data generated by large models to distill smaller models allows billion-parameter models to gain usable reasoning capabilities, enabling single-GPU deployment.

The compounding effect of these four factors turned the “train a top-tier reasoning model for a few million dollars” slogan into reality.

3. MoE as the Flagship Standard and Multimodal Breakthroughs

In January 2025, the industry released a flagship MoE model pre-trained on over 20 trillion tokens, comprehensively surpassing contemporary open-source MoE models and the largest open-source dense models on multiple mainstream benchmarks. In the same month, open-source vision-language models (VLMs) were released in 3B, 7B, and 72B sizes, significantly improving image parsing accuracy and breakthoughly supporting over 1 hour of video understanding.

Several key engineering aspects on the multimodal side are worth noting:

• Dynamic Resolution Modeling: Images are split into a variable number of visual tokens at their native resolution, avoiding the loss of fine details like small text and charts through crude scaling.
• Video Temporal Sampling: Uniform frame sampling for long videos while retaining timestamp encoding balances “understanding every frame” with “comprehending global temporal sequences.”
• Size Tiering: The 3B model targets edge and mobile devices, 7B targets standard GPUs, and 72B targets accuracy-priority scenarios. A single training recipe covers diverse compute budgets.

4. From Closed to Open: Strategic Shifts by Major Vendors

Open source is no longer a detour for small teams; it is a strategy openly adopted by major vendors. On March 16, Baidu released a new generation of native multimodal foundation models and deep thinking models, opening them for free. The former’s multimodal capabilities align with top international closed-source models, with API pricing at roughly 1% of comparable competitors. The latter pioneers autonomous tool calling, allowing on-demand combinations of search, drawing, and code interpreters to complete complex tasks. On June 30, Baidu further open-sourced 10 models across various sizes and modalities, completing its pivot from closed to open source.

Concurrently, open-source reasoning models in the 32B parameter range emerged, with performance matching the phenomenal open-source reasoning models from early 2025, and deployment thresholds low enough for a single machine. On May 28, a leading open-source reasoning model released an iteration. Although officially termed a “minor upgrade,” tests showed it approached the strongest closed-source reasoning models on mainstream coding benchmarks, with weights continuing to be open under permissive licenses like MIT.

The logic behind major vendors collectively shifting to open source is straightforward:

1. Ecosystem Niche: The more developers use, fine-tune, and integrate a model, the more entrenched its default position in the tech stack becomes.
2. Trust: Auditable weights and private deployment capabilities are prerequisites for enterprise clients to place orders.
3. Standards: Open-source models become the de facto capability baseline; closed-source services must continually maintain a generational gap to command a premium.

5. The Value of Permissive Licenses: MIT is More Than “Free”

The MIT license allows commercial use, modification, and redistribution without requiring derivative works to be open source. For enterprises, this means three things: data stays in-domain (sensitive scenarios can be privately deployed), capabilities can be accumulated (continue training on open weights to inject proprietary knowledge), and no vendor lock-in (migration is possible at any time). On the community side, download counts and the number of derivative models in open-source repositories have become factual indicators of influence. In 2025, multiple open-source series consistently topped global download charts, signifying an irreversible shift in the open-source ecosystem’s center of gravity.

6. Developer Practices: Selection, Deployment, and Cost Optimization

1. Four Questions for Model Selection

• Does the task require deep reasoning? Choose reasoning models for math, code, and multi-step planning; use general models for extraction, classification, and chitchat.
• What are the context and modality needs? For long documents and long videos, prioritize long-context and multimodal versions.
• What is the latency and concurrency budget? CoT increases time-to-first-token; interactive scenarios must evaluate the streaming experience.
• Does the license meet compliance requirements? Verify commercial terms, data cross-border policies, and auditing needs.

2. Two Deployment Paths

Path	Applicable Scenarios	Advantages	Considerations
Fully Managed API	Fast prototyping, volatile traffic	Zero ops, elastic scaling	Requires evaluating data cross-border policies
Private Deployment	Sensitive data, stable high traffic	Data stays in-domain, deep control	Requires building in-house inference and ops capabilities

Mainstream cloud platforms like Baidu AI Cloud typically package model inference, fine-tuning, monitoring, alerting, and logging into general LLM services, suitable for rapid prototyping and elastic loads. The private deployment route involves open-source weights + inference acceleration frameworks + quantization (INT8/INT4 weight quantization is a common starting point), enhanced with continuous batching and paged KV cache to boost throughput.

3. API Example: Distinguishing CoT from Final Answer

# Pseudo-code: Calling the general chat interface of an open-source reasoning model (streaming)
import json
def ask_reasoning_model(question: str):
    payload = {
        "model": "reasoning-32b",              # Using a 32B open-source reasoning model as an example
        "messages": [
            {"role": "system", "content": "You are a rigorous assistant. Think first, then answer."},
            {"role": "user", "content": question},
        ],
        "temperature": 0.6, "top_p": 0.95,
        "max_tokens": 8192, "stream": True,
    }
    # POST {inference_endpoint}/v1/chat/completions, an industry-standard chat completion interface format
    resp = http_post_stream(f"{ENDPOINT}/v1/chat/completions",
                            headers={"Authorization": f"Bearer {API_KEY}"},
                            body=json.dumps(payload))
    for line in resp.iter_lines():
        if not line.startswith("data: "):
            continue
        data = line[6:].strip()
        if data == "[DONE]":
            break
        delta = json.loads(data)["choices"][0]["delta"]
        if delta.get("reasoning_content"):
            render_thinking(delta["reasoning_content"])   # Chain-of-thought: can be collapsed on the frontend
        elif delta.get("content"):
            render_answer(delta["content"])               # Final answer

Engineering takeaway: The CoT and the answer are output separately, allowing the frontend to collapse the thinking process by default. For reasoning tasks, a temperature of 0.5–0.7 is recommended. Too low, and it easily falls into repetitive loops; too high, and it disrupts the stability of the long CoT.

4. Three Cost Optimization Tactics

def choose_model(prompt: str) -> str:
    score = estimate_complexity(prompt)   # Rule-based or small model scoring
    if score < 0.4:
        return "chat-7b"        # Simple tasks: small model, low latency and cost
    if score < 0.75:
        return "chat-72b"       # Medium tasks: general flagship
    return "reasoning-32b"      # Complex tasks: deep thinking model

• Request Routing: As shown above, never waste reasoning model compute on simple requests.
• Prefix Caching: Reuse KV caches for fixed system prompts and long documents; cached portions are not recalculated.
• Batching and Asynchronous Processing: Offline tasks go through batch channels for lower unit costs.

7. Implications of the Cost Curve: The Application Layer Window is Opening

Inference costs are dropping by roughly an order of magnitude annually. As the price of “deep thinking” approaches that of a “standard answer,” numerous scenarios previously shelved due to cost—such as code review, investment analysis, industrial diagnostics, and complex customer service—will enter the economically viable range. For developers, the 2025 takeaway can be condensed into one sentence: Capabilities are rapidly equalizing; differentiation will come from engineering and scenarios. Establishing architectural habits around “multi-model routing + private fallback + cost observability” early on is more important than betting on any single model.

8. Conclusion

From the astonishing debut of low-cost reasoning models to the flourishing of MoE and multimodal systems, and the collective pivot to open source by major vendors, 2025 established a new consensus: the scarcity of intelligence is being dissolved by algorithmic and engineering efficiency. For technical teams, the true moat is no longer “which model you use,” but seemingly mundane engineering capabilities like data quality, evaluation systems, routing strategies, and cost control. Under the open-source wave, everyone has access to a top-tier engine; victory depends on who builds the best car around it.