The 100x Price Gap in LLMs: Converging Intelligence, Model Selection, and Evaluation

I. Market Observation: Converging Intelligence, Diverging Prices

The past week likely saw the highest density of Large Language Model (LLM) releases this year: four leading vendors released four new models within three working days. Comparing official announcements with third-party evaluation data reveals an interesting “scissor gap”:

• Capability Dimension: The comprehensive intelligence indices of the four models were 66, 62, 61, and 59 respectively (based on the highest reasoning tier scores). The gap between the highest and lowest is only 7 points. The top tier has squeezed into the same range regarding “general intelligence.”
• Price Dimension: Output token pricing ranged from about $3.75 to $50 per million tokens—a 13x difference. If we include ultra-low-price tiers (around $0.20 per million tokens) that allow customer data to be used for training, the overall price gap stretches to over 100x.

Converging capabilities and layered pricing is the most crucial signal for developers this week. It means the answer to “which model to choose” is shifting from “who is the smartest” to “whose cost-effectiveness and engineering controllability best fit my task distribution.”

1.1 Timeline of a Saturated Release Week

• Monday: A leading vendor adopted a “dual-version” strategy, splitting the same base model by safety levels. One version was open to all users, while the other was provided only to approved institutions via a trusted access program. Safety tiering itself became a part of product capability.
• Tuesday: Two vendors made their moves. One claimed the “largest leap in model performance to date,” with coding capabilities benchmarked against the previous generation’s flagship. Another launched a new lightweight model—notably, its third lightweight release in six weeks and just three weeks after the previous generation.
• Wednesday: The finale featured rhetoric about “entering the AGI era.” However, the roadmap teaser released the day before was mocked by the tech community as “the new model isn’t quite ready, so they released a teaser first.” Under dense release schedules, marketing first and product shipping at the last second has become an industry norm.

1.2 Understanding Market Tiering

Based on public information this week, the current market can be roughly divided into four tiers:

Tier	Positioning	Typical Characteristics	Suitable Scenarios
Flagship	Capability Ceiling	Highest intelligence index, output priced around $50/M tokens, cache prices continually decreasing	Complex agent long tasks, high-difficulty reasoning
Mid-range	Substitute for Prev. Flagship	Matches previous flagship at about 1/5 the price	The vast majority of production workloads
Lightweight	High Speed, Low Cost	Intelligence reaches mid-level flagship standards, 1-2 orders of magnitude cheaper	High concurrency, short context, deterministic tasks
Low-cost Compliance	Data for Price	Extremely low price, but includes data usage terms	Non-sensitive data, offline batch processing

1.3 The Competitive Dimension Has Shifted

When score gaps shrink to single digits, release cadence, safety tiering, cache pricing, and ecosystem support become new battlegrounds for differentiation. For developers, this means evaluation and selection methodologies must be upgraded: making decisions based solely on a comprehensive index is no longer sufficient.

II. Decoding Three Technical Signals

2.1 Signal 1: Top-tier Models Bet on Long-task Agents and Cache Economics

Two things appeared in tandem with this week’s flagship models: raising the upper limit for agent long tasks and lowering cache read prices. To understand their connection, we must look at the cost structure of agent long tasks.

A typical agent task might involve dozens or even hundreds of tool calls. Since mainstream conversational models are stateless, every round requires resending the system prompt, tool definitions, and complete interaction history. Take an 80-round task: resending ~30,000 tokens per round accumulates to ~2.4 million input tokens. At $50/M, the input cost alone is ~$120—for a single task.

Prompt caching (which reuses the intermediate key-value states of stable prefixes during inference) hits this pain point perfectly. The stable prefix is computed once, and subsequent rounds read it at a fraction of the standard price. A rough estimate: assuming the stable prefix accounts for 90% and the cache read price is 10% of the standard rate (refer to official pricing pages for exact ratios), the input cost for the same task drops to ~$23, less than one-fifth of the original. Cache pricing directly dictates the economic model of agent applications, which is why it is always emphasized alongside long-task capabilities.

2.2 Signal 2: Mid-range Models Match Previous Flagships at 1/5 the Price

This week’s mid-range models officially claimed to “match the coding capabilities of the previous flagship.” This is driven by the continuous democratization of inference efficiency technologies: Sparse Mixture-of-Experts (MoE) architectures reduce activated parameters, distillation compresses flagship capabilities into smaller models, quantization and speculative decoding lower inference costs, and KV cache compression extends effective context. These standard industry routes are turning “last year’s flagship capabilities” into “this year’s mid-range standard.”

The direct implication for developers is: unless your task strictly requires current top-tier capabilities, most production workloads can stop at the “previous flagship level”, getting 80-90% of the effect at one-fifth of the price. A safer approach is to run both the “previous flagship” and “current mid-range” models in your evaluation scripts—for specific task distributions, the latter is likely sufficient.

2.3 Signal 3: Lightweight Models Rapidly Approaching Mid-tier Flagships

A vendor releasing three lightweight models in six weeks speaks for itself: small models have lower training and validation costs and shorter feedback cycles, resulting in faster capability climbing. This week’s lightweight intelligence index has already touched the mid-level flagship tier, meaning many scenarios that “previously required flagships” now have cheaper alternatives:

• Online services requiring high concurrency and low latency (classification, extraction, rewriting, format conversion);
• Pipeline segments with short contexts and deterministic task structures;
• First-level routing in agent systems—digesting simple subtasks locally and only passing hard steps to the flagship tier.

III. How to Read Intelligence Indices: What They Can and Cannot Do

Third-party comprehensive intelligence indices are usually weighted results of public benchmarks like reasoning, code, math, and long context. They are good for quick initial screening, but three limitations must be clear:

1. Saturation Effect: Top models generally approach perfect scores on traditional benchmarks, reducing differentiation. A 7-point gap doesn’t necessarily correspond to a significant difference in real tasks.
2. Data Contamination: Public benchmarks might already be in the training corpus, inflating scores.
3. Variance and Prompt Sensitivity: The same model can show significant score fluctuations under different prompts and sampling parameters. A single benchmark run is not entirely trustworthy.

The correct usage is: treat public indices as “initial screening” and your custom task set as the “final verdict.” What truly matters is “how this model scores on my task distribution,” not “where it ranks on the leaderboard.”

IV. Engineering Practice: Evaluating Multiple Models in One Script

4.1 Unified Access Abstraction

Currently, mainstream LLM service platforms (including Baidu AI Cloud and others) generally offer message-array-based conversational completion interfaces. The request and response field semantics are highly consistent, and most provide general capabilities like batch inference and context caching. This makes “evaluating multiple models with one script” engineering-light: you only need to maintain a profile configuration for each candidate model.

4.2 Calling Skeleton and Cost Accounting (Illustrative Code)

import time, requests
from dataclasses import dataclass
@dataclass
class ModelProfile:
    name: str
    base_url: str      # Platform conversational completion interface URL
    api_key: str
    model_id: str      # Model identifier within the platform
    in_price: float    # Price per million input tokens
    out_price: float   # Price per million output tokens
def chat(p: ModelProfile, messages, temperature=0, max_tokens=2048):
    t0 = time.time()
    resp = requests.post(
        p.base_url,
        headers={"Authorization": f"Bearer {p.api_key}"},
        json={"model": p.model_id, "messages": messages,
              "temperature": temperature, "max_tokens": max_tokens},
        timeout=120,
    )
    resp.raise_for_status()
    data = resp.json()
    usage = data.get("usage", {})
    cost = (usage.get("prompt_tokens", 0) / 1e6 * p.in_price
            + usage.get("completion_tokens", 0) / 1e6 * p.out_price)
    return {"text": data["choices"][0]["message"]["content"],
            "latency": time.time() - t0, "cost": cost}

4.3 Scoring and Statistical Design

• Dual-track Scoring: Use rules for objective tasks (run unit tests for executable code, validate formats for structured output). For open-ended tasks, use a model as a judge to score based on rubrics, ensuring the judge model is decoupled from the models being evaluated.
• Control Variables: Fix the temperature parameter, run multiple rounds per use case, report the mean and variance to avoid mistaking random fluctuations for model differences.
• Cost and Latency as First-class Metrics: Average cost per question, P50/P95 latency, and token consumption must be recorded in the database and presented with the same weight as quality scores.
• Long-task Specifics: Set up separate evaluations for agent scenarios—completion rate within step limits, tool call success rate, and recovery capability after mid-task failures.

4.4 Interpreting Results: The Quality-Cost Frontier

Plot all models on a “quality score - cost per question” scatter plot, keeping only the candidates on the Pareto frontier. You will find that in this week’s landscape, the frontier often contains flagship, mid-range, and lightweight tiers simultaneously. The conclusion is usually not to “pick one model,” but to “compose a layered combination based on task difficulty.”

V. The Cost Engineering Trinity: Caching, Routing, Downgrade

1. Cache-friendly Prompt Structure: Place stable content (system prompts, tool definitions, few-shot examples) at the very beginning, and dynamic content at the end. Avoid placing volatile fields like timestamps or random IDs at the start, which would break prefix cache hits.
2. Model Routing and Cascade Downgrade: Use rules or a lightweight classifier to gauge request difficulty first. Route simple requests to the lightweight tier and hard requests to the flagship tier. Alternatively, use a cascade strategy—let the lightweight tier answer first, and if confidence is low, upgrade and retry.
3. Batch Processing and Compliance Review: Route offline tasks to batch interfaces (the industry commonly offers lower-priced batch tiers). Pay special attention: ultra-low-price tiers often come with “data can be used for training” clauses. When dealing with sensitive business data, review these clause by clause, and if necessary, define data boundaries explicitly through an enterprise-level contract.

VI. Model Selection Decision Checklist

Before starting your evaluation, answer these questions:

• What are the maximum and typical context lengths required by the tasks?
• Does it involve multi-round tool calling long tasks? What is the proportion of the stable prefix?
• What is the online latency budget? Can cascade downgrade be accepted?
• Is the data allowed to be used for training? Where is the compliance red line?
• Does the evaluation set cover the real task distribution? Are online bad cases continuously flowing back?

Conclusion

With intelligence indices differing by only 7 points but prices varying by 100x, the focus of model selection has shifted from “which model is the smartest” to “which combination offers the highest cost-effectiveness and engineering controllability for my task distribution.” Running evaluation scripts first, prioritizing cost engineering, and refining caching and routing strategies will determine your application’s ultimate success far more than chasing any single product launch.