LLM Model Selection: Flagship vs. Economy Tiers and How to Use the Three Thinking Modes

    Choosing a large language model isn't a single-answer question about picking "the strongest model." It's a multi-objective decision spanning model tier, thinking mode, and invocation pattern. This article explains the core differences between the flagship and economy tiers (similar reasoning ability, very different world knowledge), how the three thinking modes trade test-time compute for accuracy, and which tier-and-thinking combination fits which task. It also covers managing million-token context windows, why enabling thinking on the economy tier can backfire, how to encode selection rules in a routing layer with graded fallback, and how to model the real cost of input tokens, output tokens, and reasoning tokens—ending with a pre-launch checklist and a data-driven way to tune thinking budgets.

    1. First, clarify what model selection actually decides

    During a phase of rapid LLM iteration, “which version should we use?” and “should we turn on thinking?” have become questions almost every application team has to answer. After several rounds of trial and error, many teams still can’t say whether their configuration is reasonable. The reason is that people often treat “which model to pick” as a single-choice question, when it is really a multi-objective decision.

    A complete selection decision involves at least three dimensions:

    Dimension Options Main impact
    Model tier Flagship, Economy World-knowledge coverage, ceiling for complex tasks, price per token
    Thinking mode Off, Light, Deep Accuracy on hard problems, latency, output length
    Invocation pattern Single-turn Q&A, multi-step agent, batch/offline jobs Concurrency load, failure tolerance, room for caching and batching

    Together these three dimensions determine four outcome metrics: quality, cost, latency, and throughput. So the goal of selection isn’t to “find the strongest model” but to “find the best combination of cost and latency while keeping quality acceptable.”

    There’s an easily overlooked premise here: quality is a task-level property, not a model-level one. The same economy model might be more than enough for text summarization yet clearly struggle with multi-hop factual Q&A. So selection should use the task as the smallest unit, then map it onto a “tier × thinking mode” combination.

    2. With a million-token context now a baseline capability, rethink what “context” means

    Today’s mainstream LLMs commonly support a context window of around one million tokens—roughly 750,000 Chinese characters, enough to read an entire copy of Romance of the Three Kingdoms or the whole Three-Body trilogy in one pass. And a million-token context is shifting from a paid add-on on a few flagship models to a baseline capability of mid-to-high-end models. This raises three new requirements for engineering practice.

    First, long isn’t the same as accurate—watch the “effective context.” When vendors release long-context capabilities, they usually publish “Needle in a Haystack”–style results, i.e., locating a randomly inserted key fact within a very long text. These tests are close to saturated, but real tasks often require linking information across multiple fragments, resolving conflicts, and doing multi-step reasoning—far harder than retrieving a single sentence. The commonly observed “Lost in the Middle” effect shows that, all else equal, information in the middle of the context is more likely to be ignored. Practical mitigations include:

    • Put key instructions at the very start or end of the context;
    • Keep the most relevant evidence close to the user’s question;
    • Summarize long documents in layers first, then let the model synthesize.

    Second, cost grows roughly linearly with length, while latency grows faster than linearly. Input tokens are usually billed by volume. Attention computation grows roughly with the square of sequence length, so the longer the sequence, the more noticeable first-token latency and overall response time become. So if retrieval and summarization can compress the input to tens of thousands of tokens, there’s no need to stuff in an entire document.

    Third, context is a resource that needs a “budget.” You can allocate a context budget per call based on experience:

    • System instructions: keep within 1k tokens, concise;
    • Retrieved evidence: 8k–30k tokens, sorted by relevance and de-duplicated;
    • Conversation history: use a sliding window plus rolling summary to avoid unbounded growth.

    Anything over budget should be handled by chunking, layered summarization, or asynchronous batch processing.

    3. Flagship vs. economy: the core differences come down to two points

    Setting aside the noisy leaderboards, the core differences between the two tiers reduce to two.

    First, reasoning ability is close. On tasks that reason “from scratch,” such as code generation and mathematical derivation, the gap between economy and flagship is usually small. On some public code benchmarks, the two tiers, when run at a higher thinking intensity, often differ by only a few percentage points. So if what matters is “whether the derivation is correct,” the economy tier is enough in most cases.

    Second, world-knowledge coverage differs markedly. On factual Q&A benchmarks, the flagship tier usually leads the economy tier by a clear margin. The underlying reason is parameter count: the economy tier typically has only a fraction—sometimes much less—of the flagship’s parameters, so the encyclopedia knowledge it can “remember” is inherently limited. There’s a simple test for this gap—if an answer can’t be derived from the given material and must rely on the model’s own knowledge, the economy tier is more prone to factual errors.

    This yields a selection rule:

    If knowledge is missing, use the flagship tier—or put the knowledge into the context; only if reasoning is the bottleneck do you truly need flagship tier plus deep thinking.

    The engineering logic behind this rule is that missing knowledge can usually be supplied externally through retrieval, a knowledge base, or tool calls at controllable cost, whereas a weakness in reasoning is hard to patch with prompting. So before upgrading tiers, first check whether the input material is sufficient—many cases of “the model isn’t good enough” are really “the context wasn’t rich enough.”

    4. The three thinking modes: trading compute for accuracy

    The basic principle of thinking modes isn’t complicated: before giving a final answer, the model first generates a span of intermediate reasoning. This lets it break a complex problem into small steps and self-check before answering. Its essence is “test-time compute”—increasing the amount of computation during inference to gain higher accuracy.

    Take the common three-tier design as an example (naming and implementation details vary by platform, but the positioning is the same):

    Tier Behavior Typical benefit Main cost
    Off Answer directly, no intermediate reasoning Lowest latency, lowest cost, most stable format Multi-constraint complex tasks prone to omissions or errors
    Light A little intermediate reasoning, quick self-check Clear accuracy gains on medium-difficulty problems, controllable cost increase Slight increase in latency and output length
    Deep Long reasoning chains, comparing multiple options Significant gains on hard reasoning problems Latency several times higher, verbose output, higher cost

    When using thinking modes, two things must be clear:

    1. Thinking tokens are usually billed as output. Most services count the tokens generated during thinking as output tokens. So turning on thinking isn’t just “slower”—it’s “more expensive.” If the platform lets you set a thinking budget, treat it as an important cost parameter.
    2. Higher tier isn’t always better. For a simple task that can be done in one step, deep thinking can cause “overthinking”: the conclusion drifts during the long reasoning, the output gets verbose, and latency and cost both rise—leaving you worse off.

    A practical decision framework is: task difficulty × latency budget = thinking tier. Only hard tasks without strict latency requirements deserve deep thinking; everything else should default to off or light.

    5. Quick scenario reference: pairing tiers with thinking modes

    Scenario Recommended tier Thinking tier Key reason
    Intent recognition, keyword extraction, format conversion Economy Off Single-step tasks; prioritize throughput and cost
    Product descriptions, marketing copy, multilingual polishing Economy Off or Light Relies on language generation; low demand for niche knowledge
    Structured extraction (contracts, invoices, forms) Economy Light Many field rules need reasoning to validate, but the material is fully provided
    Code completion, unit test generation Start with Economy Light Reasoning is close to flagship; simple logic needs no deep thinking
    Complex refactoring, production incident triage Flagship Deep Needs cross-file reasoning and multi-hypothesis verification
    Long-document summarization, contract clause comparison Flagship Light Needs cross-paragraph linking and knowledge to spot omissions
    Multi-hop factual Q&A, industry knowledge Q&A Flagship Light to Deep World knowledge sets the ceiling on the answer
    Math problems, algorithm problems, solution design Flagship Deep Long chains and error-prone; needs an ample reasoning budget
    Multi-step agent tasks (tool calls, workflows) Mostly Flagship Set per stage Light thinking for planning and reflection; off for execution and format output
    High concurrency, strong real-time (dialogue first response, risk decisions) Economy Off Latency-sensitive; can pair with asynchronous or streaming output

    This table has an implicit rule: thinking mode correlates with task difficulty, not much with model tier. Even on the flagship tier, simple tasks should use off or only light thinking, reserving deep thinking for the genuinely hard parts.

    6. Why does “turning on thinking make the economy tier worse”?

    You’ll often see feedback in the community that “turning on thinking makes the economy tier worse than leaving it off.” This isn’t necessarily a model defect; more commonly it’s due to the following reasons.

    1. Insufficient training-data coverage. The model has seen fewer examples in thinking mode, so reasoning chains easily pick up irrelevant content or even drift further off course.
    2. Improperly set thinking budget. Too small a budget truncates reasoning midway; too large a budget makes the model repeatedly doubt itself and waver among candidate answers.
    3. The task itself isn’t suited to thinking. Tasks like extraction and formatting have a single correct answer and clear steps; thinking offers no benefit and may even change wording and structure, breaking downstream parsing.
    4. Sampling parameters weren’t adjusted per tier. Thinking modes often need relatively more conservative sampling; reusing a high-temperature setting tuned for fast direct answers makes output more likely to diverge.
    5. The testing method isn’t rigorous. Judging by a few subjective cases, or comparing without fixing the prompt, version, and evaluation set, easily leads to wrong conclusions.

    The right approach is: build a small regression set of 50–200 items per task type, hold other variables fixed, and compare accuracy, average output length, P95 latency, and per-call cost across off/light/deep—then let the data decide whether to enable thinking.

    7. Engineering practice: encode selection rules in a routing layer

    Selection conclusions ultimately have to land in code. The recommendation is to add a “model routing layer” between the business and the model service, centrally managing tier, thinking tier, timeout, retry, and fallback strategies, rather than scattering model names across business modules. Here’s a minimal example (standard library only; the interface contract can be adjusted to your actual service):

    1. import os
    2. import json
    3. import urllib.request
    4. # The gateway address is injected via an environment variable so you can switch backends across environments
    5. GATEWAY = os.environ.get("LLM_GATEWAY")
    6. # Task profile: scenario type, estimated difficulty, real-time requirement
    7. def profile(task_type: str, difficulty: str, realtime: bool) -> dict:
    8. if realtime or task_type in ("extract", "rewrite", "format"):
    9. return {"tier": "economy", "thinking": "off"}
    10. if difficulty == "hard":
    11. return {"tier": "flagship", "thinking": "high"}
    12. return {"tier": "flagship", "thinking": "low"}
    13. def chat(messages, cfg, timeout=60):
    14. # The field names and values for thinking and the thinking budget follow the API docs of the service you use
    15. payload = {
    16. "model": cfg["tier"],
    17. "messages": messages,
    18. "thinking": cfg["thinking"],
    19. "stream": False,
    20. }
    21. req = urllib.request.Request(
    22. GATEWAY + "/chat/completions",
    23. data=json.dumps(payload).encode("utf-8"),
    24. headers={"Content-Type": "application/json"},
    25. )
    26. with urllib.request.urlopen(req, timeout=timeout) as resp:
    27. return json.loads(resp.read().decode("utf-8"))
    28. def ask(messages, task_type, difficulty, realtime=False, validate=None):
    29. # First try the routing layer's preferred combination
    30. try:
    31. result = chat(messages, profile(task_type, difficulty, realtime))
    32. if validate is None or validate(result):
    33. return result
    34. except Exception:
    35. pass # Exceptions such as timeouts and rate limiting fall through to fallback
    36. # Fallback safety net: flagship tier + deep thinking, to keep results usable
    37. return chat(messages, {"tier": "flagship", "thinking": "high"})
    38. if __name__ == "__main__":
    39. msgs = [{"role": "user", "content": "Please rewrite the following text in more formal written language: ..."
    40. }]
    41. # Real-time rewriting uses the economy tier with thinking off; it auto-escalates on validation failure or call errors
    42. out = ask(msgs, task_type="rewrite", difficulty="easy", realtime=True,
    43. validate=lambda r: bool(r.get("choices")))
    44. print(out)

    A few things to watch when putting this into production:

    • Separate config from code. Tier, thinking tier, timeout, and retry count should go into a config center to make gradual rollouts and quick rollbacks easy.
    • Grade the fallback instead of an all-or-nothing switch. A fallback order could be: economy off → economy light → flagship light → flagship deep. This lets you trade cost against quality smoothly.
    • Turn off thinking during tool-call stages. In a multi-step agent, steps that must output tool parameters should have thinking off, so the model doesn’t mix explanatory text into the arguments and break parsing.

    When building this kind of application on Baidu AI Cloud, you can combine several general cloud capabilities:

    • Object storage: store source documents and evaluation sets;
    • Message queues: level the load for batch/offline jobs;
    • Container platform: host the routing layer and scale it elastically;
    • Logging and monitoring/alerting: continuously observe call volume, latency, and cost.

    8. Cost and capacity: do the math on “cheap”

    Start with the bill structure. The cost of an LLM call is determined by both input tokens and output tokens, and thinking tokens usually count as output. Using hypothetical unit prices—1 yuan per million input tokens and 4 yuan per million output tokens—with 500 input tokens and 300 output tokens, a single call costs about:

    • Thinking off: 500/1000000×1 + 300/1000000×4 = 0.0017 yuan
    • Thinking on, with output rising to 1200 tokens: 500/1000000×1 + 1200/1000000×4 = 0.0053 yuan, about 3x the former

    The long-context bill is equally worth noting. Assuming the input price is unchanged, a call with 100k input tokens costs about 0.1 yuan—tens of times a normal call. If you average one million calls a day, that’s 100,000 yuan a day. So retrieval compression, cache reuse, and batching aren’t nice-to-haves but the backbone of cost control.

    Next, price and supply. Limited-time discounts at a new model’s launch are a common marketing tactic that can meaningfully lower inference cost in the short term, but they shouldn’t go straight into a long-term budget. Capacity follows a similar pattern:

    • Because the flagship tier invests more compute, its throughput tends to be relatively limited when high-end compute supply is tight, so peak periods may require queuing, fallback, or off-peak scheduling;
    • As a new generation of high-density compute clusters is deployed at scale, the unit cost of compute usually falls, leaving room for further flagship price cuts.

    Also, many vendors are migrating their training and inference foundations from the traditional GPU ecosystem to domestically produced accelerators to ease high-end compute supply constraints. Such migrations involve multi-layer tuning—operator adaptation, communication libraries, and compiler toolchains—and may see performance fluctuations along the way, but in the long run they help lower inference costs. Correspondingly, when choosing a platform, focus on three things:

    • Does it offer mature observability?
    • Does it provide capacity guarantees and fallback plans?
    • Can it stay compatible with mainstream API conventions to lower migration cost?

    Finally, self-hosting vs. API. For high-volume, stable-traffic scenarios, you can self-deploy inference based on weights open-sourced under permissive licenses like MIT. This lets you tune batching and parallelism to your needs, but you must factor in hidden costs—hardware purchase or rental, inference-framework operations, and version upgrades. For teams with spiky traffic that want to launch fast, cloud model services are usually less hassle.

    9. Pre-launch checklist

    Before going live, confirm each item:

    • Has every task type had its “tier + thinking tier” combination defined and written into the config center?
    • Is there a small regression set covering easy, medium, and hard samples?
    • Are all four metrics—accuracy, output length, P95 latency, and per-call cost—tracked together?
    • Are graded fallback and timeout safety nets implemented, and can they switch smoothly when flagship capacity is tight?
    • For long-context scenarios, has retrieval compression been evaluated and verified against the “Lost in the Middle” effect?
    • Is the cost dashboard broken down by business dimension, and are budget alerts set?
    • After a model version update, are regressions re-run and thinking tiers recalibrated?

    Summary: The essence of LLM selection is finding a balance among quality, cost, and latency, not chasing the “strongest” at all costs. The economy tier handles high-frequency, simple, low-latency tasks; the flagship tier handles tasks that depend on world knowledge and complex reasoning. Thinking modes work like a transmission: simple tasks output directly, medium tasks use light thinking, and hard problems get deep reasoning. Encoding these rules into the routing layer and the evaluation system—and continuously recalibrating with data—is what makes every token of budget count.