LLM API Selection in Practice: How to Choose Model Tiers and Use Thinking Mode

Background: Tiered Models and Thinking Toggles Are Becoming Standard

When mainstream model providers ship an LLM API today, they rarely offer a single “does-everything” model. Instead, the offering is split into multiple tiers: one geared toward high performance, with stronger reasoning and a higher unit price; another geared toward cost efficiency, with faster responses and lower prices. At the same time, more services are introducing a “thinking mode” toggle that lets developers control whether the model performs explicit long-chain reasoning before producing its final answer.

These look like product features, but in essence they hand compute allocation decisions to developers: spend compute and budget on deep reasoning for hard tasks, and push simple tasks through in the cheapest possible way. Choose well, and the same budget runs several times more calls; choose poorly, and you either lose control of costs or ship quality that falls short and triggers rework. On cloud platforms such as Baidu AI Cloud, LLM services are likewise offered as multiple tiers with optional reasoning depth, and the selection logic is the same. This article lays out an actionable selection methodology in five parts: tier positioning, cost estimation, thinking mode, compute supply trends, and engineering practice.

How the Two Tier Types Differ

Flagship tier (high performance)

• Capabilities: Larger parameter scale and deeper reasoning; more reliable on complex math, algorithmic code, long-context understanding, multi-step planning, and fine-grained instruction following.
• Cost: Higher unit prices, typically billed separately for input and output tokens, with output priced noticeably above input.
• Supply: Heavily dependent on high-end compute. Some providers cap concurrency and throughput for flagship tiers when compute is tight, which shows up as peak-hour rate limiting, queuing, or slower responses.

Lightweight tier (cost-efficient)

• Capabilities: Built for high-frequency, short-context, pattern-stable tasks such as classification, intent recognition, information extraction, text rewriting, and simple summarization.
• Cost: Can be a fraction of flagship pricing; some new-generation lightweight tiers already price output in the single-digit CNY per million tokens range.
• Supply: Small compute footprint with high throughput and low latency, well suited to large offline batches and high-concurrency online serving.

Mapping tasks to tiers

Task type	Recommended tier	Thinking mode
Math derivations, algorithm problems, hard debugging	Flagship	On
Long-document analysis, cross-paragraph reasoning	Flagship	Depends on complexity
Code generation, refactoring suggestions	Flagship	On
Text classification, sentiment analysis	Lightweight	Off
Field extraction, format conversion	Lightweight	Off
Everyday Q&A, small-talk fallback	Lightweight	Off

Core principle: bucket tasks by difficulty first, then assign a tier to each bucket—don’t run the entire pipeline on a single model.

Cost Estimation: Turning “Feels Expensive” into “Calculable”

The basic billing model

LLM APIs almost universally bill by token, with input and output priced separately; output typically costs several times more than input. Monthly cost can be abstracted as:

Monthly cost = Σ(input tokens × input price + output tokens × output price) × average daily calls × 30

A worked example

Assumptions (for illustration only—check the provider’s official pricing for actual numbers): lightweight tier input at 0.5 CNY per million tokens and output at 2 CNY per million tokens; flagship tier output at roughly 6 CNY per million tokens after discount.

Consider a customer-service assistant making 5,000 calls per day, with 2,000 input tokens and 500 output tokens per call:

• Input: 5,000 × 2,000 = 10 million tokens/day; at 0.5 CNY per million ≈ 5 CNY/day
• Output: 5,000 × 500 = 2.5 million tokens/day; at 2 CNY per million ≈ 5 CNY/day
• Lightweight tier total: about 10 CNY/day, or roughly 300 CNY/month. If every call ran on the flagship tier, output alone would climb to 15 CNY/day; add the input price gap and the extra output tokens that thinking mode generates, and total cost could multiply several times over.

The example makes two points: output tokens dominate cost, so trimming output length delivers the most direct savings; and tier price gaps are usually multiples, not percentages, so savings from routing are substantial.

Common cost-reduction techniques

• Prompt slimming: Remove irrelevant conversation history and redundant instructions; input tokens drop immediately.
• Prefix caching: Reuse stable prefixes in multi-turn conversations; cached portions are usually billed at a lower rate.
• Output constraints: Cap length with max_tokens and use structured output to cut filler.
• Batch processing: Run offline workloads through batch APIs, trading off-peak compute for lower unit prices.
• Result caching: Serve cached answers for identical or near-identical questions without calling the model.

Thinking Mode: A Paid Accelerator for Deep Reasoning

What thinking mode is

When enabled, the model generates a stretch of intermediate reasoning before producing the final answer, markedly improving success rates on math, code, and multi-step logic. The trade-off is equally clear: the thinking content also consumes output tokens, so costs rise, and both first-token latency and total latency increase noticeably.

When to turn it on

• Math computation, formula derivation, operations planning
• Writing and reviewing algorithms and complex code; localizing stubborn bugs
• Logic checks and rule comparisons that require strict step-by-step verification
• Key decision points in an agent, such as tool-call orchestration

When to turn it off

• Pattern-driven tasks: classification, extraction, rewriting, translation
• Latency-sensitive, high-frequency online calls
• Simple Q&A and greetings
• Simple offline batch jobs

Rule of thumb: if the answer is obvious at a glance, thinking is pure waste; if a wrong answer is expensive, the accuracy gain from thinking far outweighs the token cost.

How to control it (illustrative)

Parameter names vary across platforms. Common forms include a boolean switch or a “thinking budget” field; the latter caps thinking tokens, giving you a continuous dial between quality and cost:

{
  "model": "llm-pro",
  "messages": [{"role": "user", "content": "Review this code for concurrency safety issues"}],
  "thinking": {"enabled": true, "budget_tokens": 4096},
  "max_tokens": 4096
}

Field names and values should follow your platform’s official documentation; the example above only illustrates the design. In production, you can also add a simple routing layer:

def route(task):
    if task.type in ("math", "code_review", "planning"):
        return call_llm(task.prompt, tier="pro", thinking=True)
    if task.type in ("classify", "extract", "rewrite"):
        return call_llm(task.prompt, tier="lite", thinking=False)
    return call_llm(task.prompt, tier="lite", thinking=False)  # conservative default tier

Compute Supply: Tier Prices Are Not Constants

One more underlying shift deserves attention: some newer models, at release, no longer rely on the traditional mainstream compute ecosystem, having moved training and inference validation onto alternative compute systems. Migrations at this scale are major engineering efforts, and they are one reason a few model launches have slipped.

Supply shapes price. When high-end compute is scarce, flagship throughput is constrained and prices stay elevated. As large-scale compute clusters—such as supernode-class AI clusters—are delivered in volume and unit compute costs fall, there are already public industry expectations that flagship-tier prices will drop significantly once supply eases.

Two implications for selection:

1. Don’t treat current prices as long-term constants. Build a model abstraction into your access layer and keep tiers, models, and providers configurable and swappable, so you capture the next price cut as soon as it lands.
2. Have a plan for throughput limits. Flagship tiers may be rate-limited, so build retry, queuing, and fallback strategies into the pipeline—for example, automatically degrading to the lightweight tier at peak times, or switching to asynchronous batch processing.

Engineering Checklist

1. Unified API gateway: Abstract away differences between tiers and models, and manage the mappings in a configuration center so switching never touches business code.
2. Task routing: Split traffic by task type and difficulty; simple tasks should never reach the flagship tier.
3. Thinking-mode rollout: Enable it on a small share of traffic first; compare accuracy, latency, and cost before expanding.
4. Fallback chain: Flagship rate-limited or timed out → automatically switch to the lightweight tier → failures go into a queue for asynchronous retry.
5. Usage observability: Track token usage, unit price, latency, and error rate per business line so cost anomalies can be traced to a specific endpoint immediately.
6. Quality evaluation set: Keep a small set of human-labeled samples for downgrade decisions; run evaluations before any tier change ships.
7. Cache first: Exhaust the trio of cached responses for frequent identical requests, stable-prefix caching, and batch processing before optimizing the model itself.

Wrap-Up

Tier selection and the thinking toggle are two knobs for spending compute where it matters most: the tier decides how much compute you use, and thinking mode decides how deep the reasoning goes. The supporting engineering work is routing, fallback, caching, and monitoring. Get that combination right, and as LLM call volumes grow fast, you can hold both the cost curve and the quality curve steady. And as new compute supply continues to scale up, flagship prices still have meaningful room to fall—keep your architecture flexible so every price cut becomes your gain.