Multi-Model Invocation and Cost Optimization with a Unified Quota Pool
As applications increasingly aggregate multiple LLMs behind a single shared quota pool, deciding which model to call—and how to keep spend under control—becomes an ongoing engineering discipline rather than a one-time choice. This article explains how shared quota pools and per-call multipliers are metered, how to evaluate models across quality, quota consumption, latency, and stability, and how three common task types (long-document extraction, code generation, and retrieval-augmented research) differ in their cost profiles. It then lays out practical optimization techniques—tiered routing, context reduction, structured output with automatic validation, caching and batching, off-peak scheduling, usage logging, and quality safeguards—and shows how to combine them in a unified invocation layer with fallback and metering. The core takeaway: cost savings come not from finding a single "cheapest" model, but from running a closed loop of task tiering, quality evaluation, cost metering, and continuous optimization.
1. The Multi-Model Era: How Invocation Is Changing
Model selection for LLM applications is shifting. In the past, an application was typically bound to a single model, because model APIs, billing schemes, and account systems were siloed from one another and switching costs were high. Today, more and more platforms aggregate models from multiple providers into a unified quota pool: different models draw from the same balance, each call deducts quota by a shared multiplier, and some platforms even offer an auto mode that selects a model for you.
For developers, this means three shifts:
- Selection moves from “either/or” to “composed orchestration.” Different tasks can be matched to different models, balancing cost, quality, and latency together.
- Cost moves from “per-call billing” to “weighted metering.” The same call placed on different models can consume several times more or less quota.
- Optimization moves from “tuning prompts” to “tuning the call chain.” Context construction, routing strategy, cache reuse, and execution time all directly affect the bill.
To turn these possibilities into real savings, you first need to understand how quota pools are metered, then evaluate task performance with a reproducible method. Consider a real-world scenario: a team needs to process the annual reports of dozens of companies every earnings season. Doing it by hand—reading each page and entering key metrics—takes days of human effort. After connecting multi-model capabilities, the same work can be handed to an automated pipeline of “document extraction → table output → key-point summary.” But during model selection they quickly hit a contradiction: more capable reasoning models produce better results but consume more quota. So the real question becomes: how do we do the same work well while spending the least quota?
1.1 Three Basic Concepts First
Quota pool and multiplier. A quota pool is the metering balance shared by all models in an account. The multiplier is the weight applied to the quota consumed by a given call, described by the following model:
Single-call quota = (input tokens × input multiplier + output tokens × output multiplier) × time-of-day factor
Actual rules follow each platform’s official documentation, but this structure generally holds, and it yields three implications:
- Input and output are metered separately. Many reasoning models price output higher than input, so “long-output” tasks such as code generation should pay particular attention to the output multiplier.
- The multiplier is not a quality ranking. It reflects only metering weight; task fit must be measured empirically.
- The quota pool is not the same as the monetary bill. A shared pool simplifies unified management and cross-task comparison, but the monthly bill is still settled per model according to each model’s actual usage.
Context and tokens. The context is all the information a model can read in a single inference pass, including the system prompt, conversation history, and any appended material. A token is the smallest unit of text after tokenization: Chinese text is roughly “one character ≈ one token” with some fluctuation, while English, numbers, and punctuation vary with the tokenization rules. A rough rule of thumb is that every additional 1,000 Chinese characters in the context adds on the order of 1,000 tokens on the input side, scaled up by the multiplier. This is also the main source of cost for long-document tasks.
Auto mode. In auto mode, the system chooses among candidate models based on task characteristics, effectively a default routing-layer policy. It lowers the barrier to entry, but it also means the selection process is opaque to the caller. In production, you need a quality spot-check mechanism to verify that its output stays consistently on target.
1.2 From “Quota Consumption” to “Cost per Unit of Effective Output”
Evaluating multi-model calls by single-call quota alone is not rigorous. Poor-quality answers trigger follow-ups, retries, and even rework, and the hidden cost can far exceed the quota saved. Two metrics help:
Expected quota consumption = single-call quota ÷ first-pass rateCost per unit of effective output = expected quota consumption ÷ quality score
The “first-pass rate” is the proportion accepted without human edits; the “quality score” comes from a scoring rubric. If a task involves multiple turns, you should also add the history re-carried as input on each turn to the total. Multi-turn conversations usually cost far more than a single turn, which makes context management a key lever for reducing cost.
2. Evaluation Design: Four Metrics and Three Task Types
2.1 Metrics: Quality, Quota, Latency, Stability
| Dimension | What It Measures | Data Source |
|---|---|---|
| Completion quality | Accuracy, completeness, format compliance, scored against a rubric | Human or rule-based review |
| Quota consumption | Input/output token counts, corresponding multipliers, converted quota | Platform usage details |
| Response latency | Time to first token, total completion time | Invocation logs |
| Stability | Consistency across repeated runs, success rate of structured-output parsing | Repeated-run statistics |
Stability is the easiest to overlook. In real business, the consistency of output format often matters more than an occasional “brilliant” result.
2.2 Tasks: Three High-Frequency Scenarios
- Long-document understanding and structured extraction: turning dozens of pages into a key-point summary and a structured table.
- Code generation and verification: producing runnable code from a natural-language requirement and validating it with test cases.
- Retrieval-augmented research writing: generating a sourced brief grounded in retrieval results.
Covering these three task types is enough to reflect the cost structure of most teams’ day-to-day work.
2.3 Controlling Variables
- Every model uses the same prompt set and the same material;
- Parameter settings stay consistent, disabling highly random parameters where needed;
- Each task type is run multiple times, taking the median rather than a single result;
- Scoring is done against a unified rubric by people who do not know which model produced which output, to avoid bias.
2.4 Example Scoring Rubric
| Scoring Item | Points | Criterion |
|---|---|---|
| Data accuracy | 40 | Values match the source, year-over-year calculations correct |
| Completeness | 20 | No required fields missing |
| Format compliance | 20 | Directly parseable into a table or structured data |
| Traceability | 10 | Each value annotated with a source page number |
| Expression quality | 10 | Clear conclusions, no redundancy |
3. Typical Task Review
3.1 Long-Document Extraction: Heavy Input, Light Output—the Difference Shows on the Bill
Task: Read a mock annual report spanning dozens of pages and produce two outputs:
- A key-point summary of no more than 800 words, covering operating performance, revenue and profit changes and their causes, profit quality, cash-flow and accounts-receivable risks, and subsequent events;
- A key financial metrics table containing revenue, net profit attributable to the parent, net profit attributable to the parent excluding non-recurring items, net operating cash flow, R&D investment and its share of revenue, and accounts receivable—each with the current-year value, prior-year value, year-over-year change, and source page—output as a table file.
Example prompt:
[Role] You are a financial analyst.[Material] The attachment is an excerpt from a listed company's annual report, spanning several pages.[Task]1. Write a highlights summary of no more than 800 words, addressing in order:(1) the overall operating performance for the reporting period;(2) changes in revenue and net profit, and the main reasons;(3) profit quality, such as the impact of non-recurring gains/losses and changes in accounting estimates;(4) risks relating to cash flow, accounts receivable, and customer concentration;(5) significant subsequent events that may affect future performance.2. Output a key financial metrics table with the following fields:Metric | Current-year value | Prior-year value | YoY change | Source page- Metric scope: revenue, net profit attributable to the parent,net profit attributable to the parent excluding non-recurring items,net operating cash flow, R&D investment and its share of revenue, accounts receivable- Use a single amount unit (ten-thousand yuan), keep two decimal places, express YoY as a percentage[Constraints]- Data must match the source; do not compute values not given in the source- Each data item must be annotated with a source page number- Output in JSON format: {"summary": "...", "metrics": [...]}
Evaluation observations: When the material is complete, mainstream models can all handle structured extraction. The real gap shows up in three areas:
- Localization ability in long context: the longer the material, the higher the chance of missed reads and mis-matched page numbers.
- Consistency of values and units: mixing ten-thousand yuan and hundred-million yuan units, and wrong signs on year-over-year changes, are frequent issues.
- Cost structure: the output is short text plus structured data, while the input is the entire long document. Quota is almost entirely consumed on the input side, and differences in the input multiplier are preserved in full, and even amplified.
For this kind of task, favor models with input multipliers that are relatively low and that match long-context and document-understanding needs. For extremely long documents, extract in segments first, then merge and cross-check.
3.2 Code Generation: Heavy Output, Fast Feedback—the Gap Is in First-Pass Rate
Task: Generate a reimbursement calculation tool that finance can use directly, with these requirements:
- Support categories such as travel, transport, meals, and office supplies;
- Amounts over a per-unit cap must be self-paid;
- Calculate a travel allowance based on the number of accommodation days;
- Flag “manager approval required” when the total exceeds an approval threshold;
- Output the details as a spreadsheet file.
Example code (core calculation logic):
from decimal import Decimal, ROUND_HALF_UP# Per-category reimbursement limits (yuan/day or yuan/instance)LIMITS = {"travel": Decimal("800"), # business travel accommodation"traffic": Decimal("200"), # local transport"meal": Decimal("100"), # meals"office": Decimal("500"), # office supplies}# Per diem standard (yuan/day) and approval threshold (yuan)PER_DIEM = Decimal("120")APPROVAL_THRESHOLD = Decimal("5000")CENT = Decimal("0.01")def settle(claims, trip_days=0):"""Compute reimbursement details: amounts over the limit are self-paid, and nights must not exceed trip days."""if trip_days < 0:raise ValueError("trip days cannot be negative")rows, payable = [], Decimal("0")for c in claims:category = c["category"]if category not in LIMITS:raise ValueError(f"unknown reimbursement category: {category}")amount = Decimal(str(c["amount"]))days = int(c.get("days", 1))if amount < 0 or days <= 0:raise ValueError("amount must be non-negative, days must be a positive integer")if category == "travel" and days > trip_days:raise ValueError("nights cannot exceed trip days")quota = LIMITS[category] * daysreimbursed = min(amount, quota)payable += reimbursedrows.append({"category": category,"claimed": str(amount),"limit": str(quota),"reimbursed": str(reimbursed),"self_paid": str(amount - reimbursed),})if trip_days > 0:payable += PER_DIEM * trip_dayspayable = payable.quantize(CENT, rounding=ROUND_HALF_UP)return {"items": rows,"allowance": str(PER_DIEM * trip_days) if trip_days > 0 else "0","total": str(payable),"need_approval": payable >= APPROVAL_THRESHOLD,}# Example: 2 nights of accommodation totaling 1000 yuan, meals 300 yuan, 2 trip daysclaims = [{"category": "travel", "amount": 1000, "days": 2},{"category": "meal", "amount": 300, "days": 1},]result = settle(claims, trip_days=2)# Accommodation limit 800×2=1600 yuan, fully reimbursed;# Meal limit 100×1=100 yuan, 200 yuan self-paid;# Allowance 120×2=240 yuan; total 1440 yuan, no manager approval requiredassert result["total"] == "1440.00"assert result["need_approval"] is False
Evaluation observations: Three difficulties dominate code tasks: misunderstanding requirements (“per unit” — per day or per instance?), edge-case handling (over-limit amounts, empty lists, negative values), and monetary precision (floating-point error). This kind of task produces noticeably more output tokens than extraction tasks, so the output multiplier has a larger effect. More importantly, code is only truly usable when it “compiles, runs, and passes all cases” — so the first-pass rate is the main discriminating metric.
If a team uses intelligent coding tools to assist development, the same principles apply: choose the model by task complexity, and replace manual line-by-line review with automated tests.
3.3 Retrieval-Based Research: The Longest Chain—Watch Latency and Verifiability
Task: Research coffee consumption trends in recent years and produce a sourced brief for management, covering market-size changes, price-band distribution, channel structure, and potential risks.
Evaluation observations: Retrieval-augmented tasks pass through several stages — “retrieve → read → synthesize → cite.” If the chain includes multiple model calls, each round re-carries the retrieved material, so token consumption accumulates. This kind of task should focus on:
- Whether the retrieval results are actually used, rather than answered from memory;
- Whether citation sources are traceable and verifiable;
- Whether overall latency is within a range the business can accept.
A common engineering practice is “retrieve and rerank first, then summarize and compress, and generate last,” sending only the content most relevant to the question into the model.
4. Optimization Paths: Bring Cost Down, Keep Quality Up
4.1 Tiered Routing: Light Loads for Simple Tasks, Heavy Forces for Complex Ones
| Task Characteristics | Recommended Tier | Rationale |
|---|---|---|
| Intent recognition, format conversion, field extraction | Lightweight tier | Clear rules, low error tolerance; lightweight models suffice |
| Code generation, complex reasoning, long-document analysis | Reasoning-enhanced or code-specialized tier | First-pass rate takes priority; rework costs are higher |
| Creative writing, summary polishing | General balanced tier | Balances quality and latency |
| Research writing | General balanced tier + retrieval pipeline | Quality and timeliness depend on external information |
Routing strategy should link to business rules. For example, in a reimbursement scenario, receipts below the approval threshold can go to a lightweight model with automatic validation first; those with large amounts or anomalous fields are then escalated to a stronger reasoning model and routed to human review.
4.2 Context Reduction
- On-demand injection: carry only the history needed for the current turn, summarizing and compressing early conversation first.
- Retrieval up front: for long documents, recall relevant passages via keyword or vector retrieval rather than stuffing in the whole document.
- Trim redundant system prompts: remove repeated phrasing and ineffective examples; the system prompt is re-consumed on every turn.
4.3 Structured Output and Automatic Validation
Structured output is the prerequisite for automation. Give a clear JSON schema example in the prompt, and on parse failure, automatically retry or switch to a fallback model:
import jsondef extract_json(text):"""Parse JSON from model output, tolerating Markdown code block wrapping."""start, end = text.find("{"), text.rfind("}")if start == -1 or end <= start:raise ValueError("No JSON object found in output")return json.loads(text[start:end + 1])def validate_table(data, required_fields):"""Validate extraction result: fields present, types correct, page number a positive integer."""rows = data.get("metrics", [])for row in rows:missing = set(required_fields) - set(row)if missing:raise ValueError(f"Missing fields: {missing}")if not isinstance(row.get("source_page"), int) or row["source_page"] < 1:raise ValueError("Invalid source page number")if not data.get("summary"):raise ValueError("Summary is empty")return datadef run_extraction(model_client, prompt, fields):for name in model_client.candidates: # candidate models ordered from lowest to highest costtry:data = extract_json(model_client.call(name, prompt))return validate_table(data, fields)except (ValueError, json.JSONDecodeError, TimeoutError):continueraise RuntimeError("No candidate model produced a compliant result")
Decoupling business rules from model output is also an important way to control cost. Implementing reimbursement validation in code, with the model responsible only for understanding and extraction, avoids rules being “freely interpreted” and reduces the cost of repeated correction.
4.4 Caching and Batching
- Exact caching: keyed on “model + full request,” returning directly on a hit; suited to repetitive tasks with fixed templates.
- Semantic caching: reuse results for high-frequency similar questions via vector retrieval; set a similarity threshold and allow a short expiry.
- Batching and asynchrony: submit non-real-time batch tasks asynchronously, often combined with platform batch discounts to lower unit cost.
- Result reuse: persist extraction results for a given document so downstream tasks—summaries, Q&A, comparisons—can share them, avoiding repeated parsing.
4.5 Off-Peak Scheduling
Some platforms offer quota discounts outside weekday daytime hours or on holidays. For tasks without tight time constraints—overnight batch reports, non-urgent research—schedule them in discount windows. A scheduler dynamically chooses execution time and model based on “time-of-day factor × model multiplier,” shifting deferrable tasks away from peak hours.
4.6 Usage Governance and Observability
Recording a structured log for every call is the foundation of cost governance:
{"task_id": "report_extract_20250612_001","task_type": "extract","model": "<the model actually called>","route": "auto","prompt_tokens": 18500,"completion_tokens": 1200,"quota_cost": 39.4,"latency_ms": 12800,"retry": 0,"quality_score": 92}
With logs in place, you can tally quota consumption by team, business line, and task type, identify high-cost low-value steps, and provide a basis for budget allocation and alert thresholds.
4.7 Quality Safeguards
- Layered validation: use rule checks for format issues, cross-checks for factual issues (e.g., comparing extraction results against source passages), and bring in a second model for review on complex judgment calls.
- Auto mode still needs spot checks: auto mode suits exploration and low-risk scenarios; once in production, spot-check its output at a fixed ratio and track the first-pass rate.
- Keep human review at key nodes: content involving amounts, compliance, and external publication should retain a human review step.
5. Engineering Implementation: A Unified Invocation Layer and Fallback Design
Multi-model capability ultimately has to land on a stable engineering implementation. A maintainable unified invocation layer should include at least these elements:
- Model registration and routing configuration: model identifiers, multipliers, and capability tags configured centrally, with routing strategy decoupled from business code.
- Timeout, retry, and fallback: automatically switch to the next candidate when the primary model times out or is rate-limited.
- Usage collection: read token usage from the response, convert it to quota with the multipliers, and write it to the log.
- Limits and circuit breaking: set quota caps per user or task to prevent anomalous calls from draining the shared pool.
import timefrom dataclasses import dataclass, field@dataclassclass ModelSpec:name: strinput_multiplier: float # input multiplier, per the platform's official documentationoutput_multiplier: floatcapabilities: set = field(default_factory=set)@dataclassclass Usage:prompt_tokens: int = 0completion_tokens: int = 0quota: float = 0.0model: str = ""latency_ms: int = 0class UnifiedModelClient:"""Unified invocation layer: routing + fallback + metering, abstracting away model differences."""def __init__(self, registry, time_factor_fn, max_attempts=3, timeout=30):self.registry = registry # model id -> ModelSpecself.time_factor = time_factor_fn # returns the time-of-day factor for the current timeself.max_attempts = max_attemptsself.timeout = timeoutdef route(self, task_type, prefer=None):"""Filter candidates by capability tag, sorted by input multiplier ascending."""names = [prefer] if prefer else self.registry.by_capability(task_type)specs = sorted((self.registry.get(n) for n in names), key=lambda s: s.input_multiplier)return specs[:self.max_attempts]def chat(self, prompt, task_type, prefer=None):last_error = Nonefor spec in self.route(task_type, prefer):start = time.monotonic()try:raw = self.registry.transport(spec.name).chat(prompt, timeout=self.timeout)usage = Usage(prompt_tokens=raw.usage.prompt_tokens,completion_tokens=raw.usage.completion_tokens,model=spec.name,latency_ms=int((time.monotonic() - start) * 1000),)usage.quota = round((usage.prompt_tokens * spec.input_multiplier+ usage.completion_tokens * spec.output_multiplier)* self.time_factor(), 2)self.registry.collector.record(task_type, usage)return raw.textexcept Exception as exc: # timeout, rate limiting, transient failures, etc.last_error = excraise RuntimeError(f"All candidate models unavailable: {last_error}")
On the security and compliance side, input sent to the model service should go through sensitive-information redaction and prompt-injection detection, and output should go through content-safety filtering. Follow the principle of least privilege: be explicit about which data may be sent to the model and in what form.
6. Limitations and Considerations
- Continuous versioning: model capabilities and multipliers change with version updates, so evaluation conclusions should be re-tested periodically rather than “measured once and reused indefinitely.”
- Scoring bias: automated scoring is easily influenced by factors such as output length and formatting; key tasks still need human review.
- Auto mode is opaque: it is recommended to track auto mode’s quality separately by task type, and switch to explicit model selection when needed.
- Hidden costs: retries, rework, and human-review time should all be counted in total cost, not just the quota figures.
- Data compliance: for tasks involving personal information or commercially sensitive data, favor invocation methods that meet your organization’s data-compliance requirements, and perform local redaction or private deployment when necessary.
7. Conclusion
A unified quota pool turns model selection from a “one-time decision” into “continuous operations.” The key to reducing cost lies not in finding some single “cheapest” model, but in building a closed loop of “task tiering → quality evaluation → cost metering → continuous optimization”:
- Use a unified rubric and controlled evaluation to find each task type’s quality threshold;
- Among the models that clear it, choose the one with the lowest quota consumption;
- Use context reduction, cache reuse, off-peak scheduling, and fallback design to push per-task cost into a reasonable range;
- Use usage logs and quality spot checks to keep verifying that cost optimization has not come at the expense of quality.
On enterprise-grade model platforms such as Baidu AI Cloud’s Qianfan platform, unified multi-model access, token usage metering, and cost management are becoming standard platform capabilities. For enterprises, real competitiveness lies not in how many models they use, but in whether they have built model-selection and cost-governance capabilities matched to their own business.
