Multi-Model Routing and Cost Control for Office AI Agents

    Office AI agents handle tasks with very different context-length and reasoning demands. This article explains when multi-model routing is worth the added complexity, how to implement a unified model interface and router, how to calculate quota multipliers and off-peak discounts, and how to coordinate elastic compute while protecting billing accuracy.

    AI agents in office scenarios handle very different tasks. Sending a notification email may take only a few seconds, while writing an industry analysis report may require reading a large amount of material and performing multiple rounds of reasoning. The two have completely different requirements for model capability and quota. With only one model, simple tasks waste quota, while complex tasks may underperform. As a result, many teams connect multiple models and route tasks by type. This article explains the layered design, routing code, quota calculation method, and key points for coordinating off-peak hours and elastic compute. The numbers in this article are examples only and do not represent the pricing of any real product.

    When Is Multi-Model Routing Needed? Check These Three Conditions First

    A multi-model architecture is an engineering choice, not a mandatory option for every product. When there are few task types and quality requirements are uniform, a single model is simpler and has lower maintenance cost.

    Routing brings clear benefits only when the following three conditions are all true.

    1. The gap in context length and reasoning depth between tasks is large.
    2. Cost has a clear impact on the product, and quota needs fine-grained control.
    3. The team can maintain a model registry and routing rules and regularly verify prices.

    If only one of these conditions is met, start with a single model. Split tasks only after data shows that it is necessary.

    Layered Design: The Business Layer Only Sees a Unified Interface

    The key to implementation is decoupling. The business layer calls only a unified model interface and does not directly depend on any vendor SDK. The unified abstraction layer handles parameter conversion, authentication, and timeout control. The routing engine selects a model based on task type, context length, and cost ceiling.

    When adding or retiring a model, only the registry needs to be changed; business code does not need to be modified.

    How to Write the Routing Code

    The following code uses Python 3.9 or later and does not depend on third-party packages.

    1. from typing import Optional
    2. # Example values, not real pricing
    3. MODEL_META = {
    4. "light-v1": {"context_len": 32000, "cost_multiplier": 1,
    5. "strengths": ["email_notice", "summary"],
    6. "input_price": 0.001, "output_price": 0.002},
    7. "heavy-v1": {"context_len": 200000, "cost_multiplier": 5,
    8. "strengths": ["logic_reasoning", "industry_report"],
    9. "input_price": 0.005, "output_price": 0.01},
    10. "ultra-v1": {"context_len": 200000, "cost_multiplier": 10,
    11. "strengths": ["industry_report"],
    12. "input_price": 0.01, "output_price": 0.02},
    13. }
    14. # Minimum context length required by each task type (unit: tokens), example values
    15. TASK_CONTEXT_REQUIREMENT = {
    16. "email_notice": 4_000,
    17. "summary": 16_000,
    18. "logic_reasoning": 32_000,
    19. "industry_report": 128_000,
    20. }
    21. UNKNOWN_TASK_CONTEXT = 128_000 # Unknown task types are treated as the most conservative long-context case
    22. def task_context_needed(task_type: str) -> int:
    23. return TASK_CONTEXT_REQUIREMENT.get(task_type, UNKNOWN_TASK_CONTEXT)
    24. class ModelRouter:
    25. def __init__(self, model_registry: list[dict]):
    26. # Each item format: {"id": str, "context_len": int, "cost_multiplier": float, "strengths": list[str]}
    27. self.models = model_registry
    28. def select_model(self, task_type: str, max_cost_multiplier: float) -> Optional[str]:
    29. if max_cost_multiplier <= 0:
    30. raise ValueError("max_cost_multiplier must be greater than 0")
    31. need = task_context_needed(task_type)
    32. candidates = [m for m in self.models if m["context_len"] >= need]
    33. affordable = [m for m in candidates if m["cost_multiplier"] <= max_cost_multiplier]
    34. if not affordable:
    35. return None # The caller should prompt the user to raise the quota ceiling or split the task
    36. matched = [m for m in affordable if task_type in m["strengths"]]
    37. pool = matched or affordable
    38. # When multipliers are equal, sort by id to keep results consistent across instances
    39. best = min(pool, key=lambda m: (m["cost_multiplier"], m["id"]))
    40. return best["id"]
    41. registry = [{"id": mid, **meta} for mid, meta in MODEL_META.items()]
    42. router = ModelRouter(registry)
    43. print(router.select_model("email_notice", 3)) # light-v1
    44. print(router.select_model("industry_report", 5)) # heavy-v1
    45. print(router.select_model("industry_report", 3)) # None

    In the output, the third call returns None. An industry report requires 128K context, and the two models that meet the length requirement have multipliers of 5 and 10, both above the ceiling of 3.

    The caller must handle this empty value and must not pass it directly to the model interface.

    1. model_id = router.select_model("industry_report", 3)
    2. if model_id is None:
    3. print("The quota ceiling is too low. Increase max_cost_multiplier or split the task into multiple subtasks.")
    4. else:
    5. call_model(model_id) # Call your own unified interface

    How to Calculate the Quota Consumption Multiplier

    The quota consumption multiplier converts the costs of different models into the same virtual quota. There are two types of numbers to distinguish here.

    The first type is the internal cost unit prices input_price and output_price, calculated per 1,000 tokens, reflecting real cost. The second type is cost_multiplier, set by the product side to create a quota gap between different models.

    The base cost is multiplied by the multiplier and then by the discount coefficient to get the final quota. The multiplier should be multiplied only once in the calculation function; do not multiply it repeatedly elsewhere in the call chain.

    Append the following code to the end of the previous file to run it.

    1. from datetime import datetime, timezone
    2. from typing import Callable, Optional
    3. from zoneinfo import ZoneInfo # Python 3.9+; on Windows, run pip install tzdata
    4. PRICE_UNIT_TOKENS = 1000 # Unit price is calculated per 1,000 tokens
    5. BILLING_TZ = ZoneInfo("Asia/Shanghai") # Billing time zone is unified to the server time zone
    6. OFF_PEAK_HOURS = (0, 8) # Off-peak hours: 00:00 to 08:00 Beijing time, left-closed and right-open
    7. def get_model_meta(model_id: str) -> dict:
    8. if model_id not in MODEL_META:
    9. raise KeyError(f"Unregistered model: {model_id}") # Do not handle silently
    10. return MODEL_META[model_id]
    11. def is_off_peak(now: datetime) -> bool:
    12. if now.tzinfo is None:
    13. raise ValueError("now must include time zone information")
    14. hour = now.astimezone(BILLING_TZ).hour
    15. start, end = OFF_PEAK_HOURS
    16. return start <= hour < end
    17. def get_discount_rate(now: datetime, fetch_rule: Optional[Callable[[], dict]]) -> float:
    18. # If there is no rule source, the query fails, or the value is invalid, bill at the original price (coefficient 1.0)
    19. if fetch_rule is None:
    20. return 1.0
    21. try:
    22. rate = float(fetch_rule()["discount_rate"])
    23. except Exception:
    24. return 1.0 # In a real project, log and alert here
    25. if not (0 < rate <= 1):
    26. return 1.0
    27. return rate if is_off_peak(now) else 1.0
    28. def calculate_quota(model_id: str, input_tokens: int, output_tokens: int,
    29. now: Optional[datetime] = None,
    30. fetch_rule: Optional[Callable[[], dict]] = None) -> float:
    31. if input_tokens < 0 or output_tokens < 0:
    32. raise ValueError("Token counts cannot be negative")
    33. meta = get_model_meta(model_id)
    34. base_cost = (input_tokens * meta["input_price"]
    35. + output_tokens * meta["output_price"]) / PRICE_UNIT_TOKENS
    36. now = now or datetime.now(timezone.utc)
    37. discount = get_discount_rate(now, fetch_rule)
    38. quota = base_cost * meta["cost_multiplier"] * discount # The multiplier is multiplied only here, once
    39. return round(quota, 4)
    40. def rule_offpeak():
    41. return {"discount_rate": 0.5}
    42. t_night = datetime(2026, 11, 20, 2, 0, tzinfo=ZoneInfo("Asia/Shanghai"))
    43. t_day = datetime(2026, 11, 20, 14, 0, tzinfo=ZoneInfo("Asia/Shanghai"))
    44. print(calculate_quota("light-v1", 2000, 500, t_night, rule_offpeak)) # 0.0015
    45. print(calculate_quota("light-v1", 2000, 500, t_day, rule_offpeak)) # 0.003
    46. print(calculate_quota("heavy-v1", 100000, 4000, t_night, rule_offpeak)) # 1.35

    The table below corresponds to the three calls above. The values are only for illustrating the calculation process.

    Item Example A (night) Example B (daytime) Example C (night, industry report)
    Model light-v1 light-v1 heavy-v1
    Input / output tokens 2000 / 500 2000 / 500 100000 / 4000
    Base cost 0.003 0.003 0.54
    cost_multiplier 1 1 5
    discount_rate 0.5 1.0 0.5
    Final quota 0.0015 0.003 1.35

    Although Example B has an off-peak rule available, daytime is not within the off-peak period, so the coefficient is 1.0.

    What to Watch Out for in Off-Peak Billing

    Off-peak hours are determined based on server time. In the code, time is uniformly converted to Asia/Shanghai to avoid billing deviations caused by different client time zones. Times stored on the server must include time zone information; local time without a time zone must not be used for comparison.

    The discount coefficient must be greater than 0 and no more than 1. If the rule query fails, a field is missing, or the value is invalid, the code bills at the original price. Users may pay slightly more, but the business side will not underbill. If degradation occurs frequently during a certain period, it means the rule service has a problem and should be fixed as soon as possible rather than making users pay more over the long term.

    Quota is rounded to four decimal places. The exact precision should be consistent with financial rules.

    How Elastic Compute Fits In and Where the Risks Are

    The role of elastic compute is to run tasks that can be delayed on off-peak hours or idle resources. It is suitable for tasks such as nightly batch report generation, offline archiving, and batch rewriting. It is not suitable for email notifications or online Q&A that require immediate responses.

    When coordinating elastic compute, pay attention to four types of risk.

    • Queueing: Compute resources may be tight during off-peak hours. Tasks need a maximum waiting time; after timeout, they should move to the normal queue and be billed at the original price.
    • Retries: Timeout retries may cause duplicate charges. Each task should carry an idempotency key, and the same task should be billed only once.
    • Price changes: cost_multiplier and off-peak hours may be adjusted. Bills need to record the rule version used at the time.
    • Model switching: If a model is switched while a task is running, the recorded quota should be based on the model actually called.

    Pre-Launch Checklist

    Confirm the following items one by one before launch.

    • Registry fields are complete, including context_len, cost_multiplier, strengths, input_price, and output_price.
    • All example prices have been replaced with the product’s real internal basis and confirmed by finance.
    • The billing time zone is configured as Asia/Shanghai, and server time includes time zone information.
    • Off-peak rules have caching and failure retries, and failure events enter monitoring and alerting.
    • Each call records the model id, token counts, multiplier, discount coefficient, and final quota for later review.
    • Unknown task types trigger alerts and do not silently use the long-context rule.
    • The message shown when quota is insufficient is live, and users know how to raise the ceiling or split tasks.
    • The reconciliation script can compare call logs with bills by day and alert when differences exceed a threshold.
    • The user agreement or billing description states that billing is at the original price when a query fails and defines the compensation method.

    Common Errors and Reconciliation Methods

    Several types of errors occur frequently in real projects.

    • The calculation function forgets to return a result at the end, the caller gets None, and all subsequent billing is wrong.
    • The multiplier is multiplied repeatedly in multiple places, and the same call is billed twice.
    • Off-peak determination uses client time, so users in different regions are charged inconsistently.
    • When a rule query fails, the code throws an error directly and interrupts the whole task. It should degrade to the original price and keep logs.
    • After routing returns None, the caller does not handle it and passes it directly to the model interface.

    During reconciliation, first aggregate the token counts and final quota for each model from the call logs, then compare them line by line with the bill. Differences mainly come from three causes: time zone conversion, rounding precision, and duplicate billing from retries. Check each category to locate the problem.