How Office AI Agents Integrate Multiple Models: Selection, Billing, and Practice

    Office scenarios involve tasks of widely varying complexity, which makes relying on a single LLM inefficient in terms of quality, speed, and cost. This article explains how to design a multi-model office agent: task grading, model capability profiles, quota-and-multiplier billing, routing architecture, pre-execution cost estimation, and budget control, with pseudocode examples for routing and cost estimation.

    1. Why Office Agents Need Multiple Models

    Tasks in office scenarios differ greatly. Summarizing meeting notes and extracting fields from tables are well-structured tasks with short outputs; they require limited reasoning depth and place more weight on response speed and cost. Writing industry analyses, comparing information across documents, and building multi-step work plans demand stronger reasoning and long-context capabilities, and users are far less tolerant of poor results.

    A single model can rarely be optimal across quality, speed, and cost at the same time. Routing everything to a high-capability model wastes resources on simple tasks and raises user costs. Routing everything to a lightweight model increases failure rates on complex tasks, forcing users to revise outputs repeatedly and actually reducing efficiency. A common design, therefore, is to integrate several models into one product and let the system—or the user—select the right model for each task.

    This design imposes three requirements on the system. First, it must describe the capability and cost characteristics of each model. Second, it must map tasks to suitable models. Third, it must clearly show users the cost differences that each choice implies. The following sections cover these three points in turn.

    2. Core Concepts

    2.1 Task Grading

    Before routing, the system needs to assess task difficulty. Common grading criteria include:

    • Input size: document page count, character count, number of attachments;
    • Reasoning depth: whether multi-step derivation, cross-source comparison, or numerical calculation is required;
    • Output constraints: whether a structured format is required and whether strict field validation applies;
    • Latency requirements: whether the user is waiting for a result in a real-time conversation.

    A simple approach is to divide tasks into three levels: lightweight (summarization, translation, format conversion), standard (information extraction, content rewriting, table analysis), and complex (multi-document reasoning, strategy planning, long-chain tasks). Grading does not need to be highly precise; it only needs to avoid obvious mismatches in most cases. Grading rules should be documented with typical examples so they can be maintained and reviewed later.

    2.2 Model Capability Profiles

    Each model should maintain a capability profile containing at least the following fields: context window size, offline evaluation scores across task types, average time to first token, average full-output latency, and a billing coefficient per unit of usage. Because model versions iterate quickly and old data becomes stale, profiles must be refreshed regularly, with the data collection time recorded.

    2.3 Quotas and Multipliers

    Many products do not expose raw token usage directly. Instead, they use a “quota” as the billing unit, converting the resource consumption of different models into a single number that users can understand. Conversion typically uses a multiplier:

    Quota consumed per task = base usage × model multiplier × time factor

    The multiplier is a consumption coefficient relative to a baseline model; the lower the multiplier, the fewer quota units are consumed for the same task. The time factor enables off-peak discounts and can be less than 1 during non-peak hours.

    For example, suppose a task has a base usage of 1,000 metering units and the baseline multiplier is 1.0. Model A, with a multiplier of 0.5, consumes 500 quota units; Model B, with a multiplier of 2.0, consumes 2,000 quota units. These figures are only for illustrating the calculation and do not represent the pricing of any actual product.

    2.4 Cost Accumulation in Multi-Step Tasks

    Agent-style products decompose a goal into multiple sub-steps, such as “retrieve materials, extract key points, generate a table, write the conclusion.” Each step may call a model, so total cost is the sum of per-step costs and is further affected by retry counts and tool-call counts. Looking only at the price of a single conversation underestimates the actual spend. Billing and budget mechanisms should use “completing one goal” as the statistical granularity, not “one API call.”

    3. Routing Architecture Design

    A typical processing pipeline looks like this:

    User request → task parsing → routing decision → model gateway → execution and tool calls → result validation → response to user

    The responsibilities of each stage are:

    • Task parsing: extract the task type, input size, and user preferences;
    • Routing decision: select candidate models based on task level, user selection, quota balance, and model availability;
    • Model gateway: centrally handle authentication, rate limiting, timeouts, and logging, masking differences between model APIs;
    • Result validation: check output format, length, and key fields, triggering retries or escalation when necessary.

    There are four common routing strategies:

    1. Explicit selection: the user specifies the model, and the system handles execution and billing. Suitable for users familiar with the differences between models.
    2. Rule-based routing: map task types or levels to fixed models. Simple to implement and predictable in behavior.
    3. Cascaded invocation: try a lightweight model first and escalate to a stronger model if validation fails. Effective for cost control, but increases end-to-end latency.
    4. Classifier-based routing: use a small model or a rule set to judge task difficulty, then dispatch the task. Highly flexible, but the classification data must be maintained.

    Below is illustrative pseudocode for the routing logic:

    1. function route(task, user):
    2. level = classify(task) # lightweight / standard / complex
    3. if user.selected_model is not null:
    4. candidates = [user.selected_model]
    5. else:
    6. candidates = models_for_level(level)
    7. for model in candidates:
    8. cost = estimate_cost(task, model)
    9. if model.available and user.quota >= cost:
    10. return model
    11. return fallback_model # degrade to a low-multiplier model when quota is insufficient

    Two points deserve attention. First, cost estimation must happen before execution; otherwise, problems are only discovered after a task fails or exceeds its budget. Second, the degradation path must be defined in advance, and users must be informed which model was actually used, so they do not unknowingly receive output of a different quality.

    4. Cost Estimation and Budget Control

    4.1 Pre-Execution Estimation

    Cost estimation requires input length, expected output length, and the expected number of steps as inputs. Expected output length is usually based on the historical average of similar tasks, and the estimate should be presented as a range labeled “estimated consumption” in the UI.

    1. function estimate_cost(task, model):
    2. input_units = count_units(task.input)
    3. output_units = historical_mean_output(task.type)
    4. base = input_units * INPUT_WEIGHT + output_units * OUTPUT_WEIGHT
    5. return base * model.multiplier * time_factor(now(), model)

    Here, INPUT_WEIGHT and OUTPUT_WEIGHT are conversion coefficients determined by resource cost. They should be managed through configuration rather than hardcoded.

    4.2 Quota Caps and Degradation

    Budget control typically involves three layers of rules:

    • Per-task cap: when the estimated consumption exceeds a threshold, require user confirmation before execution;
    • Periodic cap: daily or monthly quota…