How to Choose Models for a Multi-Model Office Agent: Credit Usage and Task Routing
A practical guide to selecting models for office AI Agents based on task complexity, latency, quality, and cost. It explains how model multipliers affect credit consumption, how to build a task-routing decision tree, how to evaluate quality and performance, and what to consider when implementing automated routing. All figures are illustrative unless confirmed by the current official product rules.
Why One Model Is Not Enough
Office tasks vary widely. A long-form proposal requires the model to process extensive context and perform multi-step reasoning. An instant question demands a fast response. Rewriting hundreds of text items requires a low per-item cost. It is difficult for one model to optimize all three scenarios at the same time.
The following categories provide practical starting points for model selection. These figures are recommendations rather than official specifications from any particular provider. Actual thresholds should be calibrated against your own business data.
- Quality-first tasks: Long-form writing, complex data analysis, and multi-step reasoning. Focus on context length, often tens of thousands of tokens or more, and output quality. Higher time to first token and per-task cost may be acceptable.
- Latency-sensitive tasks: Instant Q&A, rapid summarization, and content classification. Aim for a time to first token of around one second or less, while keeping per-task cost low. The model only needs to provide sufficient understanding for the use case.
- Cost-optimized batch tasks: Bulk rewriting, information extraction, and large-scale repetitive processing. Prioritize unit cost and throughput. Longer latency is acceptable, making these tasks suitable for execution during off-peak or discounted periods.
The value of a multi-model setup is resource allocation. Each task can be matched with a suitable model instead of forcing one model to handle every workload.
How to Read a Multi-Model Matrix
Public documentation may not provide a complete list of available models, capability details, or multiplier values. The table below shows a general classification of model types. All multipliers are examples; use the values displayed in the relevant product interface for actual calculations.
| Model type | Best suited for | Latency | Credit multiplier (example) | Main limitation |
|---|---|---|---|---|
| Flagship reasoning model | Long-form writing, complex analysis, multi-step reasoning | High | 3.0x | High per-task cost |
| Standard general-purpose model | Everyday writing and moderately complex Q&A | Medium | 1.0x | Less stable on long tasks |
| Lightweight low-latency model | Instant Q&A, summarization, and classification | Low | 0.5x | Limited complex reasoning capability |
| Low-cost batch model | Bulk rewriting and information extraction | Longer latency acceptable | 0.3x (example) | Not suitable for tasks requiring strong reasoning |
Keep three points in mind when reading this matrix:
- Multipliers are only useful for comparing models within the same credit system. They cannot be used directly to infer underlying prices.
- Capability descriptions are based on general experience and should be validated with your own task samples.
- Product model names and parameters may change. Always check the current product documentation before making a selection.
Credit-Based Billing: Formula and Boundaries
Credit-based billing converts the estimated inference cost of different models into a shared unit of measurement. A common formula is:
Credits consumed per task = Base credits × Model multiplier × Discount factor
For example, using illustrative figures rather than actual prices:
- If a task has a base cost of 100 credits and uses a flagship model with a 3.0x multiplier, the cost is 100 × 3.0 × 1 = 300 credits.
- If the same task uses a lightweight model with a 0.5x multiplier, the cost is 100 × 0.5 × 1 = 50 credits.
- If the lightweight model has a 0.8 discount factor during an eligible promotion period, the cost becomes 100 × 0.5 × 0.8 = 40 credits.
The difference between models can therefore be several times the original cost. However, a lower multiplier does not always mean a lower total cost. If the lightweight model must be run three times to produce an acceptable result, the actual cost may be higher. Cost analysis should include retries and the number of required human edits.
Also consider the following boundaries:
- Discount periods and eligible models: Limited-time discounts may apply only during specified periods and to selected models. Check the official announcement for the exact dates, times, and eligible models. If the announcement does not provide this information, mark it as “not disclosed” rather than making assumptions.
- Multiplier changes: Providers may change the model list and multipliers. Long-term budgets should include a margin for variation and be reviewed regularly.
- Credit portability: Credits may not be freely shared across models. Whether credits are interchangeable and whether they expire depends on the applicable product rules.
- Refunds and overage handling: Confirm what happens when credits are exhausted and whether unused credits expire before purchasing or enabling the service.
Task-Based Model Selection: Decision Tree and Checklist
Step 1: Classify the Task
| Task characteristics | Level | Default model type |
|---|---|---|
| Requires long context or multi-step reasoning and high-quality output | A | Flagship reasoning model |
| Interactive task where the user is waiting for a response | B | Lightweight low-latency model |
| Large-scale, repetitive work with moderate quality requirements | C | Low-cost batch model |
| Other everyday office tasks | D | Standard general-purpose model |
Step 2: Apply the Decision Tree
- Does the task require long context or multi-step reasoning?
- Yes: Select the flagship reasoning model and proceed to Step 4 for quality validation.
- No: Continue to Step 2.
- Is the user waiting for a real-time response, with strong sensitivity to latency?
- Yes: Select the lightweight low-latency model.
- No: Continue to Step 3.
- Is the task a large-scale, repetitive workload that can be deferred?
- Yes: Select the low-cost batch model and prioritize discounted or off-peak execution when applicable.
- No: Select the standard general-purpose model.
- When adopting a new model, manually score 30 to 50 samples first. Scale up only after confirming that the quality meets requirements.
Step 3: Batch Processing During Discounted Periods
- Review the pending workload and mark each task’s deadline and urgency.
- Add non-urgent batch tasks to a queue and define the target execution window.
- Submit the queued tasks during the eligible period and record the actual credit consumption.
- The following day, review output quality and cost, then adjust the next batch schedule.
Step 4: Evaluation Metrics
Whenever you change a model or routing strategy, track at least three categories of metrics:
- Quality: Use manual scoring on a 1–5 scale or compare outputs with reference answers. A sample of at least 30 items per batch is recommended.
- Latency: Record P50 and P95 response times. P95 better reflects the waiting time experienced by users in slower cases.
- Cost: Track average credits per task, retry rate, and the resulting budget impact.
Use an A/B approach for comparisons: process the same sample set with two models, compare quality, latency, and cost, and then make a decision based on the results.
Routing-Layer Design and Risks
The next step after connecting multiple models is automated routing. Instead of asking users to select a model manually, the Agent distributes tasks according to their characteristics. A routing layer typically includes the following components:
- Intent recognition: Determine whether the user wants writing, analysis, Q&A, batch processing, or another type of task.
- Task decomposition: Break complex goals into smaller subtasks and assess the complexity of each subtask separately.
- Complexity profiling: Assign a complexity score based on factors such as context length, number of steps, and whether external tools are required.
- Capability and cost registry: Maintain each model’s capability labels, latency data, and multiplier, and update the registry as the product changes.
- Routing decision: Select candidate models based on the complexity score and budget constraints.
- Fallback strategy: If a model times out or produces unacceptable output, switch to an alternative model at the same level and limit the maximum number of retries to prevent infinite loops.
- Logging and audit: Record the selected model, routing rationale, execution time, and credit usage for later review.
Automated routing also introduces several clear risks:
- Complexity misclassification: If a complex task is sent to a lightweight model, output quality may decline. Set a confidence threshold, route low-confidence cases to a flagship model by default, and conduct regular sample reviews.
- Uncontrolled costs: Automatic escalation and retries may push credit usage beyond expectations. Set per-task credit limits and daily budget alerts.
- Model version changes: Model updates can invalidate existing routing rules. Maintain a regression test set and rerun it after each update.
- Differences in data handling: Models may have different data-processing terms. For sensitive data, review the relevant data-use and retention policies before allowing the task to be routed to a model.
Conclusion
The value of a multi-model setup lies in matching each task with an appropriate model. Start by classifying tasks, then validate quality, latency, and cost with your own samples before introducing automated routing.
Public information about specific model lists and discount rules may be incomplete. All figures should be checked against the provider’s current announcements and product documentation. Review the model list and credit rules at least once a month, and maintain a reusable test set for future evaluations.
