Engineering a Multi-Model AI Platform: Unified Adapters, Task-Based Model Selection, Quota Multipliers, and Time-Limited Promotion Scheduling

    This article describes the engineering design of a multi-model AI platform, covering a unified adapter layer, task-based model routing, quota multiplier settlement, time-windowed promotion scheduling, enterprise permissions and security, observability, and implementation recommendations.

    1. Background: A Single Model Rarely Covers Every Workload

    Office AI platforms support many task types, including long-document summarization, spreadsheet data cleanup, code snippet generation, multi-turn Q&A, and batch copy rewriting. Different models vary in reasoning depth, response speed, and invocation cost. If a platform binds itself to only one model, simple tasks may waste compute and quota, while complex tasks may fall short on quality.

    As a result, multi-model access is a common design for many platforms. The engineering core of this capability has three parts: unified invocation adaptation, clear selection criteria, and quota metering that can be accounted for and explained. The following sections discuss each in turn.

    2. Designing a Unified Adapter Layer

    Different model services do not share the same interface parameters, authentication methods, or response structures. If each model’s invocation logic is written separately in business code, the cost of adding or replacing models later rises quickly. A recommended approach is to add an adapter layer between the business logic and the models, so that the business side only works with a unified request and response structure.

    Key design points include:

    • A unified request structure: message list, generation parameters, timeout, and task identifier.
    • One adapter per model, responsible for protocol conversion and error code mapping.
    • A registry that stores model metadata, such as capability tags, context length limits, quota multipliers, and availability status.

    Example code:

    1. from dataclasses import dataclass
    2. from typing import Protocol
    3. @dataclass
    4. class ChatRequest:
    5. messages: list[dict]
    6. max_tokens: int = 1024
    7. timeout_s: float = 30.0
    8. class ModelAdapter(Protocol):
    9. name: str
    10. def chat(self, req: ChatRequest) -> str: ...
    11. class ModelRegistry:
    12. def __init__(self):
    13. self._models: dict[str, ModelAdapter] = {}
    14. def register(self, adapter: ModelAdapter) -> None:
    15. if adapter.name in self._models:
    16. raise ValueError(f"Model {adapter.name} is already registered")
    17. self._models[adapter.name] = adapter
    18. def get(self, name: str) -> ModelAdapter:
    19. if name not in self._models:
    20. raise KeyError(f"Unknown model: {name}")
    21. return self._models[name]

    With this approach, adding a model only requires implementing an adapter and registering it. Upper-layer task scheduling, metering, and promotion logic do not need to change.

    3. Selecting Models by Task Type

    Model selection should not rely on a single metric. An evaluation table can be built from the following four dimensions:

    Dimension Description Typical scenarios
    Reasoning capability Ability to handle complex logic and multi-step reasoning Code generation, data analysis
    Response speed Time to first token and overall latency Interactive Q&A
    Context length Input size that can be processed in a single request Long-document understanding
    Usage cost Quota consumed per task Batch tasks, high-frequency calls

    Routing rules can start with a simple mapping and then be optimized gradually based on production data. Example:

    1. # Task type to model tier mapping; the actual configuration should be loaded externally
    2. ROUTE_RULES = {
    3. "long_document_summary": "high_capacity",
    4. "code_generation": "reasoning",
    5. "quick_qa": "lightweight",
    6. "batch_rewrite": "lightweight",
    7. }
    8. # Tier to concrete model mapping, managed by the configuration center
    9. TIER_TO_MODEL = {
    10. "high_capacity": "model-long-context",
    11. "reasoning": "model-reasoning",
    12. "lightweight": "model-fast",
    13. "general": "model-general",
    14. }
    15. def select_model(task_type: str, user_choice: str | None = None) -> str:
    16. # Respect an explicit user choice first
    17. if user_choice:
    18. return user_choice
    19. tier = ROUTE_RULES.get(task_type, "general")
    20. return TIER_TO_MODEL[tier]

    Explicit user selection should take precedence over system routing. However, the interface should also display the model’s quota multiplier so that users can make decisions based on clear information.

    4. Quota Multiplier Settlement

    A quota multiplier is a way to normalize resource consumption across different models. The lower the multiplier, the less quota the same task consumes. The formula is:

    Actual quota consumption = base quota × model multiplier

    The base quota is determined by task scale, for example, converted from input and output character counts or token counts. The metering unit should be determined after the request finishes based on actual usage, not estimated before submission. If the estimate deviates too much from actual usage, users will question the billing result.

    1. import math
    2. def settle_quota(base_units: float, multiplier: float) -> int:
    3. """Settle quota by multiplier, rounding up to avoid undercounting."""
    4. if multiplier <= 0:
    5. raise ValueError("Multiplier must be positive")
    6. return math.ceil(base_units * multiplier)

    Metering design should consider the following three points:

    • The multiplier should be visible in the interface, with its effective time clearly indicated.
    • Deductions for the same request must be idempotent; retries should not charge twice.
    • Failed requests should be refunded or not charged according to defined rules, and the failure reason should be recorded.

    5. Scheduling Time-Limited Promotions

    A common platform practice is to lower the multipliers of some models during specific time periods, such as designated weekday hours or holidays. In implementation, promotions should be abstracted as rules with time windows and driven by configuration, rather than hard-coded in business logic.

    1. promotions:
    2. - name: Weekday lunch promotion
    3. timezone: Asia/Shanghai
    4. days: [mon, tue, wed, thu, fri]
    5. window: "12:00-14:00"
    6. models:
    7. model-fast: 0.5
    8. - name: Holiday promotion
    9. timezone: Asia/Shanghai
    10. dates: ["2026-01-01..2026-01-03"]
    11. window: "00:00-23:59"
    12. models:
    13. model-fast: 0.6

    Scheduling requires attention to the following details:

    1. Time zone consistency: use the server-agreed business time zone to determine time periods, avoiding mismatches between user local time and server time.
    2. Priority: when multiple rules match at the same time, define the value strategy clearly, such as taking the lowest multiplier or using rule priority.
    3. Boundary handling: document clearly whether the start and end points of a time window are inclusive.
    4. Traceability: each settlement record should store the matched rule ID to support review of disputed orders.

    Whether promotions can stack with other promotions is a product rule and should be decided and published before launch.

    6. Permissions and Security in Enterprise Scenarios

    When enterprise users access external models, key management and data boundaries usually require extra care.

    • Do not write secrets into code or logs. Store them in a key management service or environment variables, and rotate them regularly.
    • Partition invocation permissions by organization, project, or user, following the principle of least privilege.
    • Apply desensitization and compliance checks to input and output, and define which data is allowed to be sent to external models.
    • Record audit logs, including the caller, model, task type, time, and settled quota. Logs should not contain plaintext sensitive content.

    7. Observability and Cost Governance

    After a multi-model system goes live, the following metrics should be monitored continuously:

    • Invocation success rate, timeout rate, and average latency for each model.
    • Average quota consumption for each task type, as well as user repeat usage.
    • Changes in invocation volume and total quota during time-limited promotions, to determine whether promotions bring the expected usage growth.
    • Abnormal consumption alerts, such as a single user’s quota spiking within a short period.

    It is recommended to aggregate metrics by both model and task type so that the source of problems can be located quickly.

    8. Implementation Recommendations

    1. Build the unified adapter layer and model registry first, then connect concrete models one by one.
    2. Start routing rules with simple mappings and iterate based on production data.
    3. Make multipliers and promotion rules configurable so that adjustments do not require a new release.
    4. Settle quota based on actual usage and ensure that deductions are idempotent.
    5. Write contract tests for each adapter, covering timeouts, rate limiting, and error code mapping.