Multi-Model Integration and Credit-Based Pricing: Designing an Office Assistant Router

    When an office assistant evolves from relying on a single in-house model to freely switching among multiple third-party models, the product complexity comes not from the number of models but from three engineering problems it introduces: inconsistent capability abstraction, incomparable unit costs, and user choices made without a clear basis. This article lays out a practical design across four layers—model abstraction, pricing, routing, and governance. It shows how a model gateway normalizes heterogeneous capabilities behind a single contract, how a credit-consumption multiplier restores cost comparability, how time-based discounts turn static pricing into a time-dependent function, and how a three-layer routing strategy together with enterprise controls keeps the system reliable as the model list grows from a handful to dozens.

    When an office assistant evolves from relying on a single in-house model to freely switching among multiple third-party models, the product complexity grows not primarily from the number of models but from three engineering problems that arise as a result: inconsistent capability abstraction, incomparable unit costs, and user choices made without a clear basis. These problems can be tolerated implicitly when only one or two models are available, but once the option list expands, they must be solved explicitly. The discussion below is organized across four layers: architecture, pricing, routing, and governance.

    1. The Model Abstraction Layer: Collapsing Heterogeneous Capabilities into a Single Contract

    The first step of multi-model integration is not connecting models—it is defining a contract. If each model’s calling convention, context length, and structured-output capability are exposed directly to upper-layer business logic, the business code becomes tightly coupled to specific models, and switching a model means rewriting the business. A sound approach is to insert a model gateway between the business and the models, with the gateway responsible for protocol adaptation, parameter normalization, and retry and degradation.

    A minimal abstraction contract typically contains the following fields:

    1. @dataclass(frozen=True)
    2. class ModelProfile:
    3. name: str # Model identifier, used internally by the gateway only
    4. max_context: int # Upper bound of the context window
    5. supports_json_schema: bool # Whether constrained structured output is supported
    6. supports_vision: bool # Whether multimodal input is supported
    7. latency_band: str # Speed band: fast / balanced / deep
    8. credit_multiplier: float # Credit-consumption multiplier

    The key principle is this: credit_multiplier and capability descriptions must live in the same registry as the calling logic, rather than being scattered across a config center, a frontend constants file, and a billing system. Scattering them inevitably produces inconsistent definitions when multipliers are adjusted—users see numbers that do not match what is actually deducted.

    2. Credit-Consumption Multiplier: Solving Comparability with a Single Coefficient

    Different models consume different amounts of compute to complete the same task. A model built for long-chain reasoning and a model optimized for low latency can differ severalfold in resource cost on the same workload. Yet in the interface, users see only a single unit: credits. Without an adjustment mechanism, credits become a distorted currency—task complexity stays the same, but how fast available credits are consumed is decided entirely by the model, and users have no way to judge value.

    The credit-consumption multiplier is a pricing coefficient designed to remove this distortion. It does not change task complexity; it only determines the rate at which account credits are consumed:

    1. Credits consumed = Base token-converted credits × Task multiplier × Time-window discount
    2. Base token-converted credits = Input tokens × k_in + Output tokens × k_out

    Three points deserve care when designing this mechanism:

    1. The multiplier must have a public and stable definition. If the base conversion charges only on output tokens, it will severely underestimate cost on input-heavy tasks like long-document summarization; conversely, it distorts the cost of short Q&A. Separating input and output conversion better reflects the real resource profile.
    2. The multiplier should be tied to cost, not to a “premium feel.” The basis for adjusting a multiplier should be changes in underlying inference cost, not steering users toward a particular model. The former is explainable; the latter undermines the credibility of the pricing system.
    3. Every multiplier change must be traceable. Each price adjustment should retain its effective time, scope of impact, and before/after values, for billing reconciliation and complaint review.

    3. Time-Based Discounts: Turning Pricing from a Static Table into a Function of Time

    The essence of a limited-time promotion is extending a static multiplier table into a pricing function with a time dimension. There are two implementation paths, and their engineering implications differ substantially:

    • Path 1: Time-window discounts. Apply an overall discount to the participating models within a specified window. This is relatively simple to implement, but it requires introducing the dimension of “task occurrence time” into the billing pipeline and ensuring consistent time zones and holiday calendars; otherwise, long tasks that cross midnight produce disputes over which discount applies. For streaming tasks, whether attribution is based on task start time or settlement time must be stated explicitly in the product documentation.
    • Path 2: Targeted multiplier reduction. Directly lower the multiplier of specific models. This path is better suited to high-frequency, lightweight scenarios—summarization and rewriting, formatting, and information extraction, where call frequency is extremely high. A 10% multiplier reduction is amplified by the number of calls into a meaningful absolute saving that users clearly perceive, while the change to the platform’s cost structure remains controllable.

    It is worth emphasizing that the credibility of any discount mechanism depends on metering accuracy. If credit deductions cannot be matched call by call against usage details, any promotional design degenerates into a source of complaints. The billing pipeline should ensure that every call generates a unique billing record capturing the task ID, model ID, input and output tokens, effective multiplier, discount factor, and final credit deduction, and that these are asynchronously aggregated to a settlement service via a message queue, so that metering and inference requests do not block each other on the same synchronous path.

    4. Routing Strategy: Moving the Basis for Choice into the Interface and the Code

    Once multiple models are available, the real product value lies not in “having options” but in “choosing well.” Automatic routing typically unfolds in three layers of decision-making:

    Layer 1: Capability hard-constraint filtering. Based on task features, first eliminate non-qualifying models. Tasks requiring structured output filter out models that do not support constrained decoding; tasks containing images filter out non-multimodal models; ultra-long documents filter out models with insufficient context windows.

    Layer 2: Cost-and-quality tradeoff scoring. For the candidate set, compute a cost function that weights the multiplier, historical success rate, and speed band:

    1. def score(profile: ModelProfile, task: Task, stats: Stats) -> float:
    2. cost = profile.credit_multiplier * task.est_tokens
    3. fail = 1.0 - stats.success_rate(profile.name, task.type)
    4. latency = stats.p95_latency(profile.name, task.type)
    5. return w_cost * cost + w_fail * fail + w_lat * latency

    Layer 3: Degradation and fallback. When the preferred model returns an error, times out, or hits rate limits, switch to the next model in the candidate list in a preset order, and record the degradation event in monitoring alerts. Degradation must be silent and usable; otherwise, a fluctuation in a third-party service is exposed directly to users as a fault.

    At the same time, showing the multiplier directly on the model-selection page is effective for users—it changes the basis for choice from “guessing by feel” to “a visible tradeoff between multiplier and task type.” For most office users, however, a more reasonable product form is to offer three presets—automatic / economical / high-quality—letting the system decide on the user’s behalf using the scoring logic above, while displaying the currently effective multiplier and estimated consumption as reference information.

    5. Enterprise-Side Governance and a Pre-Launch Checklist

    For team and enterprise subscription scenarios, multi-model integration brings additional quota and compliance demands: not wanting employees to send sensitive documents to uncontrolled models, not wanting a single project to drain the shared quota, and not wanting call records to be unauditable. The responses include setting quota caps by department or project, configuring allowed-model whitelists by task type, centrally archiving model-call logs, and setting anomaly detection for credit-consumption rate in monitoring alerts.

    Before launch, it is advisable to check the following:

    • Whether the capability descriptions, context limits, and multipliers in the model registry match the gateway’s actual behavior;
    • Whether billing records can be reconciled call by call, and whether multipliers and discount factors are fixed and retained with each record;
    • Whether the degradation path has been validated through fault injection, and whether billing is correct after degradation;
    • Whether the attribution rule for time-window discounts on cross-midnight long tasks has been defined and documented;
    • Whether enterprise customers have any path that forces a model switch; if so, a policy-configuration capability must be provided.

    Building these capabilities typically requires the coordinated operation of foundational components such as a model gateway, a metering service, a logging service, monitoring and alerting, and a message queue—all of which can be implemented through standard cloud-product combinations on platforms such as Baidu AI Cloud, without building the entire infrastructure from scratch. The engineering focus of multi-model integration has never been the number of models connected, but whether the abstraction layer is clean, whether pricing is trustworthy, and whether routing is explainable. Get these three right, and the system stays controllable whether the option list grows from two models to twenty.