Multi-Model Access and Quota Multiplier Billing for Enterprise AI Platforms

    This article explains how to build an enterprise AI platform that connects multiple large language models. It covers a unified AI gateway for model routing, quota multipliers for billing, how to select models by cost and acceptance criteria, and time-limited discounts for low-multiplier models. It includes a structural Python example and a worked billing calculation.

    As large language models mature, enterprise office platforms are moving from a single model to multi-model collaboration. Platforms have typically relied only on the vendor’s own basic and advanced models. However, different models vary significantly in logical reasoning, text generation, response speed, and resource consumption, so a single model rarely delivers good cost-effectiveness across all scenarios. Connecting several mainstream third-party models and establishing explainable compute-based billing has therefore become an important part of platform development. This article follows the implementation sequence and explains how to design the unified gateway, quota multipliers, model selection, time-limited discounts, and launch checks.

    Building a Unified AI Gateway and Model Routing

    Tasks in enterprise office scenarios vary widely. Code review and data analysis require large models with strong reasoning, while email drafting and meeting minute preparation depend more on response speed. Connecting models from several vendors to the same platform covers more task types. However, each additional vendor adds another interface, another authentication scheme, and another pricing rule. The unified AI gateway wraps these differences in one layer, so upper-layer business applications only see a single calling convention.

    A gateway typically performs three functions. First, it hides differences between model interfaces. Second, it routes each request to a model, either by rule or by the user’s choice. Third, it handles authentication, quota checks, and metering before and after each call. Users can select a model directly in the interface, or let the gateway choose one automatically based on the task type.

    The code below is a structural sketch and should not be deployed as is. _invoke_model is pseudocode that raises NotImplementedError. In actual use, replace it with a real HTTP call. The code demonstrates a routing table, timeout control, limited retries, and request IDs. Run it on Python 3.9 or later, and implement the metering service separately.

    1. import time
    2. import uuid
    3. class ModelGateway:
    4. """Pseudocode: demonstrates only the structure of routing and metering. Not for direct use in production."""
    5. ROUTE_TABLE = {
    6. "code_generation": "pro_reasoning",
    7. "deep_analysis": "pro_reasoning",
    8. "email_draft": "lite_fast",
    9. "simple_summary": "lite_fast",
    10. }
    11. DEFAULT_MODEL = "third_party_flash"
    12. MAX_RETRIES = 2
    13. TIMEOUT_SECONDS = 30
    14. def __init__(self, metering, rate_table):
    15. self.metering = metering # Metering service, responsible for quota checks and deductions
    16. self.rate_table = rate_table # Multiplier table, structured as {model name: multiplier coefficient}, with a version number
    17. def route(self, task_type):
    18. return self.ROUTE_TABLE.get(task_type, self.DEFAULT_MODEL)
    19. def invoke(self, task_type, prompt, user_id, request_id=None):
    20. request_id = request_id or str(uuid.uuid4())
    21. model = self.route(task_type)
    22. # The quota check runs before the upstream call. If quota is insufficient, return immediately without incurring upstream cost.
    23. if not self.metering.check_quota(user_id, model):
    24. raise RuntimeError("QUOTA_EXCEEDED")
    25. last_error = None
    26. for attempt in range(self.MAX_RETRIES + 1):
    27. try:
    28. result = self._invoke_model(
    29. model, prompt,
    30. timeout=self.TIMEOUT_SECONDS,
    31. request_id=request_id,
    32. )
    33. # Charge only after a successful response, deduplicated by request_id, so retries never cause double charges.
    34. self.metering.charge(
    35. user_id, request_id, model,
    36. usage=result["usage"],
    37. rate=self.rate_table[model],
    38. )
    39. return result["text"]
    40. except TimeoutError as e:
    41. last_error = e
    42. time.sleep(2 ** attempt) # Exponential backoff
    43. raise RuntimeError(f"UPSTREAM_FAILED: {last_error}")
    44. def _invoke_model(self, model, prompt, timeout, request_id):
    45. # Pseudocode: this is where authentication, the HTTP call, and error code mapping should happen.
    46. # Keys are read from environment variables or a key management service, and are never written into code or logs.
    47. raise NotImplementedError

    Several points in the code need attention. First, keys are read from environment variables or a key management service, and are never written into code or logs. Second, retries apply only to recoverable errors such as timeouts, and the number of attempts is capped to avoid amplifying request volume during an outage. Third, billing is deduplicated by request_id. Fourth, quota checks and metering are implemented separately, and a failed check returns immediately.

    How Quota Multipliers Are Calculated: Billing Rules and Examples

    A quota multiplier is a pricing coefficient. It does not change how complex a task is. It only determines how much quota an account is charged for the same task. The platform calculates the deduction as follows:

    Deducted quota = Base task consumption × Model multiplier coefficient

    Base task consumption must be defined first. A common approach weights input and output tokens separately:

    Base task consumption = Input tokens × Input coefficient + Output tokens × Output coefficient

    Output tokens usually cost more than input tokens, so the output coefficient can be set higher. The platform defines its own coefficients and should document them in its product documentation. It should not simply copy a vendor’s list prices.

    For example, a task uses 2,000 input tokens and 800 output tokens. The input coefficient is 1, and the output coefficient is 4. The base consumption is 2000 × 1 + 800 × 4 = 5200. The table below shows the deduction for three models. The multiplier values are examples only.

    Model Multiplier Deducted quota
    Lightweight fast model 0.5 2600
    Third-party efficient model 1.0 5200
    Complex reasoning model 2.0 10400

    Billing rules must also cover several special cases.

    Cache hits: The input portion that hits the cache can be billed at a lower coefficient, such as 50%. The rules must state whether the cache discount is enabled and at what rate.

    Failures and refunds: If the upstream model returns no valid result, no quota is deducted. If a response is interrupted after partial output has been returned, the platform meters the tokens already returned and marks the log entry as partially completed.

    Multiplier versions: Each multiplier change must have a version number and an effective time. A new multiplier applies only to requests made after its effective time. Historical bills are recalculated using the version in effect when each request occurred, which makes reconciliation possible.

    How to Choose a Model: Multipliers, Scenarios, and Acceptance Metrics

    The table below lists relative multipliers, suitable scenarios, and acceptance metrics for each model type. Multiplier levels show only relative positions. The platform sets exact values based on cost analysis.

    Model type Relative multiplier Suitable scenarios Acceptance metrics
    Lightweight fast model Below baseline Email drafting, simple summaries, intent recognition Sampling pass rate, time to first token
    Third-party efficient model Close to baseline General Q&A, document rewriting Pass rate, average response time
    Complex reasoning model Above baseline Code review, in-depth data analysis Defect detection rate, conclusion verifiability rate

    When choosing a model, first determine whether a lightweight model can complete the task, then compare costs. In practice, sample a batch of historical tasks, run them on both models, and have business staff score the results against the acceptance criteria. Route a task type to the low-multiplier model by default only after the lightweight model’s pass rate reaches a level the business accepts. The business side sets the pass-rate threshold. The platform provides the data and handles the routing.

    Time-Limited Discounts and Checks Before Launching Low-Multiplier Models

    The platform can offer time-limited discounts on designated models during off-peak hours to encourage batch and non-urgent