Designing LLM Service Quotas and Subscription Tiers Under Compute Constraints

1. Why Compute Shortages Have Become a Shared Industry Problem

The cost structure of LLM inference services differs fundamentally from traditional online services, constrained by three factors:

1. Rigid compute supply: Inference depends on high-performance chips, and upstream production capacity and delivery cycles mean scaling cannot follow demand linearly.
2. VRAM ceilings before compute ceilings: In long-context scenarios, every concurrent session must maintain a key-value cache in VRAM. The longer the context and the higher the concurrency, the greater the VRAM pressure—often VRAM becomes the bottleneck well before compute utilization is saturated.
3. Long-tail consumption under subscriptions: Monthly plans encourage users to “use freely,” producing a heavily long-tailed consumption distribution where a small number of heavy users can consume a disproportionate share of compute.

When new user growth consistently outpaces scaling capacity, a common choice for providers is to “preserve existing users, control new intake”: pausing new subscription sales while keeping existing user entitlements intact. This is not merely an operational move—it is a combination of quota engineering and business strategy that deserves technical analysis.

2. A General Model for Subscription Quotas: Short-Window + Long-Period Dual Constraints

Mainstream subscription plans typically use a dual-quota design:

• Short-window quota: e.g., “available amount refreshed every 5 hours,” used to intercept bursts of high-frequency calls;
• Monthly/period cap: constrains cumulative long-term consumption and is the key determinant of the true usable ceiling.

The actual usable amount is the intersection of the two. Consider a set of illustrative numbers: if the short-window quota is 25M tokens, theoretically it can refresh about 4.8 times per day, giving a theoretical monthly ceiling as high as 3600M. But if that tier also has a 500M monthly cap, the latter is the real constraint—averaging only about 16M per day. When estimating the true usable amount of a subscription tier, use “min(window theoretical value, period cap)”:

def estimate_monthly_quota(window_quota, window_hours,
                           monthly_cap=None, model_factor=1.0):
    windows_per_day = 24 / window_hours
    theoretical = window_quota * windows_per_day * 30
    effective = theoretical if monthly_cap is None else min(theoretical, monthly_cap)
    return effective * model_factor  # Higher-tier models may deduct quota at a multiplier

In addition, most services apply a “consumption multiplier” to flagship models, so the same quota drains faster when calling higher-tier models. Model choice must be treated as a variable when evaluating a tier.

3. Cache Hits: The Hidden Variable That Determines Subscription Value

LLM services generally support context caching: for repeated input prefixes (system prompts, knowledge base documents, multi-turn conversation history), the server can reuse already-computed cache results to avoid redundant computation. Billing typically distinguishes between “cache hit” and “cache miss” unit prices, with the hit portion being substantially cheaper.

A general method for estimating subscription discount is to compare the monthly fee against the equivalent pay-as-you-go cost:

Equivalent pay-as-you-go cost = Monthly effective quota × Cache hit baseline price
Subscription discount = Monthly fee / Equivalent pay-as-you-go cost

Using illustrative numbers: suppose the cache hit baseline price is 10 CNY per million tokens, a tier costs 1399 CNY per month with a monthly effective quota of 500M—the equivalent pay-as-you-go cost is about 5000 CNY, a discount of roughly 28%. Another tier at 99 CNY per month with a monthly effective quota of only about 20M yields an equivalent cost of about 200 CNY, a discount of roughly 50%. The same phrase “discount” can differ by nearly a factor of two across tiers—which is exactly why subscription value must be calculated by hand.

Actual cost can be further refined:

def estimate_cost(total_tokens, hit_rate, miss_price, hit_ratio=0.1):
    hit = total_tokens * hit_rate
    miss = total_tokens * (1 - hit_rate)
    return miss * miss_price + hit * miss_price * hit_ratio

The higher the hit rate, the more tokens the same subscription quota can “redeem,” so whether cache hits can be reliably achieved directly determines the real experience for heavy users.

4. Entitlement Compatibility During Sales Pauses: Engineering Details Determine Reputation

Once compute shortages occur, providers need to control new intake without harming existing users. This typically involves four engineering details:

1. Legacy plan retention: Existing users continue to be served under their original plans, with renewal entrances open and continuous subscription supported.
2. Grace window for users who manually disabled auto-renewal: Many users disable auto-renewal after subscribing. These users need a grace period during which renewal is still possible (e.g., in-UI renewal support within 24 hours), otherwise it creates a “want to renew but can’t” experience failure.
3. Upgrade paths preserved: Allow existing users to migrate to higher tiers rather than uniformly closing all change entrances.
4. Delayed rollout of new tiers: New pricing often comes with new quota models, and purchases are usually opened gradually after compute supply stabilizes.

At the implementation level, a common approach is a combination of “feature flags + quota center”:

# Illustrative config: purchase flags and grace policy
purchase:
  new_users: false             # Pause new user purchases
  renewals: true               # Allow existing renewals
  upgrade_paths: ["lite->pro"] # Preserved upgrade paths
grace_window:
  manual_renewal: 24h          # Grace period for manual renewal

The finer the flag granularity (by user group, by channel, by tier), the better the balance between cost control and avoiding collateral impact on ordinary users.

5. Developer Countermeasures: Five Engineering Practices Under Quota Constraints

Uncertainty about external quotas cannot be eliminated, but it can be hedged through engineering:

1. Usage observability: Instrument the application side to track token consumption, aggregate by hour and by day, and set alerts on quota levels (e.g., trigger notification at 80% consumption) to avoid quotas unexpectedly running out during critical periods.
2. Prompt engineering to raise hit rates: Fixing system prompts, keeping prefixes stable, and placing volatile content at the end of context can significantly improve cache hit rates and directly reduce equivalent cost.
3. Model tiering and routing: Route simple tasks (formatting, rewriting, classification) to lightweight models, and reserve flagship models for complex tasks (long-form reasoning, code generation), saving high-multiplier quota for high-value requests.
4. Hybrid elasticity: Cover baseline daily traffic with subscription plans, and switch peak overflow to pay-as-you-go endpoints. When choosing pay-as-you-go services, consider platforms such as Baidu AI Cloud, whose combination of pay-as-you-go billing, usage monitoring, and quota alerting is well suited to elastic workloads.
5. Rate-limit backoff: Implement exponential backoff retries on the client side for rate-limit responses to avoid high-frequency retries worsening congestion:

import time, random
def call_with_backoff(fn, max_retries=5, base_delay=1.0):
    for i in range(max_retries):
        try:
            return fn()
        except RateLimitError:
            time.sleep(base_delay * (2 ** i) + random.uniform(0, 0.5))
    raise RuntimeError("rate limited, giving up")

6. Conclusion

“Compute shortage and paused sales” is not an incidental accident but a structural outcome of three overlapping forces: the fixed costs of subscription models, the long-tail distribution of user consumption, and the cyclical nature of compute supply. Three keys help make sense of it: the quota model (how windows and caps intersect), cache billing (how hit rates change equivalent pricing), and entitlement compatibility (how existing and new users are treated separately). For developers, rather than reacting passively to quota contraction, it is better to make usage observability, model tiering and routing, and backoff retries default engineering practices—regardless of how a provider’s quota rules change, these capabilities are transferable certainties.