Technical Analysis of Compute Quotas and Subscription Models in LLM Services

1. Background: When Compute Supply Cannot Keep Up with Demand Growth

Recently, an LLM service platform announced it was pausing new subscription plan purchases due to compute resource constraints, while preserving continuous subscription rights for existing users. The decision sparked widespread discussion in the developer community, with the core debate centering on two issues: the continuation and upgrade paths for legacy plans, and the reasonableness of quota pricing for new plans.

From a technical operations perspective, this is not an isolated event. As LLM inference demand surges, the elastic scaling capability of compute supply faces serious challenges. GPU cluster expansion cycles, inference framework optimization efficiency, and peak load scheduling strategies together determine the service capacity a platform can credibly promise. When actual demand growth outpaces infrastructure expansion, limiting new user intake while prioritizing existing user experience is a pragmatic capacity management strategy.

This article analyzes the situation from three dimensions—billing models, caching mechanisms, and quota design—to help developers understand the technical logic behind such phenomena and provide practical recommendations.

2. Billing Model Breakdown: The Conversion Between Token Quotas and API Pricing

2.1 Fundamental Differences Between Subscription and Pay-as-You-Go

LLM services typically offer two payment models:

• Subscription: Users pay a fixed monthly fee and receive a certain token quota (e.g., X million tokens per month). Overage may be rate-limited or require additional purchase.
• Pay-as-you-go (API mode): Billing is based on actual token consumption, typically distinguishing input/output and cache hit/miss, with unit prices changing in tiers based on usage volume.

The essence of subscription is risk hedging: users lock in a usage ceiling at a fixed cost, while the platform exchanges a discounted rate for the user’s long-term commitment. But there is a key variable here—the baseline for calculating the discount rate.

2.2 Deriving the Discount Rate

Assume the average cache-hit price in a given API mode is 10 yuan per million tokens, a subscription plan costs P yuan per month, and provides Q million tokens of quota. The equivalent discount rate is:

Discount rate = (P / Q) / 10

Using the data mentioned in the original discussion:

• Plan A: 1399 yuan/month, total quota of approximately 500 million tokens (estimated at 25 million tokens per 5 hours), equivalent unit price of approximately 2.8 yuan per million tokens, discount rate of approximately 28%.
• Plan B: 99 yuan/month, total quota of approximately 144 million tokens (estimated at 1 million tokens per 5 hours), equivalent unit price of approximately 0.69 yuan per million tokens, discount rate of approximately 6.9%.

This exposes a key problem: the discount rates vary dramatically across plans at different price points. Higher-priced plans offer far greater discounts than lower-priced ones, which may leave users of lower-priced plans feeling they are getting poor value. From the platform’s perspective, this may be intended to guide users toward higher-value plans, but if the low-priced plan’s quota is too low, it will indeed hurt user experience.

2.3 Design Considerations for the 5-Hour Rolling Window

The original discussion mentions the concept of a “5-hour quota,” which typically means the platform uses a rolling time window for rate limiting rather than a simple monthly total cap. The advantages of this design are:

• Smooth load: Prevents users from consuming their entire quota at the beginning of the month, leaving nothing for the end.
• Abuse prevention: Limits burst requests in a short period, protecting the stability of the backend inference cluster.
• Improved resource utilization: Through time-window scheduling, compute resources are distributed more evenly across users.

But the drawbacks are also obvious: users cannot intuitively perceive their remaining quota, and for burst batch inference tasks (such as batch code generation or document summarization), a 5-hour window may not be flexible enough.

3. Cache Hit Rate: The Key Variable Affecting Actual Cost

3.1 What Is a Cache Hit

In LLM inference, caching mechanisms typically refer to KV Cache reuse or prefix caching. When multiple requests share the same prefix (such as a system prompt or the same portion of a long document), the inference engine can reuse the already-computed Key-Value states, avoiding redundant computation and reducing latency and compute consumption.

Cache hit rate is defined as:

Hit rate = Tokens hitting cache / Total request tokens

3.2 Impact of Caching on Cost

Assume the unit price for cache misses is 10 yuan per million tokens and the unit price for cache hits is 2 yuan per million tokens (illustrative only). The actual cost is:

Actual cost = Total tokens × (Hit rate × 2 + (1 - Hit rate) × 10)

If the hit rate is 50%, the actual unit price is 6 yuan per million tokens; if the hit rate is 80%, the actual unit price is 3.6 yuan per million tokens. This means the higher the cache hit rate, the more pronounced the relative discount advantage of a subscription plan.

For developers, common approaches to optimizing cache hit rate include:

• Reuse system prompts: Place fixed instructions and role definitions at the request prefix, avoiding modifications each time.
• Structured input: Break long documents into reusable segments to reduce redundant transmission.
• Batch request merging: Combine multiple similar requests into a single batch to share prefix caching.

3.3 Caching Strategies in Subscription Plans

Note that different platforms may have different caching billing rules for subscription plans. Some platforms do not distinguish between cache hits and misses under subscription billing, deducting quota uniformly by token count; others may offer additional discounts for cache hits. When choosing a plan, developers should carefully read the billing documentation and estimate actual costs based on their own cache hit rates.

4. Technical Implementation of Legacy User Rights and Upgrade Paths

4.1 Continuation Mechanism for Existing Plans

The original discussion mentions that “legacy plans were not removed, current users can continue using them normally, and continuous subscription is supported.” From a technical implementation perspective, this involves plan version management. The platform needs to:

• Maintain billing logic for multiple plan versions: Legacy plans have different unit prices, quotas, and rate-limiting strategies from new plans, and the billing system must support coexistence.
• Guarantee quota supply for legacy users: During compute shortages, prioritize existing users’ SLAs rather than directing all resources to new users.
• Provide smooth upgrade paths: Allow legacy users to switch to new plans without service interruption.

4.2 Closing and Recovering Continuous Subscription

The original discussion mentions that “many users choose to manually close continuous subscription after subscribing, and these users can also renew, with a 24-hour window on the interface.” This involves the design of a subscription state machine:

• Manually closing continuous subscription: The user opts out of auto-renewal, but can still use the service normally within the current billing cycle.
• Grace period: Within 24 hours after closing, the user can still manually renew and restore continuous subscription status.
• Expiration handling: After the grace period, the plan may be downgraded to pay-as-you-go or suspended.

This design must balance user experience and resource reservation: too short a grace period and users may miss renewal; too long a grace period and the platform must reserve resources for users who may not renew.

4.3 Atomicity of Plan Upgrades

“Legacy users can choose to upgrade their legacy plan, with a 24-hour window on the interface”—the upgrade operation needs to guarantee atomicity:

• Quota carryover: After upgrading, how is the remaining quota from the legacy plan handled? Is it stacked, converted, or zeroed out?
• Billing cycle reset: Does the upgrade take effect immediately? How are the start and end times of the new cycle calculated?
• Rollback mechanism: If the user is unsatisfied after upgrading, is rollback to the legacy plan supported?

These details directly affect user experience, and platforms need to clearly explain them in the interface and documentation.

5. Engineering Best Practices for Capacity Management

5.1 Recommendations for Developers

1. Evaluate your own usage patterns: Track historical token consumption, cache hit rate, and peak concurrency to determine whether a subscription plan is a good fit.
2. Prioritize cache hit rate optimization: Reduce actual compute consumption by reusing prompts and structuring inputs.
3. Understand the plan’s rate-limiting strategy: Learn the specific rules of the 5-hour rolling window to avoid triggering rate limits during critical tasks.
4. Keep a pay-as-you-go channel: Ensure a fallback option (such as API pay-as-you-go) to maintain business continuity when subscription quota is exhausted.

5.2 Implications for Platform Operations

1. Transparent billing rules: Clearly display quota calculation methods, caching strategies, and rate-limiting windows to reduce user confusion.
2. Differentiated plan design: Offer tiered discounts for users with different usage scales to avoid low-priced plans being poor value.
3. Elastic capacity planning: Improve compute supply elasticity through hybrid deployment (owned clusters + elastic cloud resources) and inference optimization (quantization, distillation, batching).
4. Legacy user rights protection: During capacity constraints, prioritize existing users to maintain brand trust.

6. Summary

The compute quota and subscription mechanisms of LLM services are essentially a trade-off among resource supply, cost control, and user experience. The discount rate of subscription plans, the impact of cache hit rates, and the continuation of legacy user rights are all technical details developers should focus on when evaluating service value.

For developers, understanding the technical logic behind billing models and optimizing their own usage cache efficiency is an effective path to reducing AI application costs. For platforms, transparent rules, elastic capacity, and smooth upgrade paths are key to earning long-term user trust.

At a time when compute supply remains in tight balance, rationally assessing demand, designing quotas reasonably, and continuously optimizing inference efficiency will be a shared challenge for the entire industry.