Designing LLM Subscription Quotas: Logic and Engineering Practices
LLM quota design affects how users experience rate limits and how much resource a subscription plan can safely allow. This article covers the core engineering decisions behind quota rules: fixed windows and their boundary burst problem, sliding windows and token/leaky bucket alternatives, the ratio checks needed when multiple windows are stacked, and reset and carry-over rules for continuous subscriptions. Python examples are included for a sliding-window counter, multi-window validation, and next-reset calculation.
1. The Basic Problem in Quota Design
Whether an LLM service is billed by usage or by subscription, one question must be answered: how much resource can a user consume in a given period? A quota rule is that answer. If the rule is unclear, users probe the boundary repeatedly, support teams receive overlapping questions, and trust in the product erodes.
From an engineering perspective, a quota is a set of rate-limiting policies. The system counts usage inside time windows, either request counts or token counts, and rejects or degrades requests when the limit is exceeded. The window length, number of windows, their relative ratios, and the reset behavior together determine the real user experience.
A good quota design usually satisfies three requirements:
- The rule can be explained in one or two sentences.
- Users can estimate their remaining quota and the next reset time in advance.
- Multiple rules do not contradict one another, and long-term behavior feels intuitive.
The rest of this article looks at these points in more detail.
2. Common Time Window Types
2.1 Fixed Window
A fixed window divides time into equal-length intervals. Each interval is counted independently, and the counter resets at the boundary. It is cheap to implement: one counter plus a window identifier.
The main drawback is boundary burst. Suppose the limit is 100 requests per hour. A user can consume 100 requests in minute 59 and another 100 requests in minute 61. In two minutes, the user has actually consumed 200 requests, far more than the rule intended.
2.2 Sliding Window
A sliding window counts usage in the trailing T seconds from the current moment. It has no fixed boundary, so it suppresses boundary bursts. The cost is that each request timestamp must be recorded, or time is divided into fine buckets for an approximate count.
The following event-queue implementation is useful for understanding the mechanism and for small-scale validation:
import timefrom collections import dequefrom typing import Optionalclass SlidingWindowQuota:"""Sliding-window quota: tracks cumulative usage in the last window_seconds."""def __init__(self, limit: int, window_seconds: int):self.limit = limitself.window = window_secondsself.events = deque() # (timestamp, amount)self.used = 0def _evict(self, now: float) -> None:# Remove events that have fallen outside the window.while self.events and now - self.events[0][0] >= self.window:_, amount = self.events.popleft()self.used -= amountdef try_consume(self, amount: int, now: Optional[float] = None) -> bool:now = time.time() if now is None else nowself._evict(now)if self.used + amount > self.limit:return Falseself.events.append((now, amount))self.used += amountreturn Truedef remaining(self, now: Optional[float] = None) -> int:now = time.time() if now is None else nowself._evict(now)return self.limit - self.used
In production systems with high concurrency, window state is usually stored in shared storage, and bucket approximation or token buckets are used to reduce memory and synchronization cost.
2.3 Token Bucket and Leaky Bucket
A token bucket adds tokens at a fixed rate, and requests consume tokens. The bucket has a capacity limit, so it permits short bursts while keeping the long-term average rate controlled. It is suitable for scenarios that need burst capability but still require a controlled average rate.
A leaky bucket outputs requests at a constant rate; excess requests are queued or dropped. It produces the smoothest output, but users hardly perceive any burst capacity.
The choice depends on the goal: fixed windows emphasize simplicity, sliding windows emphasize boundary accuracy, and token/leaky buckets emphasize rate shaping.
3. Designing Ratios When Multiple Windows Are Stacked
Real systems typically use several windows at once, such as a short window, a weekly window, and a monthly window. When multiple windows apply, the first limit a user hits determines what is actually felt. The relationship between windows therefore matters more than any single number.
3.1 When the Ratio Is Too Wide or Too Narrow
Let T_s be the short-window duration and Q_s its quota, and let T_l be the long-window duration and Q_l its quota. If a user spreads usage evenly across the long window, the average quota available in one short window is:
E = Q_l × (T_s / T_l)
This is a useful baseline for deciding whether the short window has meaningful constraint.
- If
Q_sis significantly aboveE, the short window has little effect on users who spread usage evenly. It only catches concentrated bursts. A user might consume most of the long-window quota in one short period, which defeats the purpose of the short window. - If
Q_sis close toQ_l, the short window is basically a copy of the long window and cannot smooth usage. - If
Q_sis much lower thanE, the short window will trigger frequently. Users may feel limited even when their total quota is sufficient, so prompts and explanations are needed.
3.2 Configuration Validation Example
The following helpers calculate the expected share at configuration time, so contradictory rules can be caught before launch:
def expected_share(long_quota: float, short_hours: float, long_hours: float) -> float:"""Average quota available in one short window when usage is spread evenly."""if long_hours <= 0:raise ValueError("long window duration must be positive")return long_quota * short_hours / long_hoursdef check_window_config(short_quota: float, long_quota: float,short_hours: float, long_hours: float) -> dict:share = expected_share(long_quota, short_hours, long_hours)ratio = short_quota / share if share else float("inf")return {"expected_share": share,"short_to_share_ratio": round(ratio, 2),"note": "A ratio well above 1 means the short window is weak for evenly distributed usage; ""a ratio well below 1 means the short window will trigger often and needs explanation.",}
This function does not prescribe a universal ratio. It quantifies the design trade-off so the team can review it before rollout.
3.3 Define the Target Before Setting Ratios
The core question is: which behavior should the rule constrain?
| Constraint goal | Recommended approach |
|---|---|
| Limit instantaneous bursts and protect service stability | The short window must be binding. Its quota should be close to, or slightly above, the long-window baseline share. |
| Ensure fairness in long-term total usage | The long-window quota is the primary constraint; the short window only smooths usage. |
| Encourage users to consume at a steady pace | Short and long windows should be coordinated so ordinary usage does not trigger limits. |
Once the goal is clear, write the ratio into the rule description instead of relying on after-the-fact explanations.
4. Periodic Reset and Cross-Period Usage
4.1 Three Reset Anchors
A long-period quota must define when a period starts and when it resets. Three anchors are common:
- Calendar-based reset: Quota resets at midnight on the first day of each month. It is simple and easy to explain externally.
- Subscription start date: The cycle starts at purchase or renewal, so each cycle has a fixed length, but different users reset on different dates.
- Rolling period: At any moment, the system looks back over the past N days, so there is no single reset time.
Calendar-based reset is convenient for operational reporting. Subscription start date aligns with users’ intuitive sense of “my month.” Rolling period is closest to actual usage but has a higher explanation cost.
4.2 Calculating the Next Reset Time
With a subscription start date as anchor, the next reset time can be derived from the period length. The implementation below assumes a fixed period length, suitable for daily or hourly cycles. For calendar-month cycles, use a calendar library to handle different month lengths.
from datetime import datetime, timedeltadef next_reset(anchor: datetime, period: timedelta, now: datetime) -> datetime:"""Return the next reset time after now, based on the fixed anchor."""if now < anchor:return anchorelapsed = now - anchorcycles_passed = elapsed // period # timedelta division gives whole cyclesreturn anchor + (cycles_passed + 1) * period# Example: a subscription starts at 2026-01-15 10:00 and has a 30-day period.anchor = datetime(2026, 1, 15, 10, 0)print(next_reset(anchor, timedelta(days=30), datetime(2026, 2, 20, 9, 0)))# Output: 2026-03-16 10:00:00
4.3 Continuous Subscriptions and Quota Carry-over
When users renew continuously or prepay for a longer period, the interaction between periods becomes more complex. If each period’s quota is independent and resets to zero, unused quota is lost and users may feel their quota was “eaten.” If unlimited carry-over is allowed, quota accumulates and the rules lose their constraining effect.
Common ways to handle this include:
- Capped carry-over: Unused quota can be rolled into the next period, but the carry-over amount is capped, for example at a certain proportion of one period’s quota.
- Daily or weekly release: The total long-period quota is released in fixed daily or weekly increments. Users can see the current available amount and understand why some remaining quota is not usable yet.
- Clear rollover rules: Document how quota is calculated across periods, whether unused quota carries over, and how it is applied in the next period.
