Why AI Coding Assistant Quota Increases Feel Invisible: Dual-Window Rate Limiting and Fill-Rate Math

1. The Quota Went Up, So Why Didn’t the Coding Experience Change?

First, let’s fix the baseline so the comparison isn’t misread as “the same number before and after.” Take a high-tier plan of an AI coding assistant as an example. Before the version update, the long-window (7-day) quota W of the strong-reasoning model was roughly equal to the capacity C of one short window (about 5 hours), i.e. W ≈ C. After the update, it became W ≈ 3C. That is an increase of roughly 3x — not “3C before, 3C after.”

Setting the specific product aside, quotas like this sit on top of a dual-window rate limiting structure, which some model services adopt in similar form:

• Short window (burst window): a rolling window of a few hours, limiting instantaneous throughput per unit of time.
• Long window (periodic quota): daily/weekly/monthly, limiting cumulative consumption over a longer period.

The real usable capacity is the intersection of the two, both active at the same time, so you cannot estimate it by dividing total volume by duration. What matters first is identifying which layer is the bottleneck:

• Blocked on the short window: actual demand inside a single short window is at or above C (the burst window is saturated), while total weekly demand is clearly below W. Raising the long-window quota does nothing at all — and this is exactly the most typical cause of “the quota went up but nothing changed.”
• Blocked on the long window: the team is running at full load (demand per window is continuously at or above C) and W < N·C. Here the long-window quota is the binding constraint on total volume, and moving W from 3C to 6C extends full-load usable time from roughly 15 hours to roughly 30 hours — a change you can actually feel.

Put differently, the short window only gets a say when per-window demand is below C; once you are at full load, the long window is the bottleneck, and a quota increase is noticeable. Measure which category you are in before deciding whether governance is worth it.

2. How Do You Calculate the Real Capacity Ceiling?

Let the short-window (5-hour) quota ceiling be C (which can be converted into some number of high-intensity conversations or agent tasks), and the long-window (7-day) quota be W. Assume 40 effective coding hours per week, split into N = 40 / 5 = 8 short windows.

Fill rate                  = W / (N × C)
Long-term average rate     = W / 40h
Peak available rate        = C / 5h
Average / peak             = fill rate (same value)

Scenario	W	Fill rate	Full-load usable time (40h/week)
Before update	C	12.5%	~5 h
After update	3C	37.5%	~15 h
If raised to	6C	75%	~30 h

All of the above are illustrative calculations based on the ratio between W and C, a 40h/week assumption, and full-load usage — they do not describe any specific vendor’s rules. The point is visible: a 37.5% fill rate means that even if a developer wanted to run at full load continuously, the weekly quota is only enough to fill 3 of the 8 short windows; the remaining ~25 hours have a quota of zero.

Now layer in two forms of loss (illustrative framing; both need calibration against telemetry):

1. Retry amplification: when agent-style tasks fail, the context has to be re-run, so budget 1.3x–2x. That puts the share of real delivered work relative to actual consumption at roughly 1/2 to 1/1.3, i.e. 50%–77%. Against the post-update 15 hours, that leaves roughly 7.5–11.5 hours for genuine demand.
2. Context inflation: injecting a whole repository, long files, or large blocks of logs in one shot significantly increases the consumption of a single request. This article does not give a specific multiplier for this item — it is a hypothesis to be verified, and telemetry (Section 4.6) is what confirms it.

The conclusion is straightforward: whether a quota increase is perceptible depends on the fill rate and on which layer you are bottlenecked at; and when the fill rate is low, governing wasted consumption pays off faster than waiting for the next increase — but that judgment must be validated with telemetry rather than intuition.

3. What Hidden Damage Does a Tight Quota Cause in the Development Process?

• Decision downgrading: near the quota boundary, developers deliberately avoid large refactors and cross-module tracing, reserving the strong-reasoning model for small changes that “look safe” — and technical debt accumulates over time.
• High interruption cost: when a task hits rate limiting halfway through, the context has to be rebuilt, and re-orienting often costs more than re-running the task itself.
• Behavioral distortion: people start “hoarding quota” and “saving up for one big move,” turning a rhythm of small continuous commits into large, high-risk batch merges — which conflicts with code review and continuous integration best practices.
• Team unfairness: without scheduling, a few heavy users drain the shared quota and everyone else spends the whole week with none available.
• Distorted measurement: looking only at “number of calls” or “quota consumed” cannot distinguish consumption driven by high-value work from consumption wasted on ineffective retries.

4. How Do You Govern Quota? Six Actionable Levers

The goal of governance is not “save quota” but move quota from low-value requests to high-value requests.

4.1 Prerequisite: Identify the Bottleneck Layer First

Observation	Short-window fill rate	Periodic fill rate	Diagnosis	First-choice action
A	Frequently hits ceiling ≈100%	< 60%	Burst bottleneck	Peak shaving with queuing, lower priority for batch jobs, offload to local models
B	Long-term < 30%	Close to 100%	Total-volume bottleneck	Add a supplementary channel / adjust capacity + cut wasted consumption + reuse results
C	Both < 30%	< 30%	No obvious bottleneck	Postpone governance; shift focus to quality and accuracy

4.2 Task Tiering and Tiered Model Routing

Tier	Typical tasks	Suggested routing
L1	Single-line completion, renaming, boilerplate, test skeletons	Low-latency, small-parameter model
L2	Single-file refactoring, error localization, SQL/script writing	Medium-capability model
L3	Cross-module architectural changes, root-cause analysis of hard defects, performance profiling	Strong-reasoning model (separate quota + approval gate)

The upfront classification can be done with rules alone; no extra model call is needed. The sources of the Task fields are explicit: change_lines is the sum of added plus deleted lines aggregated from git diff --numstat, files is the number of changed files, and has_stack_trace comes from whether the error log matches a traceback/error line.

from dataclasses import dataclass
@dataclass
class Task:
    change_lines: int      # aggregated from git diff --numstat
    files: int             # number of changed files
    has_stack_trace: bool  # log matches traceback / error
    needs_approval: bool = False
def route(task: Task) -> str:
    if task.needs_approval:
        return "tier_l3"              # cross-module change; release only after review
    if task.change_lines < 20 and task.files == 1:
        return "tier_l1"              # 20 lines / single file is an empirical threshold; calibrate in shadow mode
    if task.files <= 3 and task.has_stack_trace:
        return "tier_l2"
    return "tier_l3"

Give L3 its own quota and admission conditions so it isn’t eaten away by everyday trivial tasks.

4.3 Context Budget Trimming: Trade Retrieval for Context

Set a hard token budget per request and fill it by priority: ① relevant functions in the file being edited (locate them by symbol-level search rather than pasting whole files) → ② interface definitions and type declarations that are directly depended on → ③ the most recent error stack and key log fragments → ④ necessary project convention snippets → ⑤ nothing else gets injected. For the same task, this usually lowers quota consumption and improves answer accuracy at the same time.

4.4 Reuse, Caching, and Deduplication

• Result caching: key on (prompt template + code fingerprint + context summary + dependency versions + toolchain version + model version). Once the code fingerprint changes with the repository, old cache entries must be invalidated, otherwise you get “fixes” written against stale code.
• Index reuse: pin repository indexes, dependency graphs, and document chunks to versions so they don’t have to be rebuilt every time.
• Request deduplication: merge identical questions within the team so one analysis serves everyone.
• Knowledge consolidation: distill recurring conclusions into team docs or rule files, turning “ask the model every time” into “ask once, look it up forever.”

4.5 Team-Level Quota Scheduling

The recommended approach is “token bucket + priority queue”: a short-window token bucket prevents any single caller from saturating the burst capacity, while long-window quota is allocated per person / per group and supports borrowing and reclamation. The over-quota portion for high-priority work goes into a separate overflow bucket, which is itself subject to the burst rate limit, so the short-window ceiling can’t be breached without bound.

import time
import threading
class DualWindowQuota:
    """Illustrative single-process implementation; for multi-replica deployment,
    replace with shared storage + a distributed lock."""
    def __init__(self, burst_cap, period_cap,
                 burst_secs=5 * 3600, period_secs=7 * 86400, overflow_ratio=0.1):
        self.burst_cap, self.burst_secs = burst_cap, burst_secs
        self.period_cap, self.period_secs = period_cap, period_secs
        self.overflow_cap = period_cap * overflow_ratio   # reserved overflow quota L3 may tap
        self.burst_used = self.period_used = self.overflow_used = 0.0
        self.burst_ts = self.period_ts = time.monotonic()
        self._lock = threading.Lock()
        self._queue = []          # (priority, cost, task_id)
    def _refill(self):
        # Continuously topping up the buckets only approximates a 5h / 7d rolling window;
        # a strict rolling window requires recording the timestamp of each consumption event
        now = time.monotonic()
        self.burst_used = max(0.0, self.burst_used
                              - self.burst_cap * (now - self.burst_ts) / self.burst_secs)
        self.period_used = max(0.0, self.period_used
                               - self.period_cap * (now - self.period_ts) / self.period_secs)
        self.overflow_used = max(0.0, self.overflow_used
                                 - self.overflow_cap * (now - self.burst_ts) / self.burst_secs)
        self.burst_ts = self.period_ts = now
    def try_acquire(self, task_id, cost, priority=0):
        with self._lock:
            self._refill()
            over_burst = self.burst_used + cost > self.burst_cap
            over_period = self.period_used + cost > self.period_cap
            if not (over_burst or over_period):
                self.burst_used += cost
                self.period_used += cost
                return True
            # High-priority tasks may only borrow the overflow bucket,
            # and the burst ceiling can never be breached
            if priority >= 2 and not over_burst \
                    and self.overflow_used + cost <= self.overflow_cap:
                self.overflow_used += cost
                self.burst_used += cost
                self.period_used += cost
                return True
            self._queue.append((priority, cost, task_id))
            return False        # still over the limit: queue for reclamation,
                                # or hand off to the degradation path in 4.7
    def release(self, task_id, cost, priority=0):
        """Return quota when a task is cancelled or judged not to need a retry,
        and admit waiting items from the head of the queue by priority."""
        with self._lock:
            self.period_used = max(0.0, self.period_used - cost)
            self.burst_used = max(0.0, self.burst_used - cost)
            if priority >= 2:
                self.overflow_used = max(0.0, self.overflow_used - cost)
            admitted = 0
            for item in sorted(self._queue):
                p, c, tid = item
                self._queue.remove(item)
                if self.burst_used + c <= self.burst_cap \
                        and self.period_used + c <= self.period_cap:
                    self.burst_used += c
                    self.period_used += c
                    admitted += 1
                else:
                    self._queue.append(item)
                    break
            return admitted

Deferrable tasks such as bulk code checks and documentation generation in CI pipelines run at low priority, while interactive human-facing tasks run at high priority.

4.6 Observability and Metrics

• Quota utilization: short-window fill rate and periodic fill rate (the two formulas from Section 2).
• Output per unit of quota: merged PRs / closed-loop tasks per thousand units of quota consumed.
• Waste rate: share of consumption from failed retries and abandoned tasks.
• Degradation rate: share of requests pushed back to manual work or a local model for lack of quota.
• Model distribution: share of requests and corresponding output for each of L1/L2/L3.

Of these, the waste rate is usually one of the first optimization levers to show results, and it must be validated with telemetry rather than assumed. Don’t set thresholds by feel: run a two-week baseline first, then set alert lines at the P75 percentile.

4.7 Graceful Degradation

Strong-reasoning model → Medium-capability model → Local small-parameter model (completion/static checks) → Traditional static analysis toolchain

When degrading, tell the developer explicitly which tier is in use and why, to avoid the trust damage caused by “the model suddenly got dumber.” Local models are a good fit for high-frequency, low-difficulty work such as completion, formatting, and basic lint fixes; they consume no quota but still carry compute and hardware costs, which have to be included in capacity planning.

5. How Do You Choose Capacity? The “1 + N” Mix and Common Mistakes

Dual-window quotas mean that any single subscription has a hard capacity ceiling, so the sensible approach is a combination:

• Primary subscription: covers daily high-frequency interaction, handling L1/L2.
• Supplementary channel: pay-as-you-go to absorb L3 peaks, so the long-window quota isn’t the only binding constraint.
• Local capability: handles high-frequency, low-difficulty tasks as a fallback.
• Self-hosted service: sensitive codebases can run open-source small-parameter models on a container platform, with prompt templates and evaluation sets stored in object storage.

Two mistakes are common. One is treating “raise the quota” as the only solution while ignoring fill rate and wasted consumption. The other is counting only the number of calls without any value tiering.

An example deployment shape (not a vendor recommendation): on a cloud platform, the quota scheduling layer, metric collection, and degradation decisions can be deployed as stateless services on a container platform; logs and metrics feed into unified logging plus monitoring and alerting; templates and evaluation sets live in object storage, forming a closed loop of “collect → measure → schedule → alert.” Actual quota rules and billing methods must follow whatever each provider publishes; this article only discusses vendor-neutral governance methods.

6. Rollout Roadmap

• Week 1: ship instrumentation; get a picture of fill rate, waste rate, model distribution, and the bottleneck layer (the A/B/C classification in 4.1).
• Weeks 2–3: roll out tiered routing and context budget trimming, running in shadow mode first (measure only, don’t block) to calibrate thresholds and routing accuracy.
• Week 4: connect team-level quota scheduling and the degradation path; define trigger conditions and notification methods for each tier.
• Weeks 5–6: build the metrics dashboard and a weekly review; make “merged PRs per thousand units of quota” the north-star metric.

7. FAQ

Q: Why does a quota increase still feel invisible? In most cases you are blocked on the short window: per-window demand is already saturated while total weekly demand is well below W, so raising W changes no constraint. Conversely, if you are running at full load and W < N·C, the long window is the bottleneck and the increase is perceptible.

Q: How low is a low fill rate? There is no universal threshold. Measure a two-week baseline first; a fill rate that stays below 30% long-term while the periodic quota is close to saturated indicates clear room for governance.

Q: Will governance slow down delivery? Short term, it adds instrumentation and shadow-mode cost, and the payoff comes mainly from lower waste. Measure it as “merged PRs per thousand units of quota” rather than absolute call volume.

8. Decision Checklist

1. Instrument first and identify the bottleneck layer: short window (peak shaving / degradation), long window (more capacity / add a channel), no bottleneck (shift to quality).
2. Then cut wasted consumption: context trimming + result caching + request deduplication — this is the fastest lever to show results.
3. Then do tiered routing and team scheduling, carving out a separate L3 quota with admission conditions.
4. Only then consider scaling up or adding a supplementary channel, and validate each spend with output per unit of quota.
5. Throughout, calibrate thresholds with telemetry data; don’t draw conclusions from unverified multipliers or ratios.