Metering and Quota Governance for AI Subscription Services

\n\n## 1. Problem Definition: Why “Multiplier-Based Quota Claims” Cause Disputes\n\nThe commercialization of AI subscription services depends on a quota contract. However, users usually perceive limits as “number of messages,” while the system actually deducts tokens and request counts. This creates a structural mismatch between what is promised and what is metered.\n\nMultiplier-based marketing, such as “5x” or “20x relative to the basic plan,” usually implies three assumptions that are not explicitly stated:\n\n1. Unclear baseline: The multiplier may refer to a specific plan, model, or interaction mode.\n2. Unclear metering unit: Usage may be measured by tokens, requests, or session quota.\n3. Unclear window: Short windows, such as a rolling five-hour window, and long periods, such as a weekly total, are independent quotas. A multiplier often applies to only one layer.\n\nThere have already been collective complaints and lawsuits caused by quota claims that do not match actual usage. From an engineering perspective, most such disputes can be attributed to unobservable metering definitions and quota layers that are not exposed in the interaction interface, rather than to marketing alone.\n\n### 1.1 Four Factors That Amplify Perceived Quota Consumption\n\n| Amplification factor | Mechanism | Impact on effective interaction count |\n|—-|—-|—-|\n| Context resending | Each turn in a multi-turn conversation resends historical messages | Prompt tokens grow approximately linearly with turns |\n| Long documents and code context | A single input can reach tens of thousands of tokens | The cost of a single interaction can increase by one to two orders of magnitude |\n| Reasoning model overhead | Intermediate reasoning is counted as output tokens | The cost of a single completion rises significantly |\n| Agent-style loops | One user request can trigger multiple tool calls and self-checks | One “message” may translate into multiple underlying calls |\n\nThe conclusion is that using “message count” as the external quota metric is inherently distorted. The only stable and reconcilable metric is weighted tokens and request counts.\n\n## 2. Formalizing the Quota Metering Model\n\n### 2.1 Metering Unit and Weights\n\nEach invocation can be converted into a unified “quota point”:\n\ntext\npoints = α * prompt_tokens\n + β * completion_tokens\n + γ * cached_read_tokens\n + δ * tool_call_count\n + ε * image_or_audio_units\n\n\nKey points:\n\n- The weight of cached_read_tokens should be significantly lower than that of normal input tokens, so that prompt caching and context reuse are encouraged.\n- The weight of completion_tokens should be higher than that of prompt_tokens, because the generation stage consumes more compute.\n- Weights are product parameters rather than technical parameters. They should be configurable and strictly consistent with the publicly disclosed conversion rules.\n- Public-facing materials must provide a verifiable conversion anchor, such as “1,000 quota points is approximately equal to how many medium-complexity conversations.”\n\n### 2.2 Quota Layers: Short Window, Long Period, and Execution Constraints\n\nA practical design uses multiple layers in parallel. If any layer reaches its limit, the system should degrade or reject the request:\n\n1. Short-window rate layer: A rolling five-hour or rolling one-hour session quota, used to smooth transient peaks and protect backend capacity.\n2. Period total layer: A natural week or rolling seven-day cumulative quota, used to control long-term cost and prevent single-account abuse.\n3. Concurrency and context layer: Maximum concurrent requests, maximum context length per request, and maximum output length.\n\nAn important design principle is that removing the short-window limit does not mean removing the period cap. Higher-tier plans can relax the short-window constraint and let users allocate usage across the whole period, but the period total should remain a hard constraint.\n\n### 2.3 Window Algorithm Selection\n\n| Algorithm | Accuracy | Memory cost | Suitable scenario |\n|—-|—-|—-|—-|\n| Fixed-window counter | Low; can allow double bursts at boundaries | Very small | Rough rate limiting; not recommended for billing |\n| Sliding-window log | High | Grows linearly with QPS | Low traffic or strong audit requirements |\n| Token bucket | Medium; allows bursts | Small | Rate shaping |\n| Weighted sliding-window counter | High | Constant | Recommended for quota metering |\n\nThe weighted sliding-window approach divides a large window into N equal-length buckets. Each bucket stores only an accumulated value. When reading, the current bucket and the previous bucket are weighted by time proportion. This achieves accuracy close to a sliding-window log while using constant memory.\n\n## 3. System Architecture and Consistency Design\n\n### 3.1 Layered Architecture\n\ntext\nClient\n -> API gateway (authentication, tenant identification, request deduplication key)\n -> Quota admission service (multi-layer window checks + reservation)\n -> Real-time counting layer (distributed cache + atomic scripts)\n -> Persistent metering layer (append-only usage event stream)\n -> Billing and accounting layer (reconciliation, export, visualization)\n -> Inference execution layer (model routing, cache hits, context trimming)\n -> Usage callback (actual token settlement -> reservation reversal)\n\n\nDesign principles:\n\n- Reserve first, settle later, and reverse the difference. Before a request, estimate usage based on context length and freeze the corresponding quota. After the response, settle the difference using the actual token count. This avoids over-issuance during long responses.\n- The usage event stream is immutable. Each invocation produces an event with a request_id. The real-time counter is only a materialized view of the event stream and can be rebuilt at any time.\n- Reconciliation should be routine. Periodically compare cached counters with aggregated event-stream values and alert when deviations exceed a threshold.\n\n### 3.2 Atomic Admission Implementation (Pseudo-code)\n\npython\ndef try_acquire(account_id, estimated_points, now_ms):\n key = f\"quota:{account_id}\"\n\n # Perform multi-layer checks and reservation in one script to ensure atomicity\n result = atomic_script(\n KEYS=[key],\n ARGV=[\n now_ms,\n WINDOW_SHORT_MS, # for example, 5 hours\n LIMIT_SHORT_POINTS,\n WINDOW_LONG_MS, # for example, 7 days\n LIMIT_LONG_POINTS,\n estimated_points,\n request_id, # idempotency key\n ]\n )\n\n if result.status == \"OK\":\n return Decision.ALLOW(reservation_id=request_id)\n\n return Decision.DENY(\n reason=result.reason, # SHORT_WINDOW / LONG_TOTAL\n retry_after_ms=result.retry_after # must be returned for client prompts\n )\n\n\ndef settle(request_id, actual_points):\n # Settle actual usage and release the reservation difference; repeated calls must be idempotent\n usage_store.commit_once(\n idempotency_key=request_id,\n points=actual_points,\n )\n atomic_script(settle_keys(request_id),\n [request_id, actual_points])\n\n\nEngineering details:\n\n- The estimator for estimated_points should be based on historical quantiles rather than a fixed value. A common choice is around P70. If the estimate is too high, valid requests may be rejected; if it is too low, over-issuance may occur.\n- The idempotency key must be generated by the API gateway and bound to the request. Retries, timeouts, and client disconnections must not cause duplicate charges.\n- retry_after_ms is a key field for user experience. Without it, the system cannot provide a credible “when will access be restored” prompt.\n\n### 3.3 Observability Metrics\n\n- Rejection rate and the distribution of rejection reasons, such as short-window versus period-total limits.\n- The deviation distribution between reserved and settled usage, used to calibrate the estimator.\n- Cache hit rate and the average token count per effective interaction.\n- Next-day retention of users who hit quota limits and conversion along downgrade paths.\n- Reconciliation deviation between counters and the event stream.\n\n## 4. Plan Tiering and Account Verification Mechanisms\n\nThe mainstream industry structure uses four tiers: Free, Basic, Advanced, and Flagship. Multiplier-based claims usually describe the ratio of single-session usage capacity relative to the Basic tier, rather than any form of “message count package.”\n\nThe Flagship tier can have three differentiated capabilities:\n\n1. Relaxed short-window limits: Users are no longer interrupted by rolling hourly windows and can allocate usage across the entire period.\n2. Retained period cap: The period total remains the final gate for cost control and fairness.\n3. Separate sub-item quotas: Some derived capabilities, such as sub-agents or batch tasks, can have separate weekly limits to avoid competing with the main conversation quota.\n\n### 4.1 Automatic Verification Design Points\n\nUnlocking Flagship-tier benefits usually depends on account trust verification. Engineering design should include:\n\n- A combination of verification signals: Account age, payment history completeness, subscription continuity, and abnormal call patterns can be evaluated using a hybrid of rules and models.\n- A temporary restriction period: New accounts should retain temporary window limits until verification is complete, preventing abuse immediately after registration.\n- An explicit state machine: PENDING -> VERIFIED / REJECTED. Every state transition should trigger a notification and provide an appeal entry.\n- Default exemption for existing users: Benefit changes should apply automatically to already subscribed users without manual action, avoiding the perception that rule changes harm existing customers.\n- Audit traceability: Verification basis and effective time should be traceable. This is essential for responding to compliance inquiries.\n\n## 5. Transparency Design: Resolving Disputes in the Interface\n\nHalf of quota governance happens in the interface. The following four items should be exposed:\n\n1. Remaining quota and reset countdown for the current window, clearly labeled with the corresponding window layer.\n2. Usage breakdown view: Consumption should be broken down by session, model, and tool-call dimension so users can understand “why this request was expensive.”\n3. Deterministic behavior when the limit is reached: The system should clearly define whether it queues the request, automatically downgrades to a lightweight model, or rejects the request and returns a recovery time.\n4. Conversion explanation and examples: Provide a reference range for medium-complexity conversations and state that the range varies with task complexity.\n\nCombined with front-end context management, such as history summarization, on-demand retrieval instead of full replay, and prompt cache reuse, platforms can reduce invalid token consumption without sacrificing user experience. This is both a cost optimization and an improvement in perceived quota fairness.\n\n## 6. Change Management and Compliance Baseline\n\nPlan rule changes are contractual changes. Engineering and legal teams must work together to meet the following requirements:\n\n- Advance announcement: Publish the effective time, affected scope, and comparison between old and new rules in advance.\n- Consistency checks: Every number in marketing materials must be directly derivable from the metering system configuration. Separate maintenance of copy and configuration should be prohibited.\n- Exportable usage data: Users should have basic capabilities to query and reconcile their own usage.\n- Canary release and rollback: Rule changes should first be applied to internal users and small traffic, with a one-click rollback available.\n- Retention and audit: Metering events and configuration change history should be retained for a long time to support dispute resolution.\n\nWithin Baidu AI Cloud’s general-purpose cloud offerings, these capabilities can be implemented using message queues for usage event stream peak shaving and ordering guarantees, log services for event retention and search, monitoring and alerting for deviation and rejection dashboards, and object storage for reconciliation snapshot archiving. A fully custom infrastructure is not required.\n\n## 7. Implementation Checklist\n\n1. Unify the metering definition: Define the quota point formula and weights, make them configurable, and put them under version control.\n2. Build the event stream first: Ensure every invocation is traceable and recomputable before focusing on rate limiting.\n3. Implement atomic admission: Build the reserve-settle-reverse loop with idempotency and reconciliation.\n4. Launch the transparency panel: Include remaining quota, countdown, usage breakdown, and recovery time.\n5. Establish a verification state machine: Automate benefit verification with notifications and appeals.\n6. Standardize the change process: Keep marketing copy and configuration from the same source, support canary release, allow rollback, and retain evidence for dispute resolution.\n\nThe core judgment is that sustainable operation of AI subscription services depends on whether users can understand the quota system as a set of deterministic rules rather than a black box that requires guessing. If metering units are verifiable, quota layers are explainable, and change processes are traceable, multiplier-based claims can align with the actual user experience.\n