Free LLM API Pitfalls: A Practical Guide to 5 Common Traps

    Free LLM APIs are often wired in behind a primary model as a low-cost fallback, but a "free quota" is not the same as something that's ready for production. This guide breaks down five high-frequency pitfalls when integrating free LLM APIs: free models missing from the model list, HTTP 200 responses with empty content, probabilistic 429 rate limiting, bimodal latency tails, and inconsistent response structures. For each one it explains the symptom and root cause, then shows what to do about it. Aimed at backend and platform engineers, it walks through reproducible methods for probing models, validating success, retrying with backoff, circuit breaking, graceful degradation, and monitoring free-tier health, and includes parameter-budget guidance plus a runnable Python probe script.

    Introduction

    A free quota does not mean free availability.

    Many teams put a free LLM API behind their main pipeline as a fallback: when the primary model times out, gets rate-limited, or returns an error, traffic automatically switches to the free channel to maintain availability at low cost. In the ideal case, the fallback channel only handles a small fraction of failed requests, sees far less traffic than the primary path, and its cost is negligible.

    But there is still engineering distance between “the call works” and “it’s ready for production.” When integrating a free model in practice, the common problems are usually not “the endpoint is completely down” but subtler deviations: the documentation lists a free model, yet calling it returns an insufficient-quota error; the endpoint returns HTTP 200, but the content field is empty; a single request looks fine, but repeated load tests trigger frequent 429s; average latency looks low, but the P95 tail is obvious; and different models return inconsistent structures, so parsing code fails silently.

    This article abstracts a single-environment debugging session into a general-purpose calling playbook. It focuses on five high-frequency pitfalls: wrong model selection, empty responses, rate limiting, bimodal latency, and inconsistent response structures. It is written for backend or platform engineers who want to use a free API as a fallback. The goal is not to evaluate any particular platform, but to provide a reproducible set of methods for probing, retrying, degrading, and monitoring.

    One caveat up front: models being added or removed, free-tier policies, and rate-limit thresholds change quickly, so any list of free models has a shelf life. The parameters and code here are an engineering starting point; before integrating for real, you still need to test with your own key and model IDs.

    1. The Bottom Line: One-Line Handling for 5 Common Failures

    Before diving into the details, here is a conclusion-first table. If you only care about how to handle each case, start here.

    Failure type Typical symptom One-line fix
    Free model not in the model list Can’t find the free model via the model list API, or a similarly named model returns an insufficient-quota error Don’t rely on the model list API alone; probe each candidate with a minimal request
    HTTP 200 but empty content Status code is fine, no error message, content is empty Check max_tokens, the reasoning model’s budget, and your parsing function
    Probabilistic rate limiting The same model sometimes succeeds, sometimes returns 429 Respect Retry-After, use exponential backoff, and rate-limit on the client side
    Bimodal latency Average latency is low, but P95 is very late Set timeouts per model category, and record P50/P95
    Inconsistent response structure Across models, content is a string, array, or object Unify extract_text and the is_success check

    2. Scope: Where These Conclusions Come From

    To avoid mistaking a single test run for a universal conclusion, let me first state the boundaries.

    • Account dimension: An account’s signup date, historical usage, region, and verification status can all affect free-quota allocation and rate-limit thresholds.
    • Time dimension: When a platform adjusts model availability, free-tier policies, or rate-limit strategies, it usually does not notify callers individually. A model that worked yesterday may be gone today.
    • API dimension: This article only covers inference APIs compatible with the OpenAI Chat Completions style, with parameters like model, messages, and max_tokens. Extended capabilities such as tool calling and structured output are out of scope.
    • Network dimension: Timeouts, DNS, proxies, and regional networking all affect results. In particular, latency and timeout data cannot be directly extrapolated to every environment.

    So the right way to use this article’s conclusions is to design your probing methods, fallback strategies, and monitoring metrics, not to replace your own testing.

    3. Not All Free Quotas Are the Same: Know Which One You’re Using

    The free quotas in the industry fall into roughly three categories, and they fail in completely different ways.

    1. Time-limited trial quota
    A block of trial credit granted when a new account or application is created, deducted per input/output token. When it expires, it converts to pay-as-you-go. What matters here is “how much is left.”

    2. Long-term free or very cheap lightweight models
    Platforms open smaller, lower-parameter models for free or at very low cost to grow their ecosystem. Model names usually carry suffixes like mini, lite, or flash. They suit low-complexity tasks, but their capability boundaries need separate validation.

    3. Rate-limited free quota
    This is not billed against a token balance; instead it limits the number of requests, tokens, or concurrent connections per unit of time. When your request volume is far below the primary model’s, this kind of quota is best suited for fallback. What matters here is “how many requests per second.”

    Many people, on hitting 429 for the first time, go back to the console to repeatedly check their token balance, find it normal, and start suspecting the platform’s rate limiting is malfunctioning. In reality, 429 describes the request rate, not insufficient balance. Rate-limited free quotas often have no visible balance number; you can only read the limits from the documentation’s rate statements and from fields like the Retry-After and X-RateLimit response headers.

    4. The 5 High-Frequency Pitfalls and How to Handle Them

    Pitfall 1: The Free Model Isn’t in the Model List, so Why Does Integrating from the List Fail?

    Symptom
    When you call the platform’s model list endpoint, it mostly returns the flagship paid models. You compare each against the free tier and can’t find a single one. Picking a model from the list API leaves you with no usable free model.

    A subtler problem: similar names don’t mean the same tier. A model that differs from a free lightweight model by only one letter in its suffix can still return an insufficient-quota error, no available resource package, or a permission error when called.

    Root cause
    The model list endpoint is generally built for generic clients, for example the OpenAI SDK, which relies on it to discover models. It focuses more on “commercial models available for sale,” while free quotas are essentially a marketing or ecosystem strategy that isn’t necessarily reflected in the API response.

    Fix
    The reliable way to confirm “truly free” is not to look at the list endpoint, but to run a minimal-request probe with your own key. Candidates should come from the official documentation, pricing page, or console notes, then be verified one by one.

    The probe code below reads the key from an environment variable to avoid hardcoding; sets the general budget to at least 512; and, for vision models without image input, only verifies that the text channel is available (to verify visual capability, send an additional minimal image request).

    1. import json
    2. import os
    3. import time
    4. from urllib.request import Request, urlopen
    5. from urllib.error import HTTPError, URLError
    6. BASE_URL = os.environ["LLM_BASE_URL"].rstrip("/")
    7. API_KEY = os.environ["LLM_API_KEY"]
    8. def post_json(url, payload, timeout=60):
    9. req = Request(
    10. url,
    11. data=json.dumps(payload).encode("utf-8"),
    12. headers={
    13. "Content-Type": "application/json",
    14. "Authorization": f"Bearer {API_KEY}",
    15. },
    16. method="POST",
    17. )
    18. try:
    19. with urlopen(req, timeout=timeout) as resp:
    20. body = json.loads(resp.read().decode("utf-8") or "{}")
    21. return resp.status, dict(resp.headers.items()), body
    22. except HTTPError as e:
    23. raw = e.read().decode("utf-8", errors="replace")
    24. try:
    25. body = json.loads(raw)
    26. except Exception:
    27. body = {"raw": raw}
    28. return e.code, dict(e.headers.items()), body
    29. except URLError:
    30. return None, {}, {}
    31. def _content_to_text(content):
    32. if isinstance(content, str):
    33. return content
    34. if isinstance(content, list):
    35. return "".join(
    36. p.get("text", "")
    37. for p in content
    38. if isinstance(p, dict) and p.get("type", "text") == "text"
    39. )
    40. if isinstance(content, dict):
    41. for key in ("text", "content"):
    42. if isinstance(content.get(key), str):
    43. return content[key]
    44. return ""
    45. def extract_text(body):
    46. if not isinstance(body, dict):
    47. return ""
    48. # Prefer OpenAI Chat Completions style first
    49. choices = body.get("choices")
    50. if isinstance(choices, list):
    51. for choice in choices:
    52. if not isinstance(choice, dict):
    53. continue
    54. msg = choice.get("message")
    55. if isinstance(msg, dict):
    56. text = _content_to_text(msg.get("content"))
    57. if text.strip():
    58. return text
    59. text = _content_to_text(choice.get("text"))
    60. if text.strip():
    61. return text
    62. # Also handle responses where top-level message.content is an object
    63. msg = body.get("message")
    64. if isinstance(msg, dict):
    65. text = _content_to_text(msg.get("content"))
    66. if text.strip():
    67. return text
    68. # Also handle top-level content / text / output_text
    69. for key in ("content", "text", "output_text"):
    70. if key in body:
    71. text = _content_to_text(body.get(key))
    72. if text.strip():
    73. return text
    74. return ""
    75. def classify(status, body):
    76. if status == 200 and extract_text(body).strip():
    77. return "OK"
    78. if status == 200:
    79. return "OK_EMPTY"
    80. if status == 429:
    81. return "LIMITED"
    82. if status in (400, 401, 402, 403):
    83. return "NO_FREE_QUOTA"
    84. if status in (500, 502, 503, 504):
    85. return "SERVER_ERROR"
    86. return "ERROR"
    87. def probe(model, kind="general"):
    88. # general: at least 512; vision without image input only probes the text channel
    89. budgets = {
    90. "general": 512,
    91. "reasoning": 2048,
    92. "vision": 1024,
    93. }
    94. payload = {
    95. "model": model,
    96. "messages": [{"role": "user", "content": "Reply with one word: OK"}],
    97. "max_tokens": budgets.get(kind, 512),
    98. }
    99. t0 = time.monotonic()
    100. status, _, body = post_json(f"{BASE_URL}/chat/completions", payload)
    101. elapsed = round(time.monotonic() - t0, 2)
    102. result = classify(status, body)
    103. usage = body.get("usage", {}) if isinstance(body, dict) else {}
    104. return {
    105. "model": model,
    106. "result": result,
    107. "elapsed_s": elapsed,
    108. "usage": usage,
    109. }

    The pass criteria must be consistent: a probe passes only when all three conditions hold — HTTP status 200, a non-empty content field, and no error information in the response. Checking only for HTTP 200 is not enough.

    Pitfall 2: Why HTTP 200 but Empty Content?

    Symptom
    When calling a reasoning model that shows its thinking, the endpoint returns HTTP 200 with no obvious error, but choices[0].message.content is an empty string. Or the top-level message.content is empty.

    This kind of failure is the most dangerous because it masquerades as success. If business code doesn’t validate content, it may write an empty string to the database or return an empty result to the user.

    Root cause
    Reasoning models usually generate reasoning content first, then the final answer. max_tokens limits the total length of this generation, typically including both the reasoning content and the final answer. If the request asks for a very short answer while max_tokens is very small, the budget may be exhausted during reasoning, leaving no room for the final answer.

    Another case: the reasoning content and the final answer are carried in separate fields, and the caller only reads one of them, so the content looks empty.

    Reproduction approach
    Fix the model and the question, and vary only max_tokens. For example, max_tokens=16 may return empty, while max_tokens=1024 returns normally.

    1. def empty_return_demo(model):
    2. for cap in (16, 1024):
    3. status, _, body = post_json(
    4. f"{BASE_URL}/chat/completions",
    5. {
    6. "model": model,
    7. "messages": [
    8. {"role": "user", "content": "Explain what idempotency is in one sentence"}
    9. ],
    10. "max_tokens": cap,
    11. },
    12. )
    13. print(f"max_tokens={cap:<5} status={status}")
    14. print(json.dumps(body, ensure_ascii=False)[:300], "\n")

    Fix

    1. Set parameter budgets per model category: lightweight general models can start at 512 to 1024; reasoning models should start at 2048, and complex tasks can go up to 4096.
    2. Treat empty responses as failures: empty content or whitespace-only content should trigger a retry, a channel switch, or an alert, not a direct write to the database.
    3. Support multiple response shapes: prefer choices[].message.content, then fall back to top-level message.content, content, text, and other fields.
    4. Unify the success check: don’t judge success by status code alone; use a unified function like is_success.
    1. def is_success(status, body):
    2. return status == 200 and bool(extract_text(body).strip())

    Pitfall 3: Rate Limiting Isn’t Boolean, It’s Probabilistic

    Symptom
    For the same free model, requests sometimes succeed in a row, and sometimes 6 out of 10 return 429.

    This isn’t random failure; it’s a common pattern of rate-limited free quotas. When total volume is small and intervals are short, requests may still get through; when requests come a bit denser, the token bucket drains and rejections begin.

    Root cause
    Free quotas often share a rate limiter with paid traffic. During peak hours, the volume of free calls is high, so your requests may land inside someone else’s peak and get throttled together. Different models have different popularity and therefore different degrees of sharing, so some models are stable while others throttle probabilistically.

    Fix

    1. Only retry retryable errors: 429, 5xx, and network timeouts are retryable; 400, 401, 403 and similar permission or quota errors usually should not be blindly retried.
    2. Respect Retry-After first: if the response header gives a suggested wait, use it first.
    3. Use exponential backoff with full jitter: to avoid multiple clients retrying in sync and creating a new peak.
    4. Control retries with a deadline: don’t look only at the retry count; also consider the remaining time budget.
    5. Rate-limit proactively on the client: the fallback channel especially must not dump all traffic at once when the primary model fails.

    The corrected retry function below reads the Retry-After response header and uses time.monotonic to compute the remaining deadline.

    1. import random
    2. import time
    3. RETRYABLE = {429, 500, 502, 503, 504}
    4. def get_retry_after(headers):
    5. for key, value in (headers or {}).items():
    6. if str(key).lower() == "retry-after":
    7. try:
    8. return max(0.0, float(value))
    9. except (TypeError, ValueError):
    10. return None
    11. return None
    12. def call_with_retry(
    13. model,
    14. messages,
    15. *,
    16. timeout=60,
    17. deadline=5.0,
    18. base=0.5,
    19. cap=8.0,
    20. min_tokens=1024,
    21. ):
    22. start = time.monotonic()
    23. attempt = 0
    24. while True:
    25. status, headers, body = post_json(
    26. f"{BASE_URL}/chat/completions",
    27. {
    28. "model": model,
    29. "messages": messages,
    30. "max_tokens": max(min_tokens, 512),
    31. "temperature": 0.3,
    32. },
    33. timeout=timeout,
    34. )
    35. if is_success(status, body):
    36. return status, body
    37. # Empty responses should not enter the backoff loop;
    38. # switch model, raise the budget, or alert instead
    39. if status == 200:
    40. return status, body
    41. if status not in RETRYABLE or attempt >= 3:
    42. return status, body
    43. delay = random.uniform(0, min(cap, base * (2 ** attempt)))
    44. if attempt == 0 and status == 429:
    45. retry_after = get_retry_after(headers)
    46. if retry_after is not None:
    47. delay = max(delay, min(retry_after, 30.0))
    48. elapsed = time.monotonic() - start
    49. if delay > deadline - elapsed:
    50. return status, body
    51. time.sleep(delay)
    52. attempt += 1

    Pitfall 4: Latency Is Bimodal, so Don’t Use an Average Timeout

    Symptom
    For the same free model, some requests return in 0.7 seconds, while others take 7 seconds or more. Reasoning models are even slower. Average latency looks okay, but P95 can be terrible.

    Root cause
    A free quota is essentially a priority slot in a shared resource pool. During peak hours, your request may queue behind paid requests with unpredictable wait times. This isn’t the model’s capability suddenly dropping; it’s tail latency from shared resources.

    Fix

    1. Set timeouts per category: general models 10 to 15 seconds; reasoning models 30 to 60 seconds; vision models adjusted by image size.
    2. Record P50 and P95: if P95 approaches the timeout threshold, the timeout is set too tight and may misclassify many successful requests as failures.
    3. Set a separate first-token timeout for streaming: in a shared pool, the first token may take a long time to arrive, so an overall 60-second timeout doesn’t guarantee a timely first token.
    1. timeout_policy = {
    2. "general": {"connect": 3, "first_byte": 12, "read": 15},
    3. "reasoning": {"connect": 5, "first_byte": 25, "read": 60},
    4. "vision": {"connect": 5, "first_byte": 20, "read": 30},
    5. }

    For streaming responses, it’s recommended to monitor time-to-first-token separately. The first-token timeout can be stricter than the overall read timeout, for example 10 to 15 seconds for the first token and 60 seconds for overall reading. This both avoids hanging for a long time and leaves slow reasoning enough time to complete.

    Pitfall 5: Inconsistent Response Structures Are a Landmine in Your Parsing Code

    Symptom
    Free-tier models may be maintained by different teams, so response structures aren’t fully consistent. In some, content is a string; in others, an array; some put reasoning content in a separate field; some return HTTP 200 with an error in the body when a failure occurs.

    If the caller writes only one parsing approach, a structural change causes silent failure: the program doesn’t crash, the logs show no error, but the business result is empty.

    Fix

    1. Have all callers depend on a unified extract_text and is_success; don’t each implement their own parsing.
    2. Write unit tests for empty content, array content, object content, missing choices, and similar cases.
    3. Add a separate “parse failure rate” or “empty response rate” metric to monitoring.
    4. When the parse failure rate rises abnormally, check whether the platform’s response structure has changed first, rather than blindly scaling up.

    5. Making a Free Quota a Reliable Fallback: Five Engineering Actions

    1. Capacity Planning: Free Quotas Absorb Failures, Not Peaks

    The principle of a fallback path is that the primary model handles normal traffic, and the free channel only takes over requests that failed on the primary model. To estimate the fallback peak, you can use:

    1. fallback peak QPS ≈ primary model failure rate × total request QPS

    If the estimate approaches the free quota’s rate-limit threshold, the free quota can’t hold up, and you should prepare a paid channel, caching, rule templates, or a static fallback.

    2. Path Design: Three Layers of Defense, Primary, Backup, Degrade

    1. request
    2. │
    3. ├─ primary model
    4. │ │ timeout / 429 / 5xx / empty response
    5. │ ▼
    6. ├─ free fallback channel
    7. │ │ rate limit / empty response / structural anomaly
    8. │ ▼
    9. └─ degradation strategy: cache hit / rule template / friendly message

    Record the reason for every failure at each layer. Recording only “failure” has no value; you need to record whether it was a timeout, 429, empty response, parse failure, or permission error. The distribution of reasons determines the direction of subsequent optimization: if 429s dominate, add rate limiting; if empty responses dominate, adjust the parameter budget; if timeouts dominate, switch models or relax the timeout.

    3. Client Changes: Unified Wrapper and Circuit Breaker

    Wrap retry, backoff, empty-response detection, and structure-compatible parsing into a unified calling function, such as call_with_retry. Business code should not assemble raw HTTP requests directly.

    Also add a circuit breaker: after N consecutive failures, open the circuit, and during the cooldown period route directly to the degradation path, to avoid continuing to hammer an unavailable channel.

    4. Observability: Turn Free-Quota Health into a Dashboard

    At minimum, monitor these metrics:

    • Call success rate, classified by status code and error type;
    • Empty response rate, grouped by model and max_tokens;
    • P50 and P95 latency;
    • 429 rate-limit rate;
    • Parse failure rate;
    • Remaining quota, if the platform provides a quota query endpoint.

    Set alerts: trigger one if the empty response rate exceeds 5% for 10 consecutive minutes, or the rate-limit rate exceeds 20%. This usually means the fallback channel has entered an unreliable state.

    5. Compliance and Data: The Hidden Cost of Free Quotas

    Free-quota terms may state that inputs and outputs will be used for service improvement or model training. This means sensitive data should not be sent directly to the free channel, including user privacy, business secrets, unpublished code, keys, ID numbers, phone numbers, and so on.

    If fallback requests may contain user input, desensitize or minimize them before entering the free channel. If the business is subject to personal-information-protection regulations, data export, retention, and usage terms need legal confirmation. Compliance isn’t a technical appendix; it’s a prerequisite for whether a free solution can run long-term.

    6. Parameter Budget Quick-Reference Table

    The table below is a general engineering starting point, not any platform’s official numbers. Actual values should follow the platform documentation and your probing results.

    Model category Suggested max_tokens starting point Read timeout starting point Key validation checks
    Lightweight general 512 to 1024 10 to 15 seconds Content non-empty, finish reason not length
    Reasoning model 2048+, complex tasks 4096 30 to 60 seconds Reasoning usage share, final content non-empty
    Vision understanding 1024, adjusted by image count 20 to 30 seconds Image format and size limits, content non-empty
    Vision reasoning 2048+ 30 to 60 seconds Reasoning usage, final content non-empty

    Special note: without image input, a vision model’s probe can only verify whether the text channel is available, not whether image understanding works. To verify visual capability, send an additional minimal image request and check the image input format and the base64 or URL requirements.

    7. Reproduction Script and Output Field Reference

    The script below is ready to run. It reads environment variables without hardcoding keys; probes models one by one; outputs structured results; and preserves the raw response on failure for easier debugging.

    1. import json
    2. import os
    3. import time
    4. from urllib.request import Request, urlopen
    5. from urllib.error import HTTPError, URLError
    6. BASE_URL = os.environ["LLM_BASE_URL"].rstrip("/")
    7. API_KEY = os.environ["LLM_API_KEY"]
    8. CANDIDATES = [
    9. ("<free general model A>", "general"),
    10. ("<free general model B>", "general"),
    11. ("<suspected free but possibly paid model>", "general"),
    12. ("<reasoning model>", "reasoning"),
    13. ("<vision understanding model>", "vision"),
    14. ("<vision reasoning model>", "vision"),
    15. ]
    16. def post_json(url, payload, timeout=60):
    17. req = Request(
    18. url,
    19. data=json.dumps(payload).encode("utf-8"),
    20. headers={
    21. "Content-Type": "application/json",
    22. "Authorization": f"Bearer {API_KEY}",
    23. },
    24. method="POST",
    25. )
    26. try:
    27. with urlopen(req, timeout=timeout) as resp:
    28. body = json.loads(resp.read().decode("utf-8") or "{}")
    29. return resp.status, dict(resp.headers.items()), body
    30. except HTTPError as e:
    31. raw = e.read().decode("utf-8", errors="replace")
    32. try:
    33. body = json.loads(raw)
    34. except Exception:
    35. body = {"raw": raw}
    36. return e.code, dict(e.headers.items()), body
    37. except URLError:
    38. return None, {}, {}
    39. def _content_to_text(content):
    40. if isinstance(content, str):
    41. return content
    42. if isinstance(content, list):
    43. return "".join(
    44. p.get("text", "")
    45. for p in content
    46. if isinstance(p, dict) and p.get("type", "text") == "text"
    47. )
    48. if isinstance(content, dict):
    49. for key in ("text", "content"):
    50. if isinstance(content.get(key), str):
    51. return content[key]
    52. return ""
    53. def extract_text(body):
    54. if not isinstance(body, dict):
    55. return ""
    56. choices = body.get("choices")
    57. if isinstance(choices, list):
    58. for choice in choices:
    59. if not isinstance(choice, dict):
    60. continue
    61. msg = choice.get("message")
    62. if isinstance(msg, dict):
    63. text = _content_to_text(msg.get("content"))
    64. if text.strip():
    65. return text
    66. text = _content_to_text(choice.get("text"))
    67. if text.strip():
    68. return text
    69. msg = body.get("message")
    70. if isinstance(msg, dict):
    71. text = _content_to_text(msg.get("content"))
    72. if text.strip():
    73. return text
    74. for key in ("content", "text", "output_text"):
    75. if key in body:
    76. text = _content_to_text(body.get(key))
    77. if text.strip():
    78. return text
    79. return ""
    80. def classify(status, body):
    81. if status == 200 and extract_text(body).strip():
    82. return "OK"
    83. if status == 200:
    84. return "OK_EMPTY"
    85. if status == 429:
    86. return "LIMITED"
    87. if status in (400, 401, 402, 403):
    88. return "NO_FREE_QUOTA"
    89. if status in (500, 502, 503, 504):
    90. return "SERVER_ERROR"
    91. return "ERROR"
    92. def probe(model, kind="general"):
    93. budgets = {
    94. "general": 512,
    95. "reasoning": 2048,
    96. "vision": 1024,
    97. }
    98. payload = {
    99. "model": model,
    100. "messages": [{"role": "user", "content": "Reply with one word: OK"}],
    101. "max_tokens": budgets.get(kind, 512),
    102. }
    103. t0 = time.monotonic()
    104. status, _, body = post_json(f"{BASE_URL}/chat/completions", payload)
    105. elapsed = round(time.monotonic() - t0, 2)
    106. result = classify(status, body)
    107. usage = body.get("usage", {}) if isinstance(body, dict) else {}
    108. return {
    109. "model": model,
    110. "result": result,
    111. "elapsed_s": elapsed,
    112. "usage": usage,
    113. }
    114. if __name__ == "__main__":
    115. rows = [probe(model, kind) for model, kind in CANDIDATES]
    116. print(json.dumps(rows, ensure_ascii=False, indent=2))

    The output fields are described below.

    Field Meaning Follow-up action
    OK Status 200 and content non-empty Can proceed to stability retesting
    OK_EMPTY Status 200 but content empty Check max_tokens, parsing logic, model type
    LIMITED 429 rate limited Reduce concurrency, add backoff, check Retry-After
    NO_FREE_QUOTA Quota, permission, or model unavailable Confirm the model ID and free policy, replace if needed
    SERVER_ERROR Platform-side 5xx Can retry briefly, switch channel if it persists
    ERROR Other anomaly Print the raw response and investigate separately

    Usage:

    1. Configure the LLM_BASE_URL and LLM_API_KEY environment variables; don’t commit keys to the repository.
    2. Replace the model IDs in CANDIDATES; copy them from the official documentation.
    3. Run the script and check the result field.
    4. Save the results as JSON and put them under version control. Re-run and diff after the platform updates.

    A single probe can only answer “does it work,” not “is it stable.” To test stability, add an outer loop to the script that calls each OK model 20 times in a row, then compute success rate, P50, P95, empty response rate, and 429 ratio. This result is closer to real production fallback capability than a single probe.

    8. Conclusion

    The pitfalls of free LLM APIs essentially come from the gap between what the documentation promises and what actually happens. The model list not showing free models is an API design choice; similar names but different billing is a naming and productization issue; empty responses without errors are a parameter-and-parsing compatibility issue; probabilistic 429s are the norm of a shared resource pool; and inconsistent response structures are the result of inconsistent governance of free-tier models.

    These gaps won’t disappear, but they can be covered by engineering means: probing, judging, backoff, circuit breaking, monitoring, and degradation. A free quota is not “free and usable” but “free to try.” If you treat it as a fallback, then test it, monitor it, and design a clear degradation path for it at fallback-level rigor.