Connecting Free Cloud LLMs to Coding Tools via a Local Proxy
If a coding tool lets you set a custom API base URL, you can route it to free cloud models through a local proxy gateway. This guide covers model choice, when a gateway is needed, credential checks, a FastAPI proxy, client field mapping, and a curl connectivity test.
1. Takeaway first
If a coding tool allows a custom endpoint, you can point it at free models from a cloud platform. In most cases a local proxy gateway is enough to bridge the request. Free quotas have expiry dates and usage caps. Before you rely on them, confirm the terms in the console.
2. Choosing between two model types
The platform currently offers two model families with different strengths.
Reasoning models emphasize logic and math. They are useful for algorithms, edge cases, and pseudocode before you write production code. Platform docs claim math-reasoning accuracy above 95%. That figure is vendor-reported and not independently verified here; trust your own tests.
Mixture-of-experts (MoE) models have about 230 billion total parameters and activate about 10 billion per inference. The design trades lower per-request compute for larger capacity, which often fits multi-file edits, context understanding, and debugging from error messages. Parameter counts also come from platform materials; use the official model card as the source of truth.
| Dimension | Reasoning model | MoE model |
|---|---|---|
| Best for | Algorithm design, requirement breakdown, boundary analysis | Multi-file edits, error repair, code review |
| Suggested use | Think through the approach before coding | Coding and refactoring |
| What to measure | Latency, output length | Long-context stability, output quality |
3. Decide whether you need a gateway
If the tool supports an OpenAI-compatible base URL and arbitrary model names, fill in the platform URL and key directly. Add a proxy only when the tool requires a fixed request format, extra headers, or routing by task to different models. A proxy adds latency and needs maintenance.
4. Checks before you create credentials
Sign in to the platform console, create an API key, and record each model’s base URL and model ID. Before production use, verify in the console or terms of service:
- Free-quota validity period and total volume
- Rate limits (requests per minute and concurrency)
- Commercial use and any identity-verification requirements
- Whether the free policy can change
“Zero cost” in this article means cost inside the free quota. Fees and throttling after the quota are defined by the platform.
5. Deploy the proxy gateway
Install dependencies:
pip install fastapi uvicorn httpx
Gateway code below. All secrets come from environment variables; the source contains no usable keys.
import osimport httpxfrom fastapi import FastAPI, HTTPException, Requestfrom fastapi.responses import ResponseUPSTREAM_BASE_URL = os.environ["UPSTREAM_BASE_URL"].rstrip("/")PROXY_TOKEN = os.environ["PROXY_TOKEN"]# Client-facing model aliases -> upstream model IDs and keysMODEL_ROUTES = {"reasoning": {"model": os.environ["REASONING_MODEL_ID"],"api_key": os.environ["REASONING_API_KEY"],},"moe": {"model": os.environ["MOE_MODEL_ID"],"api_key": os.environ["MOE_API_KEY"],},}app = FastAPI()@app.post("/v1/chat/completions")async def chat_completions(request: Request):if request.headers.get("authorization") != f"Bearer {PROXY_TOKEN}":raise HTTPException(status_code=401, detail="invalid proxy token")data = await request.json()alias = data.get("model", "moe")route = MODEL_ROUTES.get(alias)if route is None:raise HTTPException(status_code=400, detail=f"unknown model alias: {alias}")# Replace the alias with the real upstream model ID and use that model's keydata["model"] = route["model"]headers = {"Authorization": f"Bearer {route['api_key']}","Content-Type": "application/json",}try:async with httpx.AsyncClient(timeout=60.0) as client:upstream = await client.post(f"{UPSTREAM_BASE_URL}/chat/completions",json=data,headers=headers,)except httpx.TimeoutException:raise HTTPException(status_code=504, detail="upstream timeout")except httpx.HTTPError:raise HTTPException(status_code=502, detail="upstream unreachable")# Pass through upstream status and body so clients see real errorsreturn Response(content=upstream.content,status_code=upstream.status_code,media_type=upstream.headers.get("content-type", "application/json"),)
Set environment variables and start the server. Replace values with what you obtained from the console:
export UPSTREAM_BASE_URL="https://platform-provided-address/v1"export PROXY_TOKEN="a random string you generate for local auth"export REASONING_MODEL_ID="real reasoning model ID"export REASONING_API_KEY="cloud key for the reasoning model"export MOE_MODEL_ID="real MoE model ID"export MOE_API_KEY="cloud key for the MoE model"uvicorn gateway:app --host 127.0.0.1 --port 8000
Limitation: this code does not stream. If the client defaults to stream=true, turn streaming off first. For streaming, switch to FastAPI StreamingResponse and forward chunks.
6. Client settings and field mapping
In the tool’s model settings, choose a custom provider or OpenAI-compatible mode and fill in the table below.
| Client field | Value | Gateway behavior | Upstream field |
|---|---|---|---|
| Base URL | http://127.0.0.1:8000/v1 | Accepts the request | Platform UPSTREAM_BASE_URL |
| API Key | value of PROXY_TOKEN |
Validates the local token | Not exposed to the client |
| Model name | reasoning or moe |
Looks up the real model ID | REASONING_MODEL_ID or MOE_MODEL_ID |
The client must send aliases, not upstream model IDs. Alias-to-ID mapping lives only in the gateway.
7. Verify with curl
Before wiring the coding tool, confirm the gateway forwards correctly:
curl http://127.0.0.1:8000/v1/chat/completions \-H "Authorization: Bearer $PROXY_TOKEN" \-H "Content-Type: application/json" \-d '{"model":"moe","messages":[{"role":"user","content":"Explain in one sentence"}]}'
