Access LLM APIs on a Free Tier with a Protocol-Gateway: OpenAI-to-Anthropic Adapter in Practice
A practical guide to accessing large language model (LLM) APIs within free tiers or at low cost while keeping AI coding tools flexible. It explains how to decouple front-end coding tools from back-end models, choose inference platforms by free quota, capability, compatibility, rate limits, and compliance, and bridge the protocol gap between OpenAI-style and Anthropic-style APIs. The article includes a runnable Python gateway built on FastAPI and httpx that accepts Anthropic-style /v1/messages requests, converts them to OpenAI-style /chat/completions, and returns complete non-streaming and streaming event sequences. It also covers key security, free-tier compliance, backoff and retries, error mapping, secure deployment, observability, and a checklist of common integration mistakes.
1. Background: The Cost and Compatibility Problems of AI Coding Tools
AI coding assistants have become part of many developers’ daily workflow. Code completion, error explanation, refactoring suggestions, test generation, and cross-file edits all deliver meaningful productivity gains. At the same time, the cost of calling large model APIs puts pressure on individual developers and small teams.
A common misconception is that you must use a specific official coding tool, or bind yourself to one particular model service, to get the full experience. In reality, many tools support a custom backend model address. As long as you provide an interface that is compatible with a given protocol, you can decouple the front-end tool from the back-end model.
This article addresses two problems:
- How to obtain model inference resources within a free tier or at low cost.
- How to adapt model services with different protocol styles into an interface that common AI coding tools can recognize.
One point deserves special emphasis: this article discusses accessing within a free tier or at low cost, not a promise of zero cost. Free tiers, billing methods, rate-limit rules, and terms of service are all determined by the specific platform and change with its operating strategy. Always defer to the official documentation before use.
2. The Resource Layer: Choosing Free-Tier or Low-Cost Models
Most mainstream cloud providers and inference platforms offer OpenAI-style compatible APIs. Common configuration fields include:
base_url: The inference service address.api_key: The access key.model: The model name.
When choosing an inference platform, evaluate it across these dimensions:
| Evaluation Dimension | Recommendation | Notes |
|---|---|---|
| Free tier | Look for trials, free models, or time-limited free access | Free tiers are limited; long-term zero cost cannot be guaranteed |
| Model capability | Combine coding, reasoning, long-context, and math strengths | Different models excel at different tasks; there is no need to force a single model |
| API compatibility | Prefer OpenAI-style APIs | Higher compatibility means lower migration cost |
| Rate limiting | Check QPS, RPM, TPM, and concurrency | High-frequency scenarios should reserve room for backoff and retries |
| Stability | Check availability, response latency, and failure rate | Free resources may involve queuing or fluctuations |
| Compliance | Read the terms of service and avoid abusing the free tier | No high-frequency requests, no bypassing rate limits, no reselling |
For individual developers, a few practical strategies work well:
- Use the free tier for prototype validation, lightweight Q&A, and scripting assistance.
- Use low-cost, pay-as-you-go resources for everyday code completion.
- Configure “coding-oriented” models and “reasoning-oriented” models separately, and call them per scenario.
- Wrap different backends through a single gateway to avoid repeatedly changing configuration on the tool side.
3. The Protocol Layer: OpenAI Style vs. Anthropic Style
Once you have a model service address, configuring it directly into a coding tool often fails, and the core reason is that the API protocols are not unified. There are currently two common styles for large-model APIs: the OpenAI style and the Anthropic style.
| Aspect | OpenAI Style | Anthropic Style | What the Gateway Must Do |
|---|---|---|---|
| Request endpoint | Usually chat/completions |
Usually messages |
Forward inbound requests to the corresponding endpoint |
| System prompt | Inside the messages array, with the system role |
Usually a separate top-level field | Convert the top-level system into a system message |
| Message roles | May include system, user, assistant, etc. |
Usually only user and assistant |
Filter or convert illegal roles |
| Response body | choices[].message.content |
content array |
Map the response body back to the Anthropic structure |
| Stop reason | finish_reason |
stop_reason |
Perform field mapping |
| Streaming output | OpenAI-style SSE events | Anthropic-style event sequence | Convert events chunk by chunk and ensure complete termination |
Many excellent AI coding tools use the Anthropic-style interface. If all you have is an OpenAI-style model service, you can insert a protocol-conversion gateway in the middle:
- Outward-facing, it disguises itself as an Anthropic-style interface.
- Inward-facing, it converts requests to the OpenAI style.
- It reconverts the upstream response back to the Anthropic style before returning.
4. A Runnable Minimal Gateway: Converting an OpenAI Interface to an Anthropic Interface
The following is a runnable minimal Python example based on FastAPI and httpx. It can:
- Receive Anthropic-style
POST /v1/messagesrequests. - Convert requests into OpenAI-style
POST /chat/completions. - Support non-streaming responses.
- Support streaming responses and emit a complete Anthropic event sequence.
- Support simple gateway authentication.
4.1 Dependencies
Python 3.10 or later is recommended.
pip install fastapi uvicorn httpx
4.2 Environment Variable Configuration
export UPSTREAM_BASE_URL="https://your-provider.example/v1"export UPSTREAM_API_KEY="sk-your-upstream-key"export TARGET_MODEL="your-model-name"export GATEWAY_API_KEY="local-gateway-key"
Notes:
UPSTREAM_BASE_URL: The upstream OpenAI-style service address, which usually includes/v1.UPSTREAM_API_KEY: The upstream model service key.TARGET_MODEL: The model name the gateway ultimately uses when forwarding to the upstream.GATEWAY_API_KEY: The gateway’s own authentication key, used to prevent arbitrary calls to the local port.
4.3 Gateway Code
import jsonimport osimport uuidfrom typing import Any, AsyncGeneratorimport httpxfrom fastapi import FastAPI, Requestfrom fastapi.responses import JSONResponse, StreamingResponseapp = FastAPI()UPSTREAM_BASE_URL = os.getenv("UPSTREAM_BASE_URL", "").rstrip("/")UPSTREAM_API_KEY = os.getenv("UPSTREAM_API_KEY", "")TARGET_MODEL = os.getenv("TARGET_MODEL", "default-model")GATEWAY_API_KEY = os.getenv("GATEWAY_API_KEY", "")def auth_ok(request: Request) -> bool:if not GATEWAY_API_KEY:return Trueif request.headers.get("x-api-key") == GATEWAY_API_KEY:return Trueauth = request.headers.get("authorization", "")return auth == f"Bearer {GATEWAY_API_KEY}"def text_from_content(content: Any) -> str:if isinstance(content, str):return contentif isinstance(content, list):parts = []for block in content:if isinstance(block, dict) and block.get("type") == "text":parts.append(block.get("text", ""))elif isinstance(block, str):parts.append(block)return "\n".join(parts)return str(content)def to_openai_payload(req: dict[str, Any]) -> dict[str, Any]:messages = []system = req.get("system")if system:messages.append({"role": "system","content": text_from_content(system)})for msg in req.get("messages", []):role = msg.get("role")if role not in {"user", "assistant"}:continuemessages.append({"role": role,"content": text_from_content(msg.get("content", ""))})payload = {"model": TARGET_MODEL,"messages": messages,"stream": bool(req.get("stream", False)),"max_tokens": int(req.get("max_tokens", 4096)),}if "temperature" in req:payload["temperature"] = req["temperature"]if "top_p" in req:payload["top_p"] = req["top_p"]return payloaddef map_stop_reason(reason: str | None) -> str:return {"stop": "end_turn","length": "max_tokens","tool_calls": "tool_use","content_filter": "refusal",}.get(reason or "", "end_turn")def sse(event: str, obj: dict[str, Any]) -> str:return f"event: {event}\ndata: {json.dumps(obj, ensure_ascii=False)}\n\n"@app.post("/v1/messages")async def messages(request: Request):if not auth_ok(request):return JSONResponse(status_code=401,content={"type": "error","error": {"type": "authentication_error","message": "invalid x-api-key"}})if not UPSTREAM_BASE_URL or not UPSTREAM_API_KEY:return JSONResponse(status_code=500,content={"type": "error","error": {"type": "configuration_error","message": "UPSTREAM_BASE_URL or UPSTREAM_API_KEY missing"}})req = await request.json()openai_payload = to_openai_payload(req)if not openai_payload["stream"]:async with httpx.AsyncClient(timeout=httpx.Timeout(90)) as client:resp = await client.post(f"{UPSTREAM_BASE_URL}/chat/completions",headers={"Authorization": f"Bearer {UPSTREAM_API_KEY}"},json=openai_payload,)if resp.status_code >= 400:return JSONResponse(status_code=resp.status_code,content={"type": "error","error": {"type": "upstream_error","message": resp.text}})data = resp.json()choices = data.get("choices") or [{}]choice = choices[0]message = choice.get("message", {})usage = data.get("usage", {})return {"id": data.get("id", f"msg_{uuid.uuid4().hex}"),"type": "message","role": "assistant","model": data.get("model", TARGET_MODEL),"content": [{"type": "text","text": message.get("content", "")}],"stop_reason": map_stop_reason(choice.get("finish_reason")),"stop_sequence": None,"usage": {"input_tokens": usage.get("prompt_tokens", 0),"output_tokens": usage.get("completion_tokens", 0),}}async def stream() -> AsyncGenerator[str, None]:msg_id = f"msg_{uuid.uuid4().hex}"try:async with httpx.AsyncClient(timeout=httpx.Timeout(120)) as client:async with client.stream("POST",f"{UPSTREAM_BASE_URL}/chat/completions",headers={"Authorization": f"Bearer {UPSTREAM_API_KEY}"},json=openai_payload,) as up:if up.status_code >= 400:body = await up.aread()yield sse("error", {"type": "error","error": {"type": "upstream_error","message": body.decode("utf-8", "ignore")}})yield sse("message_stop", {"type": "message_stop"})returnyield sse("message_start", {"type": "message_start","message": {"id": msg_id,"type": "message","role": "assistant","model": TARGET_MODEL,"content": [],"stop_reason": None,"stop_sequence": None,"usage": {"input_tokens": 0,"output_tokens": 0,}}})yield sse("content_block_start", {"type": "content_block_start","index": 0,"content_block": {"type": "text","text": ""}})finish_reason = Noneoutput_chars = 0async for line in up.aiter_lines():if not line.startswith("data:"):continuechunk = line[5:].strip()if chunk == "[DONE]":breaktry:obj = json.loads(chunk)except json.JSONDecodeError:continuechoices = obj.get("choices") or []if not choices:continuechoice = choices[0]delta = choice.get("delta") or {}piece = delta.get("content")if piece:output_chars += len(piece)yield sse("content_block_delta", {"type": "content_block_delta","index": 0,"delta": {"type": "text_delta","text": piece}})if choice.get("finish_reason"):finish_reason = choice.get("finish_reason")breakyield sse("content_block_stop", {"type": "content_block_stop","index": 0})yield sse("message_delta", {"type": "message_delta","delta": {"stop_reason": map_stop_reason(finish_reason),"stop_sequence": None},"usage": {"output_tokens": max(1, output_chars // 4)}})yield sse("message_stop", {"type": "message_stop"})except Exception as e:yield sse("error", {"type": "error","error": {"type": "api_error","message": str(e)}})yield sse("message_stop", {"type": "message_stop"})return StreamingResponse(stream(), media_type="text/event-stream")
4.4 Starting the Gateway
uvicorn app:app --host 127.0.0.1 --port 8080
By default it listens only on the local machine at 127.0.0.1, which is the safer approach. If you genuinely need LAN access, bind to an internal network address instead, and pair it with a reverse proxy, TLS, and authentication.
4.5 Non-Streaming Call Example
curl -X POST http://127.0.0.1:8080/v1/messages \-H "content-type: application/json" \-H "x-api-key: local-gateway-key" \-d '{"model": "claude-style","max_tokens": 512,"system": "You are a rigorous programming assistant.","messages": [{"role": "user","content": "Write a quicksort in Python."}]}'
4.6 Streaming Call Example
curl -N -X POST http://127.0.0.1:8080/v1/messages \-H "content-type: application/json" \-H "x-api-key: local-gateway-key" \-d '{"model": "claude-style","max_tokens": 512,"stream": true,"system": "You are a rigorous programming assistant.","messages": [{"role": "user","content": "Explain HTTP status code 429."}]}'
4.7 AI Coding Tool Configuration Example
If the tool asks for an Anthropic-style interface address, configure it like this:
Base URL: http://127.0.0.1:8080/v1API Key: local-gateway-keyModel: any recognizable name; the final model is determined by TARGET_MODEL in the gateway
If the tool asks for the full interface path, you can also enter:
http://127.0.0.1:8080/v1/messages
The integration steps can be summarized as follows:
- Configure the upstream
UPSTREAM_BASE_URL,UPSTREAM_API_KEY, andTARGET_MODEL. - Start the local gateway, which listens on
127.0.0.1:8080by default. - Enter the gateway address and
GATEWAY_API_KEYin the coding tool. - Test the non-streaming call first, then the streaming call.
- After confirming the tool displays output correctly, connect it to your real development workflow.
5. Streaming Responses Must Cover the Complete Event Sequence
Many compatibility problems do not appear in ordinary requests but in streaming output. If the gateway converts only part of the content and does not send the complete event sequence, the front end may exhibit issues such as “stuck loading,” “only the beginning is shown,” or “the session is never closed after finishing.”
A minimal complete Anthropic-style streaming event sequence should usually include:
message_startcontent_block_startcontent_block_deltacontent_block_stopmessage_deltamessage_stop
If the upstream encounters an exception, you should still emit an error event and follow it with message_stop, so the front end does not wait forever.
Common streaming problems include:
- Failing to pass
stream: true, so the gateway never enters the streaming branch. - Not converting
delta.content, so the front end receives no text increments. - Not sending
message_stop, so the tool cannot finish cleanly. - Not mapping
finish_reasontostop_reason, so the tool’s state becomes abnormal. - A missing or malformed
usage, so token accounting is inaccurate.
6. Engineering Best Practices
A self-hosted gateway can solve compatibility problems at low cost, but long-term stability requires filling in the engineering capabilities.
6.1 Key Security
- The upstream
api_keymust be injected through environment variables, a key management system, or a local configuration center. - Never hardcode keys in the code.
- Never commit keys to a Git repository.
- Do not print full keys in logs; keep only the first and last few characters.
- Set an expiry for keys and support rotation.
6.2 Compliant Use of Free Tiers
Free tiers and trial resources must strictly comply with the platform’s terms of service. Recommendations:
- Do not bypass rate limits or run high-frequency bulk requests.
- Do not use the free tier for commercial resale or large-scale proxying.
- Do not keep retrying after the quota is exhausted, which creates useless load.
- Set call caps and a local budget for free resources.
- Prioritize prototype validation, development debugging, and low-frequency tasks.
6.3 Rate Limiting and Retries
For upstream rate limits, we recommend:
- Applying exponential backoff on
429. - Starting waits of 1, 2, and 4 seconds.
- Setting a maximum of 3 to 5 retries.
- Setting request timeouts to avoid connections hanging indefinitely.
- Adding a local queue or cache for high-frequency tools.
A simple backoff strategy could be:
1st failure: wait 1 second2nd failure: wait 2 seconds3rd failure: wait 4 seconds4th failure: wait 8 secondsBeyond the limit: return a diagnosable error
6.4 Error Mapping
The gateway should convert upstream errors into error structures the front end can understand. Common mappings include:
| Upstream Situation | Recommended Gateway Behavior |
|---|---|
| Authentication failure | Return authentication_error |
| Quota exhausted | Return rate_limit_error or quota_exceeded |
| Model not found | Return not_found_error |
| Content too long | Return invalid_request_error |
| Upstream timeout | Return timeout_error and preserve trace information |
Error responses should preserve as much diagnosable context as possible, such as request ID, elapsed time, and error type, but must not leak keys or sensitive content.
6.5 Secure Deployment
If the gateway is used only locally, bind to 127.0.0.1 by default. If other devices need to access it, we recommend:
- Using a reverse proxy such as Nginx or Caddy.
- Enabling HTTPS or TLS.
- Enabling gateway authentication, such as
x-api-key. - Restricting accessible IPs or internal network ranges.
- Applying a least-privilege policy to upstream keys.
- Turning off debug logs before exposing the service externally.
6.6 Observability
At a minimum, record these metrics:
- Total request count.
- Successful request count.
- Failed request count.
- Number of upstream
429s. - P95 latency.
- Average time to first byte.
- Average streaming duration.
- Remaining upstream quota.
Simple alert thresholds could reference:
- Alert when the failure rate over 5 minutes exceeds 5%.
- Alert when the
429ratio over 5 minutes exceeds 10%. - Investigate the upstream or network when P95 latency exceeds 15 seconds.
- Check the upstream connection when a streaming request sends no data for more than 60 seconds.
7. Common Errors Checklist
The following issues are extremely frequent in practice:
Misplaced
systemfield
Not converting the Anthropic top-levelsysteminto an OpenAIsystemmessage, causing the model to ignore the system prompt.Missing
streamflag
The front end requests streaming, but the gateway does not enablestreamwhen building the payload, so only a single response is received.Incorrect
finish_reasonmapping
OpenAI’sfinish_reasonis not converted to Anthropic’sstop_reason, causing abnormal tool behavior.Missing
usagefields
The gateway does not returninput_tokensandoutput_tokens, so some tools cannot count tokens.Missing
message_stop
No termination event is sent after streaming ends, so the front end may wait indefinitely.Insufficient multimodal content handling
If a request contains complex content blocks such as images or documents, you need to extract the text separately or fall back on formatting.Model name mismatch
The model name entered in the tool does not match the model actually supported by the upstream, resulting inmodel not found.Misunderstanding the free tier
Assuming the free tier can be used without limit, ignoring the platform’s rate limits, terms of service, or account restrictions.
8. Summary
The key to accessing LLM APIs at low cost is not to bind yourself to a single tool or a fixed model, but to build a decoupled “resource layer + adapter layer” design:
- The resource layer is responsible for choosing model services within a free tier, at low cost, or on a pay-as-you-go basis.
- The adapter layer smooths over the interface differences between the OpenAI style and the Anthropic style through a protocol-conversion gateway.
The value of this approach lies in being reusable, portable, and extensible:
- Switching models requires only changing the gateway configuration.
- Connecting a new tool requires only pointing it at the gateway.
- Any upstream service compatible with the OpenAI style can be brought under unified management.
- A local gateway reduces the risk of key exposure and makes it easier to add authentication, rate limiting, and logging.
For individual developers and small teams, as long as you control the boundaries of the free tier, comply with the platform’s terms of service, and complete the streaming conversion and error handling, you can build a stable, maintainable, and reusable AI coding integration setup at low cost.
