How to Connect Free Model APIs to Coding Tools and Chat Clients

    Many model service platforms now offer free tiers, but plugging them into AI tools is still tricky because API conventions vary widely—request and response formats, streaming events, tool-calling protocols, and even how API keys are passed all differ. This article presents a protocol-adapter approach: insert a lightweight local proxy between your tools and the model service to handle request and response transformation, authentication adaptation, and routing. It walks through obtaining credentials, deploying and configuring the proxy, connecting both graphical chat clients and CLI coding agents without touching client code, and then refining the setup with header hints, function-call anti-hallucination, routing, and retries. It also covers security and production considerations, with the overarching goal of making model migration a configuration-level change rather than a code change.

    Background: Model services keep growing, but integration still has barriers

    Over the past two years, the supply of large-model inference services has grown noticeably faster. To promote new models and collect feedback from real-world scenarios, many model service platforms now open up a certain quota of free trial resources to developers. This is good news for developers: without buying your own GPUs or deploying an inference framework, you can validate the capabilities of frontier models in real business scenarios.

    At the same time, developer-facing AI tools have split into two typical forms:

    • Graphical multi-model chat clients: With friendly interfaces, they make it easy to manage conversations and prompt templates, and are well suited to Q&A, solution discussions, and code snippet generation.
    • CLI-based agentic coding tools: These run inside the terminal, can read and write workspace files and execute commands, and autonomously complete tasks in a loop of “read code → modify files → run tests → fix issues based on errors”, which is closer to real engineering scenarios.

    Both types of tools communicate with model services over HTTP, but the interface conventions are not unified. Some model service platforms offer interfaces compatible with the industry’s de facto standard chat-completions style, so the client only needs to change the address to use them; others use custom request formats, with differences in field organization, streaming event structure, and tool-calling protocols. If you also factor in whether the API key goes in a request header or in a query parameter, the problem gets even more complex. In addition, most clients have only a few providers built in, so when a model is retired or its quota changes, the user experience is interrupted as well.

    Therefore, this article is not about how to use a specific product, but about a more general question: how to connect model services to both chat clients and CLI agents through a lightweight local protocol-adaptation layer, without modifying the client code, so that the integration can be smoothly migrated to other services. The technical approach described here also applies to integrating model services offered by enterprise-grade platforms such as Baidu AI Cloud.

    The overall approach: decouple the client from the model service with a protocol adapter

    The core idea is to insert a protocol-adapter proxy between the client and the model service. You can think of it as a “local gateway”:

    1. Desktop chat client ┐
    2. ├──→ Local protocol adapter ──→ Model service platform
    3. CLI coding agent ──┘ (format conversion / auth / routing / streaming)

    The proxy needs to take on four types of responsibilities:

    1. Request transformation: Convert requests sent by the client into a format the upstream model service can recognize.
    2. Response transformation: Restore the streaming or non-streaming responses returned by the upstream into the structure the client expects.
    3. Authentication adaptation: Automatically attach credentials to requests sent upstream, so the client only needs to hold a local token.
    4. Routing and governance: Handle model alias mapping, timeout control, retries, and logging.

    The direct benefit of this layering is: you configure the client side only once, and the model service can be swapped out on demand; the proxy handles the differences centrally, so upgrades are confined to a single place. In practice, you can also start from a mature open-source adapter gateway. When choosing one, focus on three points: whether it supports streaming forwarding and multiple upstream configurations, whether the community is active, and whether the transformation logic can be easily extended.

    2.1 Clarify the request, response, and authentication differences

    Before you start, first map out the differences between the two sides so that the transformation logic has a basis. Typical differences are shown in the table below:

    Dimension chat completions style (common model services) Custom message format (some CLI agents)
    System prompt A system-role message inside the messages array A separate system field
    Multimodal content content is text or an array of content blocks content is an array of content blocks, with different type names
    Streaming end marker Completion marked by a special “end” block A separate end event that also carries usage statistics
    Reasoning process Usually returned in extended fields such as reasoning_content Returned as a thinking-type content block, with a signature
    Tool calls tools array + tool_calls + tool-role message tools array + tool_use / tool_result content blocks
    Authentication Mostly the Authorization request header Supports custom header names and query parameters

    Understanding these differences also makes it clear why an adaptation layer is needed: transformation is essentially “field renaming, structure restructuring, and event re-wrapping,” and does not involve the model itself.

    Preparation: obtain access credentials for the model service

    Taking a platform’s open free model resources as an example, obtaining credentials usually takes four steps.

    3.1 Register and log in

    Visit the platform’s model experience page, register an account, and log in. Many platforms allow quick login with third-party accounts, and the process usually takes just a few minutes.

    3.2 Pick any model and get the call details

    Pick any model from the model list, and after entering its details page, find an entry like “View code”. The page usually provides call examples in multiple languages, for example:

    1. client = OpenAI(
    2. base_url="<Your Base URL>",
    3. api_key="<Your API Key>"
    4. )
    5. response = client.chat.completions.create(
    6. model="<Model Name>",
    7. messages=[{"role": "user", "content": "Write a poem"}],
    8. stream=True,
    9. )

    The example usually contains two core configuration items:

    • base_url: the root address of the service’s API;
    • api_key: the access key issued by the platform.

    The key on the page is hidden by default. After clicking a button like “Generate API Key”, you can click the eye icon to reveal it, then copy it to a safe place. Since this is a newly generated key, previously issued keys are not affected.

    3.3 Find the model identifier

    The model identifier can usually be found in the model list provided on the page. For example, to call another model, you only need to replace the model name in the example with the corresponding identifier from the list.

    3.4 Why picking any model still gives you credentials

    Many people wonder: if the goal is model A and model B, why open the example page of a third model? The reason is that the key generated by the platform is usually an account-level credential, not bound to a specific model. After logging in, you can pick any model, enter its example page, and complete the “enter example page → generate key” operation. Once you have the base_url and api_key, you can replace the model identifier as needed.

    3.5 The three essentials of a request

    Element Purpose Where to configure
    base_url Specifies which service the request is sent to The upstream configuration of the adapter proxy
    api_key Proves the caller’s identity and links to the quota The adapter proxy or the client, preferably injected via environment variables
    model Specifies which model to call The client, the adapter proxy, or the request body

    Core steps: deploy the protocol-adapter proxy

    4.1 Choosing a tool

    You can choose a mature API-adapter gateway from the open-source community. Key selection points are as follows:

    • Whether it supports bidirectional conversion between OpenAI-compatible interfaces and mainstream CLI agent formats;
    • Whether it supports streaming responses, tool calls, and image input;
    • Whether it supports multiple upstream configurations and routing via request headers or query parameters;
    • Whether it offers container images and common installation methods for easy deployment and migration.

    4.2 Installation

    Take container-based deployment as an example:

    1. # Copy the example config file to the actual config file, then modify it as needed
    2. cp config.example.yaml config.yaml
    3. # Start the container; map host port 9527 to container port 8317
    4. docker run -d --name api-proxy \
    5. --restart unless-stopped \
    6. -p 9527:8317 \
    7. -v "$(pwd)/config.yaml:/CLIProxyAPI/config.yaml" \
    8. <image-address>:<version-tag>

    Without a container environment, you can also directly download the binary for your platform and specify the config file via command-line arguments when starting.

    4.3 Configuration

    The following is a sample configuration for connecting to an upstream model service. The address, key, and model identifier must be replaced with the actual values obtained from the model service platform:

    1. # Basic configuration of the proxy itself
    2. port: 8317 # Port the proxy listens on
    3. auth-dir: "~/.cli-proxy-api" # Local auth file directory
    4. # Local access token: replace with a high-strength random string
    5. api-keys:
    6. - "Replace with a high-strength random key"
    7. # Log level: debug / info / warn / error
    8. debug: false
    9. # Compatible interface configuration used to connect to the model service platform
    10. openai-compatibility:
    11. - name: model-platform # Upstream name, for identification only
    12. base-url: "https://<your-api-root-address>"
    13. api-key-entries:
    14. - api-key: "<your-platform-key>"
    15. models:
    16. # One upstream name can map to multiple models for on-demand switching
    17. - name: "<model-identifier-A>"
    18. alias: "coding-model"
    19. - name: "<model-identifier-B>"
    20. alias: "reasoning-model"

    Two key designs are as follows:

    • Model aliases: alias is the model name the proxy exposes externally, and the client only needs to be aware of the alias. When the upstream model identifier changes, the change is confined to the config file.
    • Multiple-key rotation: api-key-entries supports configuring multiple keys, which can be automatically switched when rate limits are hit, making it suitable for multi-account or multi-project scenarios.

    If a client requires the model identifier to be part of the URL, you can also use the platform’s “model prefix” configuration so that the model identifier is reflected directly in the routing.

    4.4 Verify connectivity

    After the proxy starts, first verify the link with a minimal request:

    1. curl http://127.0.0.1:9527/v1/chat/completions \
    2. -H "Authorization: Bearer <local-access-token>" \
    3. -H "Content-Type: application/json" \
    4. -d '{
    5. "model": "coding-model",
    6. "messages": [{"role": "user", "content": "Introduce yourself in one sentence"}],
    7. "stream": true
    8. }'

    If streaming content is returned normally, it means the “proxy → upstream” link is working. Next, you just need to point the client to the proxy.

    Connecting a desktop multi-model chat client

    Create a new custom provider in the chat client, and fill in the following three items:

    1. API address: Fill in the local proxy address, such as http://127.0.0.1:9527;
    2. API key: Fill in the local access token from the proxy configuration;
    3. Model: Choose the alias coding-model or reasoning-model.

    After saving, send a message to verify. If you need multimodal capability, you can paste an image directly to test: the proxy will convert the image into the OpenAI-compatible format—an image_url-type content block carrying base64 encoding—and forward it to the upstream.

    If you cannot connect, troubleshoot in the following order:

    • Whether the proxy process is running properly and whether the port is occupied;
    • Whether the client mistakenly filled in a model identifier instead of an alias;
    • Whether the local access token matches the configuration.

    Connecting CLI agentic coding tools

    6.1 The idea: keep the entry point, swap the backend

    Some CLI agents support model services from a specific vendor only by default. To make them use other models, the idea is “keep the entry point unchanged, replace the backend”: keep the client’s existing configuration untouched, and just point the model requests to the local proxy. This requires no modification to the client source code and does not rely on any unofficial patches.

    6.2 Configuration

    macOS / Linux:

    1. # Point to the local proxy
    2. export ANTHROPIC_BASE_URL=http://127.0.0.1:9527
    3. # Fill in the local access token from the proxy configuration
    4. export ANTHROPIC_AUTH_TOKEN=<local-access-token>
    5. # Specify the model alias
    6. export ANTHROPIC_MODEL=coding-model
    7. # Takes effect only for the current terminal session; write it to the shell profile to persist

    Windows (PowerShell):

    1. # Run as administrator to set user-level persistent environment variables
    2. [Environment]::SetEnvironmentVariable(
    3. "ANTHROPIC_BASE_URL", "http://127.0.0.1:9527", "User"
    4. )
    5. [Environment]::SetEnvironmentVariable(
    6. "ANTHROPIC_AUTH_TOKEN", "<local-access-token>", "User"
    7. )
    8. [Environment]::SetEnvironmentVariable(
    9. "ANTHROPIC_MODEL", "coding-model", "User"
    10. )
    11. # Takes effect after reopening the terminal

    Note two points:

    • Do not use ANTHROPIC_API_KEY. When some tools encounter this variable, they switch to a mode that connects directly to the official interface, which bypasses the proxy.
    • Set request-header authentication in the proxy, disguising the token in the form of the official interface’s auth header, to accommodate the token-validation habits of different clients.

    6.3 Test it on a coding task

    Enter any project directory, start the CLI agent, and give it a task close to real development, for example:

    Read this project and explain how to start it; add boundary-value unit tests for the utility classes, and run the tests to make sure they all pass.

    Observe whether the agent can correctly read files, propose a modification plan, execute commands, and self-correct based on errors. In this process, what is really being tested is whether the proxy’s transformation of the tool-calling protocol is complete.

    6.4 Extra adaptation for Mixture-of-Experts (MoE) models

    Many open-source models with strong coding ability currently use a Mixture-of-Experts (MoE) architecture. Such models distribute parameters across multiple “expert” sub-networks, and each token activates only a small subset of them. The effect is that the total parameter count can be very large to hold rich knowledge, while the compute per inference stays relatively manageable. The trade-off is that VRAM usage is usually close to the total parameter count, so they mostly rely on centralized cloud deployment and are hard to run on consumer-grade local GPUs.

    For agents, there is another detail that must be handled: the transformation between the proxy and the upstream is bidirectional.

    • Downstream: The reasoning content (reasoning_content) returned by the upstream needs to be converted into thinking content blocks. With thinking adaptive enabled, the way it is returned can be automatically adjusted based on whether the thinking block carries a signature.
    • Upstream: The thinking blocks in historical messages need to be restored to the reasoning_content field, and file paths referenced in the content also need to be filled in from the tool results, so the model does not “fail to see” its own earlier reasoning and repeat its work.

    Some platforms’ compatible interfaces disable reasoning-content output by default, and you also need mechanisms such as “header hints” to enable it in the way the compatible interface expects.

    Going further: make the adapter more robust and usable

    7.1 Header hints: a small mechanism that solves big problems

    The implementations of compatible layers differ across platforms. Taking reasoning content as an example, some require the model identifier to carry a specific suffix, while others require attaching a specific request header. The proxy can maintain a “header hint” mapping table that binds a combination of header names and expected values to a model identifier:

    1. request-header-hints:
    2. suffix: ":<reasoning-identifier-suffix>" # Automatically append a suffix to the matching model identifier
    3. headers:
    4. X-<hint-header-name>: "<hint-value>" # Request header automatically attached when matched
    5. models:
    6. - "<model-identifier>"

    If the same upstream has both models with a reasoning-identifier suffix and models without it, you can split them into two separate upstreams and configure them individually.

    7.2 Function-call anti-hallucination: countering model hallucinations

    Whether an agent can reliably complete tasks depends largely on whether tool calls are well-formed. Three kinds of “hallucination” are commonly seen in practice:

    1. Generating non-existent tool names;
    2. Omitting required parameters;
    3. Writing a tool call as plain text.

    At this point, you can configure “function-call anti-hallucination” (func-antibody) on the proxy side. Once enabled, the proxy dynamically checks the tool list and historical messages:

    • On the first request, inject a prompt before the user message, reminding the model that it may only use the given tools, must provide required parameters, and must not fabricate calls in text.
    • If tool calls already exist in the historical messages, do not inject again, to avoid wasting context.
    Option Suggested value Purpose
    antibody-mode enforce Enforcement mode, enabled by default
    max-history-block 6 Do not inject into the most recent N messages, to avoid duplication
    inject-at user Inject into the user message rather than the system message, compatible with interfaces that do not support system messages
    compact false Whether to simplify the injected content

    7.3 Routing: dispatch different tasks to different models

    Different models excel in different areas. Some lean toward code generation and multi-file refactoring, while others lean toward math and logical reasoning. The proxy can route based on request characteristics:

    • Routing types support two modes: “by model” and “by request header”.
    • Strategies support round-robin, random, by remaining quota, and so on.
    • When routing by request header, if multiple matching items exist, the first matching rule in configuration order is taken.

    Configuration example (illustrative):

    1. routing:
    2. strategy: round-robin # Round-robin strategy
    3. models:
    4. - name: coding-model # Model used for routing decisions
    5. strategy: round-robin
    6. - name: reasoning-model
    7. strategy: request-header
    8. request-header:
    9. - key: "X-Task-Type" # Client sends the header by task type
    10. values:
    11. - "code" # Route to the coding model when matched with code
    12. model: "coding-model"
    13. - key: "X-Task-Type"
    14. values:
    15. - "reason" # Route to the reasoning model when matched with reason
    16. model: "reasoning-model"
    17. default-model: "coding-model" # Fallback model when no rule matches

    For the exact syntax of strategy names, refer to the official documentation of the tool you choose. More importantly, consider the engineering idea behind it: use task characteristics rather than manual judgment to decide the model, reducing switching costs.

    7.4 Retries and automatic blocking on failure

    Free quotas usually come with concurrency and rate limits. When a rate limit is hit, the probability of a failed result rises noticeably, so the proxy’s retry mechanism becomes especially important:

    • Retryable status codes cover 429 (too many requests), 5xx (server errors), and so on.
    • The number of retries, the initial interval, and the backoff multiplier can be tuned to the upstream characteristics to avoid a retry storm.
    • For an upstream that keeps failing, you can temporarily “block” it for a while and forward requests to other available upstreams during that period.

    Note that an exhausted quota usually returns 402 (payment required). Retrying cannot solve this kind of error, so you should first check the quota and switch keys.

    Security and engineering considerations

    1. Least-privilege credentials: Keep platform keys only in the proxy configuration or environment variables, and let the client hold only a local token. Do not commit config files to a code repository, and do not share screenshots of them.
    2. Network isolation: By default, the proxy can just listen on a local address. When cross-machine access is truly required, be sure to combine it with a reverse proxy and access control, and do not expose the proxy directly to the public internet.
    3. Log redaction: Debug logs may contain full prompts and code snippets, so disable them or restrict access after troubleshooting.
    4. Sustainable quotas: The rate, total volume, and availability period of free quotas can all change, so core workflows should have backup models or a paid production service ready.
    5. Enterprise adoption: For production environments, it is recommended to use a model service platform with enterprise-grade service guarantees, such as the capabilities provided by Baidu AI Cloud, together with infrastructure such as logging services and monitoring/alerting to achieve call auditing and stability assurance.

    Summary

    Around the goal of “bringing free models into everyday development tools,” this article outlines a reusable path:

    1. Register an account on the model service platform, generate a key, and obtain the base_url, api_key, and model identifier;
    2. Deploy the protocol-adapter proxy to handle format conversion, authentication adaptation, and model mapping;
    3. Point both the chat client and the CLI agent to the proxy;
    4. Refine the experience with capabilities such as header hints, function-call anti-hallucination, routing, and retries.

    The real value of this approach lies in “decoupling”: models will be updated, quotas will change, and tools will iterate, but as long as the adaptation layer stays in place, the cost of migration and switching is kept at the configuration level. Whether you want to try out a new model or build a stable enterprise-grade AI-assisted development system, this layering approach applies.