How to Connect Custom LLMs to AI Coding Tools: A Configuration Guide
This guide walks through connecting AI coding tools to a custom large language model instead of a vendor's default cloud service. It explains why you would do this, how to check API protocol compatibility (OpenAI Chat Completions vs. Anthropic Messages) and when a conversion proxy is required, how to install tools such as Claude Code, how to choose a model platform and handle API keys, how to write working configuration with concrete examples, how to verify connectivity, and how to troubleshoot common failures. It also covers data flow, key isolation, least privilege, cost observability, and tiered model usage.
How to Connect Custom LLMs to AI Coding Tools: A Configuration Guide
1. Why Connect Your AI Coding Tool to a Custom LLM?
Many AI coding assistants call a vendor-bound cloud model by default to power features like code completion, semantic explanation, snippet generation, and refactoring suggestions. This is convenient at first, but over time it tends to surface three problems:
- Accumulating costs: Under token-based billing, higher usage means faster cost growth—an issue that becomes especially noticeable as team size increases.
- Limited accessibility: When the call chain depends on a single provider, network fluctuations, rate limiting, or API changes can all disrupt the development experience.
- Compliance and control: Enterprises often have requirements around data flow, industry compliance, and model provenance, so they need the freedom to choose their own model services.
Connecting an AI coding tool to a custom LLM essentially redirects the tool’s model requests from the default service to a model platform of your choice. This lowers the cost of early experimentation and makes it easier for teams to select a model provider that meets their compliance requirements. Keep in mind that whether you can achieve a coding experience close to mainstream tools depends on whether the API protocol is compatible, whether the model’s capabilities match your needs, and whether the tool implementation is stable. No absolute guarantee can be made outside these conditions.
2. Before You Start: Do the Tool and the Model Service Use the Same API Protocol?
Many extensible AI coding tools don’t hardcode the model service address. Instead, they inject it through environment variables or a configuration file. As long as the target model service exposes a usable call interface, you can often complete the integration by overriding the service address, authentication details, and model mapping.
But there’s a critical boundary: don’t assume that “OpenAI-compatible” means every tool can connect directly.
Common API protocols include at least:
- OpenAI-style Chat Completions API: The request path is usually
/v1/chat/completions, and the request body is organized mainly around fields such asmessages,model, andtemperature. - Anthropic Messages API: Some coding tools default to a request format similar to
/v1/messages, with fields and response structures that differ from the OpenAI style. - Custom protocols: Some platforms provide their own request formats; even when the capabilities are similar, you can’t simply swap them in.
If the tool sends requests in the Anthropic Messages format while the model service only provides an OpenAI Chat Completions-compatible endpoint, entering the address directly will usually cause 400, 404, 422, or parsing errors. In this case, changing only BASE_URL isn’t enough—you need to introduce a protocol conversion proxy, such as LiteLLM, claude-code-router, or a self-hosted gateway. The proxy converts the tool’s outgoing requests into the format the target platform supports, then converts the responses back into a structure the tool can understand.
So the first thing to do before integrating isn’t to find an API key—it’s to confirm:
- Which API protocol your coding tool uses by default;
- Which protocol the target model platform provides;
- Whether the two match;
- If they don’t match, whether a mature protocol conversion proxy is available.
3. How to Install the AI Coding Tool?
Installation methods differ across tools. Here’s the common path using Claude Code as an example.
1. macOS: Install Node.js via Homebrew, then install the tool
If Node.js isn’t installed yet, install it with Homebrew:
brew install node
Then install Claude Code:
npm install -g @anthropic-ai/claude-code
If Node.js is already installed, skip the Homebrew step and run npm install -g @anthropic-ai/claude-code directly.
2. Linux: Install with npm
Most Linux environments can also install it via npm:
npm install -g @anthropic-ai/claude-code
If npm isn’t available, install the Node.js runtime first, then run the install command.
3. Windows: Install Node.js LTS, then install the tool
Windows users can install the Node.js LTS release first, then open PowerShell or Command Prompt and run:
npm install -g @anthropic-ai/claude-code
If the tool provides an official one-click install script, you can also get it from the official documentation and run it—but verify the source to avoid running unknown scripts.
After installation, verify the version first:
claude --version
The tool is installed successfully only if it prints a version number normally; only then should you continue to configure the model service.
4. How to Choose a Model Platform and Get an API Key?
When choosing a model platform, don’t look at the model name alone. Focus on these capabilities:
- API protocol: Does it offer the OpenAI Chat Completions, Anthropic Messages, or convertible compatible interface your tool needs?
- Model list: Does it provide tiers such as lightweight, main, and high-performance models, with clear model IDs?
- Transparent pricing: Does it support token-based billing, offer a free tier, and provide quota alerts?
- Compliance capabilities: Does it meet requirements for data residency, log retention, and content safety?
- Documentation quality: Does it offer copyable request examples, error code explanations, and a model ID list?
Taking the Zhipu AI Open Platform as an example, users can typically create an API key after signing up. Keep the following in mind when getting a key:
- A key is equivalent to your account’s calling credential—don’t commit it to a Git repository.
- Don’t publish screenshots of the key, and don’t paste it into public issues, group chats, or documents.
- Create separate keys for different projects to make isolation and revocation easier.
- If the platform supports permission scopes, IP allowlists, or project isolation, enable them whenever possible.
5. How to Write the Configuration: From Placeholders to a Runnable Example
Connecting a custom LLM usually requires configuring three core items:
- Service address: The address of the model service or protocol conversion proxy.
- Authentication token: The API key or proxy token used for identity verification.
- Model mapping: The model IDs that correspond to the tool’s internal tiers—for example, lightweight, main, and high-performance models.
Below are two more concrete examples.
1. If the tool natively supports OpenAI Chat Completions: Aider as an example
Aider is a common command-line coding assistant that lets you specify a custom model service via environment variables. Using the OpenAI-compatible endpoint of the Zhipu AI Open Platform as an example:
export OPENAI_API_KEY="sk-replace-with-your-real-key"export OPENAI_API_BASE="https://open.bigmodel.cn/api/paas/v4"aider --model glm-4-air
If the tool requires a provider prefix, you can write:
aider --model openai/glm-4-air
Here, https://open.bigmodel.cn/api/paas/v4 is an example of the OpenAI-compatible call address provided by the platform, and glm-4-air is a sample real model ID. Paths or model names may change across platform versions, so always defer to the platform’s documentation.
2. If the tool requires the Anthropic protocol: Claude Code with a protocol conversion proxy
Tools like Claude Code may default to the Anthropic Messages protocol. If the target model platform only provides an OpenAI Chat Completions-compatible endpoint, you can’t just fill in the platform address—you first need to start a protocol conversion proxy.
Assume you’ve already started a protocol conversion proxy locally and made it listen on:
http://127.0.0.1:4000
Then the tool’s config file can look roughly like this:
{"env": {"AUTH_TOKEN": "sk-replace-with-your-real-key-or-proxy-token","BASE_URL": "http://127.0.0.1:4000","DEFAULT_FAST_MODEL": "glm-4-flash","DEFAULT_MAIN_MODEL": "glm-4-plus","DEFAULT_TOP_MODEL": "glm-4-plus"},"hasCompletedOnboarding": true}
Field descriptions:
AUTH_TOKEN: This can be the model platform key or a token required by the proxy. If the proxy needs no authentication, follow the proxy’s documentation.BASE_URL: If the tool requires the Anthropic protocol, this should point to the local protocol conversion proxy, not directly to a model platform that only supports OpenAI-compatible calls.DEFAULT_FAST_MODEL: A model ID suited to lightweight tasks such as completion, comments, and simple explanations.DEFAULT_MAIN_MODEL: A model ID suited to everyday main tasks.DEFAULT_TOP_MODEL: A high-capability model ID suited to complex tasks.
The glm-4-flash and glm-4-plus above are specific sample model IDs. Field names can differ across tools—for example, some tools use ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_MODEL, and ANTHROPIC_SMALL_FAST_MODEL—so always defer to the official documentation of the tool you’re using.
3. Config file path examples
If the tool uses settings.json, common paths can be:
- macOS:
~/.claude/settings.json - Linux:
~/.claude/settings.json - Windows:
C:\Users\Zhang\.claude\settings.json
Here Zhang is a sample username. The actual path depends on the tool implementation and the OS user directory, so don’t apply it mechanically across tools.
After modifying the config, run three checks:
- Whether the JSON syntax is valid and free of trailing commas.
- Whether the key contains extra spaces or line breaks.
- Whether the tool has been restarted, so old config isn’t still cached.
6. How to Verify Connectivity and Confirm a Successful Integration
After configuring, verify in the following steps.
Step one: open the command line and launch the tool:
claude
If the tool has an onboarding flow on first run, confirm the prompts as they appear.
Step two: run the status query command. A common form is:
/status
Some tools may take it in the chat interface:
/status
The expected output may look like:
Model: glm-4-plusBase URL: http://127.0.0.1:4000Auth: configured
If the model name and service address in the status match your configuration, the tool has read your config. Next, ask a simple coding question, for example:
Write a Python function that reads a CSV file and prints its column names.
If the tool returns code or an explanation normally, the end-to-end chain is basically working.
If the status looks correct but requests fail, focus on whether the protocol conversion proxy is running, whether the model ID actually exists, whether the account has available quota, and whether the proxy logs show errors returned by the upstream model service.
7. How to Troubleshoot Common Errors?
When integration fails, troubleshoot in this order.
1. Mismatched API protocol or request format
This is easier to overlook than “wrong address.” If error messages contain things like invalid request, unexpected field, 400, 422, or “cannot parse response,” suspect a protocol mismatch first.
How to fix it:
- Confirm whether the tool uses OpenAI format or Anthropic format.
- Confirm which format the model platform provides.
- If the two differ, add a protocol conversion proxy.
- Don’t just change the address without changing the protocol conversion chain.
2. Incorrect API key
This is one of the most frequent issues. Check:
- Whether the key was copied in full;
- Whether it contains spaces, line breaks, or invisible characters;
- Whether the key has expired or been deleted;
- Whether the account has available quota;
- Whether the key is restricted by project-level permissions.
3. Wrong service address
Check whether the address is spelled correctly, whether it has an unnecessary trailing slash, and whether it uses the path from the platform’s latest documentation. For example, some platforms upgrade from /v1 to /v2, or migrate from an old domain to a new one.
4. Nonexistent or unactivated model identifier
The name in the model mapping must be a model that actually exists on the platform and is callable by the current account. For instance, you can’t just fill in nonexistent names like gpt-5-ultra or glm-4-max-turbo. If the tool shows model not found, check the model ID first.
5. Network unreachable
Test basic connectivity on your machine:
curl https://open.bigmodel.cn/api/paas/v4
If the request times out or DNS resolution fails, check the network, proxies, firewall, corporate intranet policies, or DNS resolution.
6. JSON config file syntax errors
Common mistakes include:
- Missing commas;
- Extra commas;
- Using single quotes;
- Unescaped double quotes in the key;
- File encoding other than UTF-8.
After fixing, save and restart the tool.
7. Protocol conversion proxy not running
If you connect through a local proxy, confirm the proxy process is running, the port isn’t occupied, and the logs show no errors. For example, check:
curl http://127.0.0.1:4000
The specific health check method depends on the proxy implementation.
8. How to Ensure Security and Control Costs?
After connecting a custom LLM, security isn’t just about keys—it also covers data flow and compliance boundaries.
1. Understand the data flow
Once a coding tool connects to a custom model service, the requests it sends usually may include:
- Current code snippets;
- User prompts;
- File contents;
- Context summaries;
- Project structure information;
- Dependency or path information.
This information is sent to the model provider you choose and may even pass through a protocol conversion proxy. Before use, confirm:
- Whether the provider’s privacy policy clearly explains how data is used;
- Whether request logs are retained by default;
- Whether the data is used for model training;
- Whether the enterprise edition offers data isolation or zero-retention options;
- Whether the terms of service of the tool you use allow custom model endpoints.
If you’re handling client code, financial data, medical data, or trade secrets, complete a security assessment before connecting to an external model service.
2. Isolate key management
Don’t hardcode keys in your project. Recommended approaches include:
- Using environment variables;
- Using the system keychain;
- Using CI/CD secret management;
- Using a key management service;
- Delivering keys through secure channels when distributing within a team.
Also mind the permissions on the config file itself, for example:
chmod 600 ~/.claude/settings.json
3. Least privilege
Create separate keys for different projects or purposes instead of sharing one key across all projects. This reduces leak risk and makes revocation easy at any time.
4. Observable costs
Regularly review call volume, token consumption, and cost trends in the platform dashboard. For team scenarios, establish a usage baseline and set alert thresholds to avoid abnormal consumption.
5. Tiered model usage
Different tasks demand different model capabilities. Allocate models by task complexity to balance quality and cost.
| Task type | Recommended tier | Example models | Trade-off |
|---|---|---|---|
| Code completion, comment generation, simple explanations | Lightweight | glm-4-flash, glm-4-air |
Lower latency, more controllable cost |
| Everyday Q&A, unit tests, routine refactoring | Main | glm-4-plus |
Balance of quality and cost |
| Complex debugging, cross-file refactoring, architecture analysis | High-performance | High-capability models offered by the platform | Usually better results, but potentially higher cost and latency |
The example models here are only for illustrating the configuration approach; actual options should follow the platform’s current model list.
9. Summary
The core of connecting an AI coding tool to a custom LLM isn’t simply replacing an address with some URL—it’s completing the configuration of an entire call chain. Focus on five areas:
- Confirm the protocol: Whether the tool uses OpenAI Chat Completions or Anthropic Messages; if the protocols differ, use a protocol conversion proxy.
- Install the tool: Know the tool name and installation method—for example, install Node.js via Homebrew, then install Claude Code via npm.
- Prepare the key: Get an API key from the model platform and store it securely.
- Write the configuration: Fill in the service address, authentication token, and model mapping, favoring reproducible real examples.
- Verify and troubleshoot: Use status queries and real requests to confirm the chain, focusing on protocol, key, address, model ID, network, and JSON syntax issues.
On the security side, keep in mind that code and prompts are sent to the model provider you choose, and confirm the tool’s terms, the platform’s privacy policy, and enterprise compliance requirements in advance. Once you have matching interfaces, reproducible configuration, clear data flow, and monitorable costs, you can achieve a more stable and controllable AI coding experience on top of a custom large model.
