A Complete Guide to Configuring and Using Local LLMs in AI Coding Tools
Learn how to connect a custom or locally deployed large language model to an AI coding tool, configure model identifiers and API credentials, use the model in a local workspace, distinguish local and cloud execution modes, and troubleshoot common connection and context issues. The guide also covers code privacy, credential security, and safe usage practices.
1. Why Connect a Local LLM to an AI Coding Tool?
AI coding tools typically include built-in models for code generation, code explanation, debugging, project analysis, and agent-based tasks. In real-world development, however, developers may need more flexibility.
Use models with specific capabilities
Built-in models may not cover every scenario. Different models can perform better at code completion, complex reasoning, long-context analysis, or multi-step task execution. Developers may want to switch models based on the task.Use a unified model access method
Organizations and individual developers may already have access to a model service and want to call it directly from their existing coding tool instead of switching between websites, command-line tools, and other applications.Improve model management flexibility
Custom model configurations allow developers to save model names, model identifiers, and access credentials in the tool, making it easier to select models across different projects.Control the scope of code access
Local workspace modes generally allow the model to access only the local folders explicitly selected by the user. This helps control the project context. However, “local mode” does not necessarily mean that inference runs entirely on the local machine. The coding tool may run locally while model requests are still sent over the network to a remote model service.
Before configuring a model, clarify your objective: do you want to use an inference model deployed on your own machine, or call a remote model service from a local coding environment? These approaches differ significantly in networking, credential management, resource usage, and data flow.
2. Preparation: Confirm the Model Service and Access Method
Before adding a custom model to an AI coding tool, prepare a callable model service. The service may be deployed inside an organization or hosted by a model platform that supports a standard interface.
Confirm the following information.
2.1 Check Whether the Model Service Supports External Calls
The model service must expose an interface that the coding tool can access. Common capabilities include:
- An interface compatible with a general chat model protocol;
- A conversational interface for code generation and code analysis;
- Model listing, model invocation, or streaming output;
- Text input and, when required, image or other multimodal input.
If the model service only supports browser-based chat and provides no external API, it cannot be connected directly through a custom model feature.
2.2 Obtain an Access Key
Access keys are usually created in the model service’s management console. Save the key immediately after creation because some services display the complete value only once.
Follow these security practices:
- Do not hard-code the key in project source code;
- Do not commit it to a version control repository;
- Do not paste it into public documents, screenshots, or tickets;
- Do not share it with unrelated individuals;
- Revoke and recreate it immediately if you suspect exposure;
- Prefer keys with limited permissions and shorter lifetimes.
If your organization supports subaccounts, permission policies, or key rotation, create a dedicated credential for the coding tool instead of reusing a high-privilege key across multiple systems.
2.3 Confirm the Model Identifier
Model services usually require an exact model identifier. This value may differ from the model’s display name and may instead be an internal name defined by the service.
A configuration may need to distinguish between:
- Display name: A label that helps developers identify the model;
- Model identifier: The value actually sent to the API;
- Provider name: The source of the model;
- Interface type: The protocol the tool uses to build requests.
If the model identifier is incorrect, the service may report that the model does not exist, reject the request, or prevent the tool from loading the model.
3. Add a Custom Model to the Coding Tool
AI coding tools that support custom models generally provide two configuration entry points. The exact labels may vary by version, but the overall process is similar.
Entry Point 1: Add a Model from the New Task Page
Open the tool’s new task page and complete these steps:
- Select local workspace mode;
- Select the local project folder to analyze;
- Open the model list in the model selection area;
- Click Add Model or a similar option;
- Enter the model service and access credential details;
- Save the configuration and return to the model list.
This method is convenient for first-time setup because model selection and task creation happen together.
Entry Point 2: Add a Model from Settings
You can also open the tool’s settings and configure the model from the model management page. The path is generally similar to:
Settings → Models → Add Custom Model
This approach is better for managing multiple models. For example, you might configure one model for code generation, another for complex reasoning, and a third for fast question answering.
Recommended Configuration Fields
Provider
If the tool includes the target service as a built-in provider, select the corresponding option. If it does not, select a general custom provider type supported by the tool.
The provider field tells the tool which request protocol to use. Do not select a provider based only on its name. An incorrect choice can cause incompatibilities in the endpoint path, request format, or authentication method.
Model Name
The model name is the user-facing label. Choose a clear name, such as:
- Code Analysis Model;
- Local Development Model;
- Enterprise Model;
- Long-Context Model.
Keep the name concise and recognizable instead of using a complicated internal identifier.
Model Identifier
The model identifier must exactly match the value required by the model service, including capitalization, separators, and version suffixes. This is one of the most important settings for successful model invocation.
If the service provides a model-listing function, copy the identifier returned by the service instead of guessing it from the display name.
Multimodal Support
If the model supports images, screenshots, diagrams, or interface analysis, enable multimodal support. The tool may then allow images to be attached to conversations.
Enabling this option does not mean that every model can understand images. If the model does not support image input, requests may fail. Always verify the model’s actual capabilities with the service documentation.
API Key
Paste the access key you created into the appropriate field. Check that it does not contain leading or trailing spaces, line breaks, or invisible characters.
If the tool provides a Test Connection function, use it before saving. A successful test generally confirms basic network access, authentication, and interface availability, but it does not guarantee compatibility with every advanced capability.
4. Use the Custom Model in Local Workspace Mode
After the configuration is saved, create or open a project in local mode.
Usage Workflow
- Launch the AI coding tool;
- Create a new local task or open an existing one;
- Select the project directory to analyze;
- Select the configured custom model from the model dropdown;
- Wait for the tool to establish a session;
- Submit requests for code explanation, refactoring, test generation, or debugging.
For the first test, use a simple request. For example, ask the model to describe the project directory structure or explain a small source file. Once the model can correctly read the project context, proceed to larger refactoring or cross-file changes.
Local File Access Scope
In local mode, the tool generally builds project context from the directory selected by the user. To reduce the risk of exposing sensitive information:
- Select only the required project directories;
- Avoid selecting a high-level directory containing many unrelated files;
- Exclude secret files, certificates, database backups, and similar sensitive content;
- Check environment variable files in the project;
- Do not include production configuration, personal information, or internal credentials unnecessarily.
Whether the model can read a particular file also depends on the tool’s indexing rules, file types, ignore configuration, and the context window limit for the current task.
5. Local Mode vs. Cloud Mode
Many tools distinguish between local and cloud environments. Even if a custom model has been saved successfully, it may not be available in every execution mode.
Local Mode
Local mode typically has the following characteristics:
- Tasks are created in the workspace on the user’s device;
- Users can select local project folders;
- Custom model configurations generally take effect in local sessions;
- It is suitable for individual development, internal projects, and controlled code analysis.
Cloud Mode
Cloud mode generally means that tasks run in a remote environment. Code, context, and execution processes may need to be uploaded to a remote workspace. Because credential management, network access, permission isolation, and data compliance policies differ, some tools do not allow custom models in cloud mode.
Therefore, if a model works correctly in local mode but cannot be selected in cloud mode, the issue may not be a configuration error. It may simply reflect a product capability boundary.
For enterprise source code, confirm the following in advance:
- Whether the code is allowed to leave the local network;
- Whether the model service meets the organization’s data-handling requirements;
- Whether the access key may be used in a remote environment;
- Whether security review or administrator approval is required.
6. Common Problems and Troubleshooting
6.1 The Custom Model Does Not Appear in the Model List
Check the following:
- Whether the configuration was saved;
- Whether the tool is running in the correct mode;
- Whether the model configuration is available to the current account or device;
- Whether custom models are limited to local tasks;
- Whether the tool must be restarted to refresh its configuration.
6.2 The Access Key Is Reported as Invalid
Common causes include:
- The key was copied incompletely;
- It contains leading or trailing spaces or line breaks;
- It has been revoked or expired;
- A key from the wrong environment was used;
- The account lacks permission to call the target model;
- The service requires a different authentication method.
You can create a new key with clearly defined permissions for testing, but do not repeatedly paste real credentials into public or untrusted environments.
6.3 The Model Does Not Exist
Check the model identifier rather than the display name. Confirm the exact value in the model service’s console or API documentation, including capitalization and version information.
6.4 Requests Time Out or Cannot Connect
Possible causes include:
- The current network cannot access the model service;
- An enterprise proxy or firewall is blocking requests;
- The service is temporarily busy;
- The endpoint or protocol type is configured incorrectly;
- The request is too large for the service’s limits.
Start with the shortest possible prompt and gradually add code context. This helps determine whether the problem is related to networking, model configuration, or request size.
6.5 The Model Cannot Understand the Entire Project
This is usually related to context scope. A model does not automatically read an entire codebase without limits. You can improve results by:
- Specifying the files that need analysis;
- Asking the model to summarize the project structure first;
- Splitting complex tasks into smaller tasks;
- Analyzing interfaces before implementations;
- Avoiding large amounts of unrelated context in a single request;
- Asking the model to cite file paths and code locations for important conclusions.
7. Security and Usage Recommendations
After connecting a custom model, the most important question is not only whether it can be called, but also which data is sent during the request.
Adopt the following practices:
- Use least-privilege credentials: Grant only the model permissions that are required.
- Redact sensitive content: Remove accounts, passwords, tokens, certificates, and personal information.
- Review before committing: Model-generated code must be compiled, tested, and checked for security issues.
- Avoid blindly executing commands: Confirm the scope and impact of deletion, migration, deployment, and other commands suggested by the model.
- Control the context scope: Provide only the files required to complete the task.
- Record configuration changes: In team environments, document the model source, configuration date, and responsible user.
- Rotate keys regularly: Establish a regular rotation process for long-lived access credentials.
Overall, using a custom local model in an AI coding tool involves five core steps: prepare a callable model service, create a secure access key, enter the correct model identifier, select the model in local workspace mode, and validate the configuration with a small task. By paying attention to interface compatibility, execution-mode limitations, code privacy, and credential security, developers can use custom LLMs more reliably for code generation, project analysis, and development assistance.
