How to Connect AI Coding Tools to Cloud-Hosted Models: A Complete Configuration Guide
A step-by-step guide to pointing an AI coding assistant at a cloud-hosted large language model. It explains why a local configuration still calls an HTTPS endpoint, how to obtain and store API keys safely, how to look up exact model IDs and inspect model capabilities, how to fill in each configuration field, how to verify connectivity and tool-calling support, and how to diagnose common failures such as 401, 404, 400 and 429 responses. It also covers token budgets, output limits, cost control and data compliance. All endpoint paths, parameters and field names are generic examples.
Many developers get confused when they connect a “local model” to their AI coding tool: there is no GPU on the machine, so why does the configuration ask for an HTTPS URL? The word “local” is usually used loosely here. It typically means that the model configuration is stored in a local file and the request is issued by a local client, while the actual inference runs on a remote cloud server. This article walks through the full path for the common scenario of letting a local coding assistant call a cloud-hosted model: obtaining a key, discovering models, filling in the configuration, verifying connectivity, troubleshooting failures, and controlling cost and compliance. All endpoint paths, parameters and fields below are generic examples; always follow the latest documentation of the provider you choose.
Why an HTTPS URL in a Local Config Still Means Cloud Access
In Chinese-language discussions, “calling a local model” often blurs two very different approaches:
| Aspect | Local inference deployment | Cloud-hosted model access |
|---|---|---|
| Compute | Your own CPU, GPU or accelerator | The provider’s remote inference cluster |
| Data flow | Data never leaves the machine | Requests are sent to a remote server |
| Setup effort | Download weights, configure a runtime, tune resource usage | Configure the client as soon as you have a key |
| Model capability | Bounded by local hardware | Choose from models with different parameter counts and capabilities |
| Cost model | Hardware depreciation and electricity | Per-token or per-call billing, sometimes with a free tier |
This article covers the second approach. Its advantage is that no extra accelerator hardware is needed locally: once the assistant is configured, it can call the cloud model, and you usually get strong code capabilities. The actual results still depend on model choice, context length and task complexity. The trade-off is that code snippets and question context leave your machine, which is why the compliance and cost sections below deserve separate attention.
How to Obtain and Store Your API Key Safely
First, register with the provider and open its console. Most cloud vendors and model service platforms expose an “API Keys” or key management entry. After signing in, you can create a key from your account settings, project settings or credentials page. Keys usually carry a vendor prefix followed by a random string; the prefix identifies the source.
Second, store the key carefully. A key is typically displayed in full only once, at creation time, and may not be viewable again afterward. Recommendations:
- Prefer a system-level secret store, such as Windows Credential Manager, the macOS Keychain,
secret-serviceon Linux, or a team-wide secret management service. - If you can only keep it as text for now, place it under your user directory, restrict file permissions, and make sure a sync tool does not upload it by accident.
- Create separate keys for different environments so you can revoke them individually.
- If you suspect a leak, disable the key in the console immediately and rotate it.
In PowerShell, write the key into an environment variable first and use it from there:
# Set for the current session so the key does not easily end up in command history$env:LLM_API_KEY = "<your-api-key>"# Build the request headers from the environment variable$headers = @{ "Authorization" = "Bearer $env:LLM_API_KEY" }
This reduces the risk of pasting the key into chat logs, support tickets, script repositories or shell history.
Looking Up the Model List to Avoid a Wrong Model ID
One of the most error-prone fields in a configuration file is the model identifier. Different platforms may name the same model with a namespace prefix, a version number, different casing, or even different hyphens. A single wrong character in the model ID can produce a 404.
So before configuring anything, query the model list endpoint. Using a generic OpenAI-compatible interface:
$resp = Invoke-RestMethod `-Uri "https://<provider-domain>/v1/models" `-Headers @{ "Authorization" = "Bearer $env:LLM_API_KEY" }$resp.data | Select-Object id, context_window | Format-Table -AutoSize
In the response, id is normally the value you put into the configuration file. The output has roughly this shape (illustrative only, not a real model list):
id context_window-- --------------<vendor>/<model-name>-<version> <context window of the model><vendor>/<another-model> <context window of the model>
Always use the real id returned by the endpoint, and do not copy the placeholders above.
A common pitfall in Windows PowerShell 5.x: do not write curl directly. In older PowerShell versions, curl may be an alias for Invoke-WebRequest, which does not understand options such as -H, -s or -o. When you need curl semantics, write the full name curl.exe. If you only need to call an API, prefer Invoke-RestMethod, which automatically deserializes JSON into objects and is less work.
If older PowerShell versions raise TLS errors, set TLS 1.2 and add a request timeout:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Fetching a Single Model’s Details to Confirm Capabilities
Once you have the model ID, you can query the details for that single model and check the context window, function calling support, vision input support, structured output support and similar capabilities.
$modelId = "<vendor>/<model-name>-<version>"$encodedId = [uri]::EscapeDataString($modelId)$model = Invoke-RestMethod `-Uri "https://<provider-domain>/v1/models/$encodedId" `-Headers @{ "Authorization" = "Bearer $env:LLM_API_KEY" }$model | ConvertTo-Json -Depth 5
If the model ID contains slashes, pasting it directly into the URL can break path resolution. In that case, URL-encode the model ID, or check whether the provider supports fetching details through query parameters or the model list endpoint instead.
The value of this step is capability alignment. If your coding assistant relies on function calling, tool calling or agent modes that read and write files and run commands, and the model you picked does not support those capabilities, you may see a conversation that works fine while tools never execute. That is a capability mismatch, not necessarily a configuration error.
Writing the Configuration File: Field by Field
An AI coding tool’s model configuration usually lives in a configuration folder under your user directory, for example .workbuddy/models.json. You can confirm the exact path in the tool’s “Model Settings” screen, or simply search for models.json.
A generic example:
{"id": "<vendor>/<model-name>-<version>","name": "Generic Placeholder Model Name","vendor": "OpenAI","url": "https://<provider-domain>/v1/chat/completions","apiKey": "${LLM_API_KEY}","maxInputTokens": 120000,"maxOutputTokens": 8192}
Field descriptions:
| Field | Required | Description |
|---|---|---|
id |
Yes | Unique model identifier; must match data[].id from the API exactly, including case, hyphens and namespace |
name |
Yes | Friendly name shown in the tool UI; free-form, and it helps to include the platform or intended use so models are easy to tell apart |
vendor |
Yes | Request protocol identifier; usually means requests are sent in OpenAI-compatible format and does not necessarily indicate where the model actually comes from |
url |
Yes | Chat completions endpoint. Some tools expect the full /chat/completions path, others expect a base URL; follow the tool’s documentation |
apiKey |
Yes | Key field. Prefer an environment variable or the key reference mechanism the tool supports; do not commit the value in plain text to a code repository |
maxInputTokens |
Recommended | Maximum input tokens allowed per request. It must be smaller than the model’s context window, with room left for the output; leave it empty if unsure and let the tool derive it from the model’s capabilities |
maxOutputTokens |
Recommended | Maximum output tokens per completion; it affects whether long code gets truncated |
rateLimit |
Optional | Client-side throttling configuration. The actual values should follow the quota shown in the console; avoid hard-coding them |
Two points need special attention.
First, the context window is not the input limit. The context window is normally the total budget for input plus output. If you set maxInputTokens to the full context window and then ask for a long output, the request can exceed the limit and fail. The 120000 in the example only illustrates the relationship between the fields; in practice it must be replaced with a value smaller than the context window of the model you use.
Second, maxOutputTokens is a ceiling, not a target. Setting it very high does not necessarily produce longer answers, and it can increase retry cost when a request fails. Setting it too low can truncate long functions or multi-file changes. For everyday questions, 2048 to 4096 is a reasonable starting point; for refactoring work, try 8192 or higher, depending on the model and the task.
Verifying That the Configuration Works
The first layer is to bypass the tool and test the endpoint directly, which tells you whether the problem is in the tool or in the API:
$body = @{model = "<vendor>/<model-name>-<version>"messages = @(@{ role = "user"; content = "Write a quicksort in Python" })max_tokens = 256temperature = 0.2} | ConvertTo-Json -Depth 5Invoke-RestMethod -Uri "https://<provider-domain>/v1/chat/completions" `-Method Post -ContentType "application/json" `-Headers @{ "Authorization" = "Bearer $env:LLM_API_KEY" } -Body $body |Select-Object -ExpandProperty choices |Select-Object -ExpandProperty message |Select-Object content
If you get a model response, the key, the endpoint URL and the model ID are basically correct.
The second layer is a minimal closed loop inside the tool. Open a small read-only file and ask the assistant to explain the logic of a function. This verifies request delivery, response parsing and streaming rendering.
The third layer is verifying tool-calling capability. Ask the assistant to “list the files in the current directory”. If the conversation replies but nothing happens, check first whether the selected model supports function calling or tool calling.
Troubleshooting Quick Reference
| Symptom | Common cause | What to do |
|---|---|---|
| 401 / 403 | Wrong or revoked key, missing Bearer prefix |
Verify the key and check whether an environment variable was overwritten; confirm the authentication format |
| 404 | url set to the base URL, or a misspelled model id |
Copy the id exactly as returned by the model list endpoint |
| 400 parameter error | Input exceeds the context window, or a parameter is not supported by the current model | Shorten the context and adjust parameter names according to the model documentation |
| 429 | Requests-per-minute or tokens-per-minute limit reached | Check response headers and console quotas, reduce concurrency, compress the context, spread requests over time, and request a quota increase if needed |
| Request hangs or times out | Long-context inference, blocked network, proxy problems | Add a timeout, and check network and proxy configuration |
| Reply is truncated | maxOutputTokens too small, or a reasoning model emits its analysis first |
Increase the output budget, or switch to a better-suited model |
| Garbled Chinese text | Inconsistent request body encoding | Use UTF-8 consistently |
| Works but expensive | The full repository context is attached on every request | Paste only relevant files and trim the conversation context |
Practical Advice on Cost, Performance and Compliance
Context budget is the main cost driver. Token usage generally grows with the number of conversation turns and the amount of attached files. Start a new session for a new task, paste only the relevant files rather than the whole repository, and turn repeated instructions into a project-level rules file instead of repeating them every turn.
When you need machine-readable output, prefer JSON mode together with schema validation; it is more reliable than repeatedly asking for JSON in the prompt.
Before rolling this out, work through a compliance checklist:
- Whether the platform uses submitted code for training.
- How long request and response logs are retained.
- Whether keys support rotation and least-privilege access.
- Whether auditing, quota alerts and abnormal-usage monitoring exist.
- Whether the setup meets your organization’s data residency requirements.
- Whether a self-hosted deployment or enterprise-grade platform service needs to be evaluated.
Also, avoid picking a model on parameter count alone. Models behind the same interface can differ considerably in code completion, long-file refactoring and tool calling. Test three to five real tasks from your own project on a small scale, and evaluate them across three dimensions: pass rate, latency and cost.
Conclusion
Breaking “local model” apart leaves four independent steps: obtain the key, discover the model ID, write the configuration fields correctly, and verify connectivity and capabilities. Each step has familiar pitfalls, especially a mistyped model ID, a wrong URL, an unreasonable context budget, key leakage, and choosing a model that does not support tool calling. Following the steps above is usually enough to connect a cloud-hosted model while reducing common failures and keeping cost and data risk within acceptable limits.
