How to Configure Command-Line AI Assistants with Large Language Models on Ubuntu 24.04

    Learn how to connect command-line AI assistants to large language models on Ubuntu 24.04. This guide covers preparing the environment, installing two example tools (a minimal Node.js CLI and Aider), configuring a custom OpenAI-compatible provider, and verifying connectivity with curl. It also walks through common errors such as 401 authentication failures, missing models, and timeouts, and closes with security best practices for handling API keys and configuration files. Ideal for developers working on headless servers, over SSH, or inside scripts.

    Connecting a command-line AI assistant to a large language model on Ubuntu 24.04 is not mysterious. At its core, it means letting a terminal tool reach a model service over an HTTP API. As long as the server is compatible with the OpenAI calling convention, the same approach works across different platforms. This article is aimed at developers who work on servers, over SSH, in headless environments, or in scripts. It explains how to prepare the environment, install example tools, configure a custom provider, verify the connection, and handle common errors. If you only use a graphical client, this article may not be your first choice; but if you need a terminal-based solution that is automatable, auditable, and light on resources, you can follow the steps below.

    1. Use Cases and Scope

    Command-line AI assistants are well suited to the following scenarios:

    • Servers or cloud hosts without a desktop environment;
    • Calling large models from within scripts;
    • Batch processing of logs, code, and configuration files;
    • Integrating model capabilities into Git hooks, CI/CD pipelines, and operations commands;
    • Managing API keys, base URLs, and model names centrally.

    This article is not tied to any specific graphical client, nor does it promise that every terminal tool can be configured in exactly the same way. The example tools include a command-line script written in Node.js and Aider. As long as the tool you use supports the elements “provider name, API address, API key, and model name,” the approach in this article applies.

    2. Environment Preparation

    Confirm the system version

    First confirm the Ubuntu version and architecture so you can later judge whether dependencies are compatible:

    1. lsb_release -a
    2. uname -m

    Ubuntu 24.04 usually provides newer software sources and dependency libraries. It is recommended to update the package index first:

    1. sudo apt update && sudo apt upgrade -y

    Install base dependencies

    It is a good idea to prepare common tools first:

    1. sudo apt install -y curl git build-essential python3 python3-pip python3-venv

    Install Node.js

    Many command-line AI assistants are built on Node.js. Node.js in the Ubuntu 24.04 repositories is usable, but if you want a more controllable version, using nvm to install an LTS release is recommended.

    Install via apt:

    1. sudo apt install -y nodejs npm
    2. node -v
    3. npm -v

    Install LTS via nvm:

    1. curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash
    2. source ~/.bashrc
    3. nvm install --lts
    4. nvm use --lts
    5. node -v
    6. npm -v

    Choosing Node.js 20 LTS or newer is recommended. If a tool requires a specific version, defer to the tool’s official documentation.

    Prepare large model credentials

    You need two pieces of information in advance:

    • API Key: the access key issued by the model platform;
    • Base URL: the endpoint of the model service, usually ending in /v1 or /v4.

    For example:

    1. export OPENAI_API_BASE="https://api.example.com/v1"
    2. export OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxx"

    In actual use, replace the example domain with the address given in the platform’s documentation.

    3. Installing Example Command-Line AI Assistants

    Below are two approaches you can copy.

    Example 1: Build a minimal command-line assistant with Node.js

    If you do not want to depend on a third-party CLI, you can write a minimal assistant using Node.js and the openai SDK.

    1. mkdir -p ~/ai-cli
    2. cd ~/ai-cli
    3. npm init -y
    4. npm install openai

    Create cli.js:

    1. const OpenAI = require("openai");
    2. const prompt = process.argv.slice(2).join(" ") || "Hello";
    3. (async () => {
    4. const client = new OpenAI({
    5. apiKey: process.env.OPENAI_API_KEY,
    6. baseURL: process.env.OPENAI_API_BASE
    7. });
    8. const response = await client.chat.completions.create({
    9. model: process.env.OPENAI_MODEL || "your-model-name",
    10. messages: [{ role: "user", content: prompt }]
    11. });
    12. console.log(response.choices[0].message.content);
    13. })();

    Set environment variables and run:

    1. export OPENAI_API_BASE="https://api.example.com/v1"
    2. export OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxx"
    3. export OPENAI_MODEL="your-model-name"
    4. node cli.js "Introduce Ubuntu 24.04 in three sentences"

    This example is suitable for learning the underlying principles and doing secondary development. It requires no configuration wizard, but it still depends on a provider, a base URL, an API key, and a model name.

    Example 2: Install Aider

    Aider is a common command-line AI coding assistant that can be installed via pipx.

    1. sudo apt install -y python3-pipx
    2. python3 -m pipx ensurepath
    3. pipx install aider-chat
    4. aider --version

    Ubuntu 24.04 ships with Python 3.12, which usually meets Aider’s basic runtime requirements. If a version-incompatibility warning appears, you can install with uv or a virtual environment.

    When using an OpenAI-compatible interface, a common approach is to specify the address and key via environment variables:

    1. export OPENAI_API_BASE="https://api.example.com/v1"
    2. export OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxx"
    3. aider --model openai/your-model-name

    The key point here is that variable names such as OPENAI_API_BASE and OPENAI_API_KEY must match the variable names the tool actually reads. If the tool reads MODEL_API_KEY, you need to change to the corresponding variable, or declare the variable name in the tool’s configuration.

    4. Understanding Custom Model Providers

    Most command-line tools abstract model invocation into a two-layer structure:

    Field Meaning Tips
    provider Where the request is sent and which credentials are used Use a readable short name, such as custom-provider
    baseUrl The model service address Must match the platform documentation; note /v1, /v4
    apiKey or apiKeyEnv The key or the key variable name Using environment variables is recommended to avoid writing the key in plaintext
    model The model name Must be a model string actually offered by the platform

    A typical configuration might look like this:

    1. {
    2. "providers": {
    3. "custom-provider": {
    4. "baseUrl": "https://api.example.com/v1",
    5. "apiKeyEnv": "OPENAI_API_KEY",
    6. "models": ["your-model-name"]
    7. }
    8. },
    9. "defaultModel": "your-model-name"
    10. }

    If a tool uses an interactive configuration wizard, it usually prompts you to:

    1. Choose Add custom provider;
    2. Enter the provider name;
    3. Enter the Base URL;
    4. Enter the API Key or the key variable name;
    5. Enter the model name;
    6. Set it as the default model.

    The wording of different tools’ menus may vary, but the core fields are essentially the same.

    5. Base URL Rules

    The Base URL is one of the parameters most prone to errors. For a platform compatible with the OpenAI protocol, common formats include:

    1. https://api.example.com/v1
    2. https://api.example.com/v4

    Keep the following in mind when filling it in:

    • Use the complete address given in the platform documentation;
    • Preserve the correct version path, such as /v1 or /v4;
    • For most OpenAI-compatible interfaces, do not add a trailing slash, otherwise the URL may concatenate into //chat/completions;
    • If the platform documentation explicitly requires a trailing slash, follow the documentation;
    • Do not treat the web console address as the API address.

    Verification method:

    1. curl -sS "$OPENAI_API_BASE/models" \
    2. -H "Authorization: Bearer $OPENAI_API_KEY"

    If it returns a model-list JSON, the address, path, and authentication are basically correct. If it returns an HTML login page or a 404, the Base URL is usually wrong.

    6. Locating and Maintaining Configuration Files

    An interactive wizard usually ends up writing to a configuration file. Locations differ across tools, but common directories include:

    1. ls -la ~/.config
    2. find ~/.config -maxdepth 3 -name config.json -type f 2>/dev/null

    Some tools may use:

    1. ~/.config/ai-assistant/config.json
    2. ~/.aider.conf.yml
    3. ~/.config/aider/aider.conf.yml

    If you find the configuration file, you can edit it manually. When editing, focus on:

    • Whether the JSON or YAML syntax is correct;
    • Whether the field names match the tool’s documentation;
    • Whether environment variables are used instead of a plaintext key;
    • Whether the default model name is correct.

    7. Connection Verification and Common Issues

    After configuring, do not put it into production use directly; verify connectivity first.

    Verify with the tool’s built-in chat

    If the tool supports a single-turn test, you can run:

    1. aider --model openai/your-model-name

    Then type a message such as “Hello.” If it returns normally, the provider, API key, and model name are most likely correct.

    Verify with curl

    Get the model list:

    1. curl -sS "$OPENAI_API_BASE/models" \
    2. -H "Authorization: Bearer $OPENAI_API_KEY"

    Call the chat endpoint:

    1. curl -sS "$OPENAI_API_BASE/chat/completions" \
    2. -H "Content-Type: application/json" \
    3. -H "Authorization: Bearer $OPENAI_API_KEY" \
    4. -d '{
    5. "model": "your-model-name",
    6. "messages": [{"role": "user", "content": "Hello"}]
    7. }'

    If curl succeeds but the tool fails, first check the variable names the tool reads, the configuration file, and the model name.

    What to do about API Key 401?

    401 or 403 usually indicates an authentication failure. You can check:

    • Whether the API key was copied completely;
    • Whether spaces or line breaks were mixed in;
    • Whether the key has expired or been revoked;
    • Whether the variable name the tool reads is correct;
    • Whether you mistakenly used a web account password instead of an API key.

    You can show only the first few characters of the key to avoid full disclosure:

    1. echo "${OPENAI_API_KEY:0:6}..."

    How to check for a missing model?

    If it returns “model does not exist” or model not found, first call /models to see the model names actually supported by the platform:

    1. curl -sS "$OPENAI_API_BASE/models" \
    2. -H "Authorization: Bearer $OPENAI_API_KEY"

    Then change the model name in the tool’s configuration to the real name in the list.

    What to do about connection timeouts?

    Timeouts are usually network, proxy, or firewall issues. First check the proxy environment variables:

    1. env | grep -i proxy

    Set them if necessary:

    1. export HTTPS_PROXY="http://127.0.0.1:7890"
    2. export NO_PROXY="localhost,127.0.0.1"

    You can also use verbose logs to pinpoint the issue:

    1. curl -v --max-time 10 "$OPENAI_API_BASE/models" \
    2. -H "Authorization: Bearer $OPENAI_API_KEY"

    If the network where the server is located cannot reach the target API, you need to change the network egress, configure a proxy, or switch to an accessible model service address.

    8. Security Best Practices

    The API key is a sensitive credential, and the following practices are recommended.

    Do not hardcode plaintext keys

    Do not write keys into scripts, project code, or commit them to Git. The project directory should include a .gitignore:

    1. echo ".env" >> .gitignore

    Variable names must match the tool

    If the tool reads OPENAI_API_KEY, export:

    1. export OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxx"

    If the tool reads MODEL_API_KEY, then export:

    1. export MODEL_API_KEY="sk-xxxxxxxxxxxxxxxx"

    Or use the following in the configuration file:

    1. "apiKeyEnv": "MODEL_API_KEY"

    Restrict configuration file permissions

    If the configuration file contains sensitive information, you can restrict it to read/write by the current user only:

    1. chmod 600 ~/.config/ai-assistant/config.json

    Rotate keys regularly

    If you suspect a key has been leaked, revoke and regenerate it on the platform side immediately. Do not keep reusing the old key.

    Principle of least privilege

    Request separate keys for development, testing, and production environments to make auditing easier and to enable quick damage control.

    9. Summary

    Configuring a command-line AI assistant to connect to a large language model on Ubuntu 24.04 comes down to four elements: provider name, Base URL, API key, and model name. Once you understand the layered structure of provider and model, whether you use a self-built Node.js CLI, Aider, or any other tool compatible with the OpenAI convention, you can troubleshoot problems with the same method.

    The recommended workflow is: first prepare a Node.js or Python runtime, then install the example tool, then configure the custom provider, then use curl to verify the /models and chat endpoints, and finally tighten permissions and manage keys. As long as you align the address, key, model name, and variable name, a command-line environment can use large model capabilities stably and securely.