Deploying an Open-Source AI Assistant Service on a Linux VM: A Practical Guide
This guide explains how to deploy an open-source AI assistant service on a Linux virtual machine, from hardware sizing and Ubuntu installation to SSH remote management, Node.js runtime control, service configuration, and messaging channel integration. It also covers callback security, message-format adaptation, rate limiting, retries, logging, and long-running operation.
1. Deployment Background and Scenario Analysis
As large language model technology becomes more widespread, more developers want to deploy a controllable intelligent chat assistant service in a private environment for message access, multi-channel distribution, and automated processing. These open-source services are often built on the Node.js runtime and expose capabilities through a “core service + channel adapter” architecture. They commonly support mainstream channels such as Telegram and Slack out of the box. For teams that rely on regional messaging platforms, however, connecting the assistant to tools such as DingTalk, QQ, WeCom, or Feishu is what makes it practical for daily use.
There are three common deployment forms: containerized deployment, virtual machine deployment, and bare-metal deployment. If the device itself has desktop capabilities—for example, some NAS systems support HDMI passthrough to output a Linux desktop—the demo effect may be good. However, these container-based desktop approaches often have complex resource isolation and incomplete lifecycle management, making them unsuitable for service applications that need to run in the background for long periods. Considering stability and maintainability, this article recommends the virtual machine approach: run an independent Linux system on the NAS virtualization platform and let the assistant service reside there for long-term operation.
2. Device Selection and Resource Planning
The hardware requirements for deploying this type of service are not high. Taking a mainstream low-power mini NAS as an example, its virtualization platform usually supports creating standard Linux virtual machines. A reference resource plan is as follows:
- CPU: 2 cores are enough for daily conversation forwarding and message processing.
- Memory: Start with 4 GB to cover both system overhead and service operation.
- Disk: 20 GB or more for the system disk and log storage.
- Network: Bridged mode is recommended so the VM obtains an independent LAN IP address, simplifying external access and port mapping.
This configuration is more than sufficient for a message-forwarding service. Even if channel adapters or logging components are added later, there is still some headroom. If you plan to run heavier tasks such as vector retrieval or local model inference at the same time, you will need to increase the resource allocation accordingly.
3. Installing the Linux Virtual Machine
Using Ubuntu 24 Desktop as an example, the deployment process is as follows:
- Create a new Linux VM in the host virtualization management platform, select bridged networking, and mount the uploaded system image.
- Install the system according to the wizard. Choose the language you are most comfortable with for later command-line and documentation work.
- After installation, complete the initial setup through the VNC console first.
After first logging into the system, switch to root privileges and install the basic toolchain:
sudo -iapt update && apt install git curl wget vim openssh-server
4. Configuring SSH Remote Management
For service applications, relying on a graphical interface or VNC for the long term is not recommended. Configuring SSH is the foundation for subsequent operations and maintenance. Edit the configuration file:
nano /etc/ssh/sshd_config
Confirm or add the following three lines:
Port 22PermitRootLogin yesPasswordAuthentication yes
Save the file and restart the service:
systemctl restart ssh
You can then log in remotely from a workstation using any SSH client. Security note: If the VM is exposed to the public internet, use key-based authentication, change the default port, and disable root password login. Restrictions can be relaxed appropriately when the VM is used only on an internal network. In addition, the assistant service itself should run as a regular user to avoid the excessive privileges and security risks associated with root.
5. Installing the AI Assistant Service
These open-source services depend on the Node.js runtime. To avoid version conflicts, use a version manager to control the Node.js version:
# Install the Node version managercurl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bashsource ~/.bashrc# Install and enable the LTS versionnvm install 22nvm use 22nvm alias default 22
Then install the service itself globally through npm:
npm install -g <service-package-name>@latest
After installation, three core configuration steps are usually required:
- Model credentials: Configure the API key for the large model in use. It is recommended to inject it through environment variables rather than writing it in plaintext into configuration files.
- Channel access: Bind the Webhook or bot credentials for the target messaging channels in the service configuration. Regional platforms are generally connected through the bot capabilities of their open platforms; generate a token according to the platform documentation and fill it in.
- Persistent operation: Register the service as a systemd service unit or use a process manager such as pm2 to start it automatically on boot, preventing the process from exiting when the SSH session disconnects.
6. Connecting Regional Messaging Channels
When connecting the service to channels such as DingTalk or QQ, several general practices should be considered:
- Callback URL: Most channels require a publicly accessible HTTPS callback URL. For internal deployments, expose the service through a reverse proxy or tunneling tool, and be sure to enable signature verification.
- Message format adaptation: Message body structures vary significantly across channels. The adapter layer must handle differentiated fields such as text, images, and @mentions.
- Rate limiting and retries: Platforms generally impose call-frequency limits. The service should implement exponential backoff retries to prevent message storms that could lead to bans.
- Logging and alerting: Integrate unified logging and monitoring alerts so that channel disconnections or token expiration can be detected promptly.
7. Summary
The overall deployment approach can be summarized as: isolate the runtime environment with a VM → manage remotely over SSH → control the runtime with a version manager → install the service itself → adapt channels and enable persistent operation. Compared with deploying containers directly on the host, the VM approach sacrifices a little resource utilization but provides a cleaner system environment and greater long-term runtime stability. It is well suited to AI assistant services that need to be online 24/7. Later, you can extend the setup as needed with scheduled tasks, knowledge-base retrieval, unified multi-channel management, and other capabilities, gradually turning it into a personal or team automation hub.
