Deploying AI Inference Platform with Large Models on Ubuntu 24.04: A Complete Guide
This guide provides step-by-step instructions for deploying an AI inference platform on Ubuntu 24.04 and integrating large language models, covering system requirements, environment setup, security configurations, and performance optimization.
System Requirements and Environment Setup
Hardware Recommendations
- Minimum: 4-core CPU with 8GB RAM
- Recommended for high concurrency: 16GB RAM
- Disk space: ≥20GB for dependencies and model files
System Initialization
# Update system packagessudo apt update && sudo apt upgrade -y# Install development toolssudo apt install -y build-essential curl git# Configure firewall (open essential ports only)sudo ufw allow 22/tcp # SSHsudo ufw allow 3000/tcp # Default web portsudo ufw enable
User Management
# Create dedicated service usersudo adduser ai-servicesudo usermod -aG sudo ai-servicesu - ai-service
Node.js Environment Configuration
Version Management with NVM
# Install NVMcurl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bashsource ~/.bashrc# Verify installationcommand -v nvm# Install LTS versionnvm install 20nvm alias default 20node -v # Should show v20.x.x
Version Management Best Practices
- Production: Use even-numbered LTS versions (e.g., v20.x)
- Development: Track latest versions
- Project-specific versions: Use
.nvmrcfiles
AI Inference Platform Installation
Global Installation
# Install latest versionnpm install -g openclaw@latest --unsafe-perm# Verify installationopenclaw --version
Troubleshooting Common Issues
Directory Conflicts:
# Clean residual filesrm -rf ~/.npm-global/lib/node_modules/openclawnpm cache clean --force# Reinstallnpm install -g openclaw@latest
Permission Errors:
- Fix ownership:
sudo chown -R $USER ~/.npm
- Or configure non-root npm:
mkdir ~/.npm-globalnpm config set prefix '~/.npm-global'echo 'export PATH=~/.npm-global/bin:$PATH' >> ~/.bashrcsource ~/.bashrc
Large Model Integration
Model Service Configuration
- Obtain API key from model provider
- Verify service access permissions
- Record service endpoint URL
Interactive Configuration
openclaw config
Follow prompts to configure:
- Provider type: Select “Custom Provider”
- Connection parameters:
- Name: Custom identifier (e.g., zhipu-glm)
- Endpoint:
https://api.example.com/v4(example) - API Key: Provider-issued authentication key
Configuration Validation
Check ~/.openclaw/config.json for:
{"providers": {"zhipu-glm": {"type": "custom","endpoint": "https://api.example.com/v4","apiKey": "sk-xxxxxxxxxxxxxxxx"}}}
Service Deployment and Security
Development Server
openclaw serve --port 3000
Verify with:
curl http://localhost:3000/health# Should return {"status":"ok"}
Production Deployment
Process management with PM2:
npm install -g pm2pm2 start `which openclaw` --name ai-service -- serve --port 3000pm2 savepm2 startup
Nginx reverse proxy configuration:
server {listen 80;server_name ai.example.com;location / {proxy_pass http://localhost:3000;proxy_set_header Host $host;proxy_set_header X-Real-IP $remote_addr;}}
Secure Access Options
SSH Tunneling:
# Local tunnel setupssh -L 3000:localhost:3000 user@server-ip# Browser accesshttp://localhost:3000
VPN Access:
- Deploy WireGuard VPN
- Configure firewall to allow 3000 only from VPN subnet
- Access services through VPN connection
Performance Optimization
Monitoring Setup
# Install monitoring toolssudo apt install -y htop sysstat# Configure Node.js monitoringnpm install -g pm2-logrotatepm2 set pm2-logrotate:max_size 10M
Performance Tuning
Adjust Node.js memory:
export NODE_OPTIONS="--max-old-space-size=4096"
Implement rate limiting:
{"rateLimit": {"max": 100,"windowMs": 60000}}
Troubleshooting Guide
Common Error Codes
| Code | Cause | Solution |
|---|---|---|
| 502 | Service down | Check service status/logs |
| 403 | Auth failed | Verify API key validity |
| 504 | Timeout | Adjust model timeout settings |
Log Analysis
Service logs:
pm2 logs ai-service
Enable debug logging:
export DEBUG=openclaw:*openclaw serve
System logs:
journalctl -u nginx --no-pager -n 100
Advanced Features
Multi-Model Support
Update configuration:
{"providers": {"model-a": {...},"model-b": {...}}}
Custom Routing
Create routes.js:
module.exports = (app) => {app.get('/custom', (req, res) => {res.send('Custom endpoint');});};
Start with:
openclaw serve --routes ./routes.js
Authentication Integration
- Install Passport.js:
npm install passport passport-jwt
- Configure JWT middleware
Maintenance and Upgrades
Version Upgrades
# Check for updatesnpm outdated -g openclaw# Perform upgradenpm update -g openclaw# Verify versionopenclaw --version
Backup Strategy
Configuration backup:
tar czvf openclaw-config-$(date +%Y%m%d).tar.gz ~/.openclaw/
Database backup (if applicable):
```bashLocate database file
find / -name “openclaw.db” 2>/dev/null
Perform backup
cp /path/to/openclaw.db /backups/
```
Best Practices Summary
- Security First: Always access admin interfaces through VPN/SSH tunnels
- Resource Isolation: Create separate users for different model services
- Monitoring: Configure alerts for critical metrics
- Maintenance: Establish weekly maintenance windows
- Documentation: Maintain detailed deployment records
This standardized approach enables developers to quickly deploy secure, production-ready AI inference platforms on Ubuntu 24.04 while maintaining flexibility for customization. The solution balances development convenience with production stability requirements.
