Building a Coding Agent from Scratch: Setup, Custom Model Endpoints, and Automated Workflows
A practical guide to installing a coding agent CLI, connecting custom or local model APIs (direct, routing, and aggregation), extending it with MCP-style tools, skills, and project memory files, and closing the loop with browser and device automation.
Why coding agents matter now
About six months ago, a widely tested view was that model quality was still catching up, but agent-style tools were already worth learning—when models improved, people who already knew the tooling would capture the gains first. That view is playing out.
Agent impact on engineering now shows up at three levels:
- Code: reading and writing code, running tests, fixing bugs, and refactoring have moved from autocomplete-style help to multi-step autonomous work.
- Process: requirement breakdown, doc lookup, environment setup, and release checks are increasingly scripted and automated.
- Mindset: the question shifts from “how do I use this tool?” to “can this system plug into an agent?” and “can this workflow be automated?”
A practical takeaway: developers who have not started should start soon. Not because the tools are perfect, but because many steps that still need humans will quietly disappear, and the experience gap will widen.
Three layers of automation: search, web, device
Before install steps, one judgment runs through the rest of this article:
How far an agent can automate depends on what it can see and touch.
Treat capability as three stacked layers:
| Layer | Capability | Typical implementation | If missing |
|---|---|---|---|
| Information | Search and fetch pages | Built-in search, generic web-fetch extensions | Stuck on training data; cannot answer “latest” or “right now” |
| UI | Open pages, click, fill forms, screenshot | Browser automation frameworks | Cannot operate web systems without APIs; E2E checks break |
| Device | Install, launch, tap, read logs | Mobile debug bridges | Cannot verify real-device behavior; mobile automation never closes |
Together they form a loop:
Search (latest docs and error fixes)↓Web / device (clicks, forms, installs)↓Screenshots / logs / network (verify real outcomes)↓Find issues → search again → fix again
Section 7 focuses on the UI and device layers.
Install and environment
Prerequisites
Using a real local baseline (Windows 11 + Git Bash + a recent JavaScript runtime). Follow official docs and your model provider console for details:
- Runtime: install a recent JavaScript runtime (LTS or newer).
node -vshould print a version. - Terminal: on Windows, prefer Git Bash or Windows Terminal; older cmd often mishandles interactive UIs.
- Network: you must reach the model endpoint (especially important when pointing at a custom provider).
Install and verify
Install the CLI globally via a package manager:
npm install -g <agent-cli># verify<agent-cli> --version<agent-cli> doctor # if a self-check command exists, run it first
In any project directory, start the interactive session. Success is seeing the chat UI. On first run, do a minimal check: ask it to list the current directory and summarize what the project is for, to confirm the model call path works.
Three ways to connect custom model APIs
By default these tools often bind to a hosted overseas model service. Connecting another provider’s API is a common need. Three patterns have clear trade-offs.
Direct connection
Put the API key and endpoint in environment variables or tool config:
export ANTHROPIC_BASE_URL=<compatible endpoint>export ANTHROPIC_AUTH_TOKEN=<your key; keep it secret>
- Pros: shortest path, lowest latency, no extra hop, billing is straightforward.
- Cons: switching models means editing config; keys scatter and are harder to manage.
- Fit: individuals who stay on one provider and want low latency.
Routing
A local switcher or gateway forwards requests to different endpoints by rule. A config file holds named endpoint/key sets; one command switches models.
- Pros: fast multi-model switching; easy A/B comparison and per-task choice (e.g. coding vs docs).
- Cons: extra maintenance.
- Fit: teams and heavy users evaluating several models or splitting work by task.
Aggregation
An aggregator proxies many models behind one key and bills by usage.
- Pros: centralized keys, unified usage reports, easy to add models.
- Cons: third-party reliability, possible markup or rate limits.
- Fit: users who do not want to register with every vendor and want usage-based spend.
How to choose: start with direct connection until the loop works; add routing when you compare models or split tasks; consider aggregation when you need multi-account, team-scale key management.
Turning the agent into a personal workbench
Extension protocol (MCP-style)
Most agents can attach external tool servers via a standard protocol (commonly MCP). After you register servers in config, the agent can call extra capabilities: browser automation, DB queries, filesystem extras, and so on. Essentials:
- Register start command and args in global or project config.
- Verify with a small task and watch tool-call logs in the session.
- Least privilege: only attach tools the current task needs.
Skills
Reusable flows (e.g. “create a module per team conventions” or “draft a release note”) can live as skill files: trigger conditions plus steps. When intent matches, the agent loads them—personal practice turned into reusable assets.
Project memory files (CLAUDE.md-style)
At the repo root, keep a memory file with:
- stack and directory conventions;
- build, test, and lint commands;
- coding rules and “do not” items (e.g. never auto-edit a given config).
The agent reads it at session start, so you repeat less. Put it in version control so the team shares one “agent behavior spec.”
Daily high-leverage habits and scheduled automation
Highest-ROI habits:
- In-session shortcuts: clear screen, replay history, compress context. Compress long chats to save tokens.
- Non-interactive mode:
agent -p "task description"so you can treat the agent as a programmable CLI in pipelines. - Scheduled loops: built-in loop or system cron for jobs like “run tests every morning and summarize failures” or “check dependency updates hourly and write a report.”
Example:
# Daily: analyze yesterday’s build log and write a summaryagent -p "Read build.log, summarize failing cases and likely causes, write report.md"
Closing the loop: web and device
Web automation
After attaching a browser automation server via MCP, you can say: open the admin page, log in, export yesterday’s orders. The agent opens a browser, finds elements, submits forms, and screenshots to confirm. Typical uses:
- Internal web apps with no public API.
- E2E regression: submit a form, screenshot, assert page state.
Device automation
After attaching a mobile debug bridge (generic ADB-style protocol) via MCP, the agent can install an APK, launch the app, tap and swipe, and pull logcat. Mobile testing shifts from manual tapping to “state the intent, execute, collect logs.”
Dual-end loop: intent only, model plans the path
When information, UI, and device layers are all connected, you describe intent and the model plans:
Example: “Verify login on the new app build.” The agent can unpack: install → launch → tap login → fill credentials → screenshot → read logs → judge pass/fail → on failure, search the error and suggest a fix. None of those steps has to be a pre-written script.
Closing: stay close to the capability edge
Agent value is not only how strong it is today, but that it improves weekly, while working smoothly with it takes time. Three actions:
- Get a minimal loop running: install → connect one custom model → finish one real task (doable in a day).
- Accumulate assets: memory files, skills, and scripted jobs compound.
- Work at the edge: periodically rerun old tasks on newer models to feel the boundary move—that is both technical awareness and a career moat.
The three automation layers are already available. The remaining work is to use them.
