Building a Coding Agent from Scratch: Setup, Custom Model Endpoints, and Automated Workflows

    A practical guide to installing a coding agent CLI, connecting custom or local model APIs (direct, routing, and aggregation), extending it with MCP-style tools, skills, and project memory files, and closing the loop with browser and device automation.

    Why coding agents matter now

    About six months ago, a widely tested view was that model quality was still catching up, but agent-style tools were already worth learning—when models improved, people who already knew the tooling would capture the gains first. That view is playing out.

    Agent impact on engineering now shows up at three levels:

    • Code: reading and writing code, running tests, fixing bugs, and refactoring have moved from autocomplete-style help to multi-step autonomous work.
    • Process: requirement breakdown, doc lookup, environment setup, and release checks are increasingly scripted and automated.
    • Mindset: the question shifts from “how do I use this tool?” to “can this system plug into an agent?” and “can this workflow be automated?”

    A practical takeaway: developers who have not started should start soon. Not because the tools are perfect, but because many steps that still need humans will quietly disappear, and the experience gap will widen.

    Three layers of automation: search, web, device

    Before install steps, one judgment runs through the rest of this article:

    How far an agent can automate depends on what it can see and touch.

    Treat capability as three stacked layers:

    Layer Capability Typical implementation If missing
    Information Search and fetch pages Built-in search, generic web-fetch extensions Stuck on training data; cannot answer “latest” or “right now”
    UI Open pages, click, fill forms, screenshot Browser automation frameworks Cannot operate web systems without APIs; E2E checks break
    Device Install, launch, tap, read logs Mobile debug bridges Cannot verify real-device behavior; mobile automation never closes

    Together they form a loop:

    1. Search (latest docs and error fixes)
    2. ↓
    3. Web / device (clicks, forms, installs)
    4. ↓
    5. Screenshots / logs / network (verify real outcomes)
    6. ↓
    7. Find issues → search again → fix again

    Section 7 focuses on the UI and device layers.

    Install and environment

    Prerequisites

    Using a real local baseline (Windows 11 + Git Bash + a recent JavaScript runtime). Follow official docs and your model provider console for details:

    1. Runtime: install a recent JavaScript runtime (LTS or newer). node -v should print a version.
    2. Terminal: on Windows, prefer Git Bash or Windows Terminal; older cmd often mishandles interactive UIs.
    3. Network: you must reach the model endpoint (especially important when pointing at a custom provider).

    Install and verify

    Install the CLI globally via a package manager:

    1. npm install -g <agent-cli>
    2. # verify
    3. <agent-cli> --version
    4. <agent-cli> doctor # if a self-check command exists, run it first

    In any project directory, start the interactive session. Success is seeing the chat UI. On first run, do a minimal check: ask it to list the current directory and summarize what the project is for, to confirm the model call path works.

    Three ways to connect custom model APIs

    By default these tools often bind to a hosted overseas model service. Connecting another provider’s API is a common need. Three patterns have clear trade-offs.

    Direct connection

    Put the API key and endpoint in environment variables or tool config:

    1. export ANTHROPIC_BASE_URL=<compatible endpoint>
    2. export ANTHROPIC_AUTH_TOKEN=<your key; keep it secret>
    • Pros: shortest path, lowest latency, no extra hop, billing is straightforward.
    • Cons: switching models means editing config; keys scatter and are harder to manage.
    • Fit: individuals who stay on one provider and want low latency.

    Routing

    A local switcher or gateway forwards requests to different endpoints by rule. A config file holds named endpoint/key sets; one command switches models.

    • Pros: fast multi-model switching; easy A/B comparison and per-task choice (e.g. coding vs docs).
    • Cons: extra maintenance.
    • Fit: teams and heavy users evaluating several models or splitting work by task.

    Aggregation

    An aggregator proxies many models behind one key and bills by usage.

    • Pros: centralized keys, unified usage reports, easy to add models.
    • Cons: third-party reliability, possible markup or rate limits.
    • Fit: users who do not want to register with every vendor and want usage-based spend.

    How to choose: start with direct connection until the loop works; add routing when you compare models or split tasks; consider aggregation when you need multi-account, team-scale key management.

    Turning the agent into a personal workbench

    Extension protocol (MCP-style)

    Most agents can attach external tool servers via a standard protocol (commonly MCP). After you register servers in config, the agent can call extra capabilities: browser automation, DB queries, filesystem extras, and so on. Essentials:

    • Register start command and args in global or project config.
    • Verify with a small task and watch tool-call logs in the session.
    • Least privilege: only attach tools the current task needs.

    Skills

    Reusable flows (e.g. “create a module per team conventions” or “draft a release note”) can live as skill files: trigger conditions plus steps. When intent matches, the agent loads them—personal practice turned into reusable assets.

    Project memory files (CLAUDE.md-style)

    At the repo root, keep a memory file with:

    • stack and directory conventions;
    • build, test, and lint commands;
    • coding rules and “do not” items (e.g. never auto-edit a given config).

    The agent reads it at session start, so you repeat less. Put it in version control so the team shares one “agent behavior spec.”

    Daily high-leverage habits and scheduled automation

    Highest-ROI habits:

    • In-session shortcuts: clear screen, replay history, compress context. Compress long chats to save tokens.
    • Non-interactive mode: agent -p "task description" so you can treat the agent as a programmable CLI in pipelines.
    • Scheduled loops: built-in loop or system cron for jobs like “run tests every morning and summarize failures” or “check dependency updates hourly and write a report.”

    Example:

    1. # Daily: analyze yesterday’s build log and write a summary
    2. agent -p "Read build.log, summarize failing cases and likely causes, write report.md"

    Closing the loop: web and device

    Web automation

    After attaching a browser automation server via MCP, you can say: open the admin page, log in, export yesterday’s orders. The agent opens a browser, finds elements, submits forms, and screenshots to confirm. Typical uses:

    • Internal web apps with no public API.
    • E2E regression: submit a form, screenshot, assert page state.

    Device automation

    After attaching a mobile debug bridge (generic ADB-style protocol) via MCP, the agent can install an APK, launch the app, tap and swipe, and pull logcat. Mobile testing shifts from manual tapping to “state the intent, execute, collect logs.”

    Dual-end loop: intent only, model plans the path

    When information, UI, and device layers are all connected, you describe intent and the model plans:

    Example: “Verify login on the new app build.” The agent can unpack: install → launch → tap login → fill credentials → screenshot → read logs → judge pass/fail → on failure, search the error and suggest a fix. None of those steps has to be a pre-written script.

    Closing: stay close to the capability edge

    Agent value is not only how strong it is today, but that it improves weekly, while working smoothly with it takes time. Three actions:

    1. Get a minimal loop running: install → connect one custom model → finish one real task (doable in a day).
    2. Accumulate assets: memory files, skills, and scripted jobs compound.
    3. Work at the edge: periodically rerun old tasks on newer models to feel the boundary move—that is both technical awareness and a career moat.

    The three automation layers are already available. The remaining work is to use them.