Building My Own Terminal Coding Agent: A 4.6MB, Self-Growing Binary

Anyone who has used a mainstream AI coding tool knows the drill: editor-plugin assistants are feature-rich, but the “heavy” and “picky about environment” problems never go away. Installers hundreds of megabytes in size, a pile of runtimes that must be pre-installed, and a jumble of plugin versions make them hard to pick up and run on demand in isolated networks, remote dev machines, and CI pipelines. So I started asking a question: could a coding agent be like a version-control tool—just a lightweight, standalone command-line program you can copy over and run? This post documents the full process of building a terminal coding agent from scratch in a systems programming language. The final deliverable is a single-file executable, 4.6MB in size, requiring no installation and no runtime dependencies.

1. Why Rebuild a Coding Agent in the Terminal

AI coding assistants generally follow two paths. The first is the editor plugin: intuitive to interact with, but its downside is that the host binds it to a specific development environment. The second is a terminal-native command-line agent that reads and writes code, runs commands, and inspects test results directly in the shell—its capability boundary is essentially the same as what the developer themselves can do in a terminal. The latter has at least three advantages:

1. Environment-agnostic: It does not depend on any particular editor or GUI. A remote server reached over SSH, a machine with no desktop environment, a CI build node—anywhere a binary can run, it can be used.
2. A complete closed loop: Coding is fundamentally a “modify—verify—fix” cycle. A terminal agent can invoke build tools and test frameworks directly, capture real compiler errors and test output, and judge from that whether a change actually works—rather than stopping at “looks correct.”
3. Composable and automatable: Command-line programs are a natural fit for scripting and pipes, and can be embedded into code review, automated fixes, and bulk refactoring workflows.

The weakness of these tools is the information density of the interaction. Rendering long diffs and multi-round reasoning in plain text is tiring, so the terminal rendering layer needs careful design too.

2. Defining Requirements: Deciding What to Deliver Before Writing Code

Before writing any code, I defined the requirements on a single page. Abstract goals cannot be accepted or verified; only measurable goals can guide every subsequent technical decision.

Hard constraints

• Distribution form: a single executable file on Windows—copy and run, no administrator privileges required.
• Runtime dependencies: the target machine has no language runtime or third-party libraries pre-installed.
• Size limit: no more than 10MB, close to the distribution footprint of common command-line tools.

Functional requirements

• Support streaming conversation and multi-round tool calls that can be interrupted and resumed.
• Provide three core tools—code search, file read/write, and command execution—with room for extension.
• Provide cross-session memory, plus the ability to distill successful experiences into reusable “skills.”

Non-functional requirements

• Startup measured in milliseconds; first-byte latency dominated mainly by network delay.
• Every operation with side effects goes through permission confirmation and path validation.
• Logs trace the input and output of every tool call.

This list drove the technology choice directly: to satisfy the first three hard constraints, a systems programming language is almost the only viable option. Its build output is native machine code with no runtime, and after static linking it depends on no system runtime libraries.

3. Overall Architecture: A Layered, Decoupled Agent Core

The system is divided into six layers by responsibility:

┌─────────────────────────────────────────────┐
│ Interaction: arg parsing / TUI rendering /  │
│              interrupt & resume             │
├─────────────────────────────────────────────┤
│ Session: task decomposition / context       │
│          building / budget control          │
├─────────────────────────────────────────────┤
│ Agent loop: reason → tool call → feed back   │
├──────────────┬──────────────┬───────────────┤
│ Tools        │ Memory       │ Model access  │
│ file/search/ │ session/proj/│ protocol      │
│ command/patch│ skills       │ adapt/stream/ │
│              │              │ error handling│
├──────────────┴──────────────┴───────────────┤
│ Infrastructure: config / logging /          │
│                 sandbox / storage           │
└─────────────────────────────────────────────┘

In practice, organize the code around “single-responsibility modules with clear interfaces.” For example, every implementation in the tool layer satisfies the same interface:

trait Tool {
    fn name(&self) -> &str;
    fn schema(&self) -> JsonSchema;          // for the model to understand parameter structure
    fn requires_confirmation(&self) -> bool; // write operations require user confirmation
    fn execute(&self, args: Json, ctx: &Ctx) -> ToolResult;
}

This has two benefits. First, the model-side tool definition and the actual execution logic share the same schema, avoiding parameter misalignment caused by the two drifting apart. Second, adding a new tool only requires implementing the interface and registering it—no changes to the main loop. Cross-cutting logic such as permission checks, timeout control, and result truncation is handled centrally in the dispatch layer, so no individual tool implementation can skip a safety measure.

4. Model Access and Streaming Parsing: Getting the Details Right

Model capability sets the agent’s ceiling; access quality sets its floor. This part is the most easily underestimated and the most prone to hidden bugs during development.

4.1 Transport Layer: Parsing the Event Stream Byte by Byte

Mainstream model services generally offer an SSE (Server-Sent Events) style streaming HTTP interface. The server pushes events one by one over a persistent long connection; each event ends with a blank line, and the payload is usually wrapped in JSON. It looks simple, but the implementation must handle several edge cases:

• Partial frames and coalesced frames: The byte stream returned by network reads does not align with event boundaries. A single event may be split across multiple reads, and multiple events may arrive in one read. The correct approach is to search for the blank-line delimiter at the byte-increment level, rather than assuming “one read equals one line.”
• Incomplete JSON: At the end of a read, the buffer often ends with a JSON object that was only half received. You must first determine whether an event is complete before handing it to the parser.
• Varied payload shapes: Beyond text increments, there may be tool-call argument increments, finish reasons, usage statistics, and other event types, which must be dispatched by type.
• Heartbeats and termination: When the connection is idle, the server may send only comment lines that should be ignored; only an explicit end event or a clean connection close should be treated as the completion of the response.

4.2 Incremental JSON Assembly

Tool-call arguments are often pushed as string fragments across multiple events—for example, a parameter value arriving in two pieces as "src/mai" and "n.rs". Parsing directly on receipt is guaranteed to fail. The reliable approach is:

1. Maintain a separate argument buffer for each tool call, accumulating fragments by index;
2. Parse the whole thing only after the end event arrives;
3. On parse failure, attempt a one-time tolerant repair of unclosed quotes and brackets; if the repair fails, feed the raw content back to the model as an error and have it regenerate.

4.3 Interruptible and Resumable

The longer a streaming output runs, the more likely the user is to interrupt it. Ctrl+C does not have fully consistent semantics across Windows and other platforms and must be handled separately: on receiving an interrupt signal, first close the HTTP connection to stop the server from continuing to bill and hold resources, then output the partial content already generated. For long tasks, completed tool calls can also be written to a local log, so a resumed session can continue from the breakpoint.

4.4 Error Classification and Retries

There are many reasons a network request can fail, and retrying everything blindly amplifies the problem. Handle by status code:

Error type	Typical scenario	Handling strategy
Authentication failure	Invalid or expired key	Terminate immediately, prompt to check configuration
Bad request	Context over limit, malformed parameters	Don’t retry blindly; trim context or fix parameters first
Rate limit	Rate limit triggered	Exponential backoff, then retry
Server error	Temporary model-service failure	Limited retries; if still failing, fall back with a message
Network fault	Timeout, connection drop	Distinguish connect timeout from read timeout, with separate thresholds

5. The Tool-Calling Loop: The Agent’s Heart

5.1 Main Loop Structure

The core of the agent is a “request—execute—feed back” loop: send the context and the list of available tools to the model; the model returns either text or tool-call requests; the system executes the tools, appends the results to the context, and requests the model again. As long as the model keeps requesting tools, the loop continues; when the model returns plain text, it considers the task complete. Pseudocode:

loop {
    response = model.stream(messages, tools)   // streaming output, interruptible
    if response.tool_calls.is_empty():
        return response.text                   // task finished
    for call in response.tool_calls:
        result = executor.run(call)            // permission check, timeout, path validation
        messages.append(tool_result(call.id, result))
    if budget_exceeded() or no_progress(3):
        return summarize_and_stop()
}

5.2 Tool Design

The three core tools each have key design points:

• Code search: A large model’s context window is limited, and you cannot stuff an entire repository into it. A search tool should first do keyword matching, then combine it with syntax-symbol information to locate code, optionally introduce a vector index for semantic recall, and finally sort by relevance and return only a small number of highly relevant snippets.
• File editing: The worst thing to do is a “full-file rewrite.” A single memory slip from the model will overwrite unrelated code. A more robust approach is structured patches: have the model provide the original snippet to be replaced and the replacement content, and have the program locate it precisely in the file and verify uniqueness. Matching in multiple places means the context is insufficient and should raise an error so the model can add more; matching nowhere means the model misremembered the original, and the real snippet should be fed back to it.
• Command execution: You must set timeouts and resource limits to prevent a build or test process from hanging the entire session. Output should be truncated intelligently: keep the beginning and the end, because compiler errors usually appear at the end. Exit codes and key error messages should be passed back verbatim so the model can fix accordingly.

5.3 Termination Conditions and Infinite-Loop Prevention

Relying solely on the model to “know when to stop” is unreliable, so hard backstops are essential:

• Step limit: A single task executes a fixed maximum number of tool-call rounds;
• Time budget: Count from task start; force a wrap-up on timeout;
• No-progress detection: If several consecutive rounds of running the same tool return the same error, or several consecutive rounds of edits still fail tests, interrupt immediately and report the current state to the user rather than continuing to burn tokens.

5.4 Self-Check and Self-Correction

“Self-correction” doesn’t have to rely on complex reinforcement learning. The most effective and lowest-cost signal is the test results themselves. A complete task flow can be designed as:

1. Read the task description and relevant code;
2. Form a modification plan;
3. Execute the changes;
4. Run the build and tests;
5. If they fail, feed a summary of the errors back to the model and re-reason;
6. If consecutive failures exceed a threshold, roll back this round of changes and report the reason.

To make step 5 efficient, the tool layer should provide a “minimal verification” capability: run only the test cases relevant to the current change, rather than the full test suite every time.

6. Context Engineering: Fitting the Most Valuable Information into a Limited Window

The context window is a hard constraint shared by all coding agents. What really determines effectiveness is not how much the model can read, but what it actually reads at each step. I use layered budget management:

Context block	Budget share (illustrative)	Content
System prompt	~10%	Role definition, behavior rules, tool-use principles
Project memory	~10%	Tech stack, directory conventions, coding standards
Skill library	~10%	Historical experience matching the current task
Retrieved snippets	~30%	Code and docs relevant to the task
Conversation history	~30%	Multi-round reasoning and tool results
Reserve margin	~10%	Model output and temporary fluctuation

When history approaches the budget ceiling, compress in this order:

1. Tool-result truncation: Keep only the task-relevant parts of large file contents; keep the head and tail of long output.
2. Rolling summary: Compress the earliest multi-round conversation into a structured summary containing the goal, changes completed, conclusions confirmed, and open questions.
3. Subtask isolation: For exploratory tasks like “scan the whole repository to find all call sites,” dispatch an independent sub-agent. The sub-agent has its own context and returns only a summary of its conclusions to the main thread. This both saves the main context and prevents intermediate steps from interfering with the main reasoning.

It’s worth stressing that “confirmed conclusions” in a summary must be clearly distinguished from “unverified speculation.” Otherwise the model will treat an early wrong judgment as fact and keep reasoning on top of it, amplifying the error layer by layer.

7. Self-Growth: Making the Agent Understand the Project Better Over Time

This is the biggest difference between this project and a traditional coding assistant. The “growth” here does not mean changing model parameters, but letting the system accumulate organizational knowledge through interaction with the developer and actively call on it in later tasks. The overall approach borrows from the cognitive-science distinction between “declarative memory + procedural memory”: the former answers “what is this project,” the latter answers “how do I solve this kind of problem.”

7.1 A Three-Tier Memory Model

Tier	Lifecycle	Stored content	Typical example
Session memory	Single session	Conversation history, tool results, task state	Which files were changed this round
Project memory	Long-term, travels with the repo	Architecture conventions, coding standards, common pitfalls	“Put all config in the config directory; no hardcoding”
Skill memory	Long-term, cross-project	Reusable task-solving procedures	“Steps to fix a certain class of build error”

For storage, there’s no need to bring in a heavyweight database. In small- and medium-scale scenarios, a local embedded file database is enough to support structured queries and full-text search; skill entries are stored in structured text files for easy human review and version control. The key principle is just one: memory must be auditable. Every memory entry must be traceable to its source, and the developer must be able to view, edit, or delete it at any time.

7.2 When to Write to Memory

The write policy determines the quality of the memory store. Too aggressive and it accumulates noise; too conservative and it loses the ability to grow. I use a dual gate of “trigger condition + confidence score”:

{
  "id": "skill-0001",
  "trigger": ["build failure", "missing system library"],
  "context": "Compiling a project with native dependencies in an environment lacking a C toolchain",
  "steps": [
    "Check whether the target platform has a C compiler and linker installed",
    "Confirm the cross-compilation target matches the prebuilt artifacts of the dependencies",
    "Re-run the build and verify the output"
  ],
  "evidence": "Fixed in one pass following this procedure in session #42",
  "confidence": 0.8,
  "use_count": 3,
  "success_count": 3
}

Write only when all of the following hold simultaneously: the task is judged clearly successful (e.g., all tests pass); the solution involved a non-obvious judgment; and no similar experience already exists, or the new experience is clearly better than the old one. Deduplication and conflict resolution are required before writing: merge rather than add when an entry is highly similar to an existing one; when conclusions conflict, keep the higher-confidence one and mark the old one for review.

7.3 How to Use It and How to Prevent Contamination

At the start of each new task, the system retrieves the skill library and project memory using the current task description and injects the Top-K results into the system prompt. Retrieval uses a hybrid of keyword and semantic recall, then ranks by relevance and confidence combined.

The thing the memory store fears most is “wrong experience reinforcing itself.” To guard against this, there are three lines of defense:

• Record the outcome every time a skill is invoked; lower its confidence on failure, and auto-archive it when confidence drops below a threshold.
• Periodically replay the skill library against a regression task set to verify it still works.
• Provide an explicit “forget” command so the developer can manually clear outdated experience.

7.4 From Skill Accumulation to Capability Evolution

Once the skill library reaches a certain scale, further offline optimization becomes possible: organize historical sessions into a “task—trajectory—result” dataset, and periodically evaluate context-building strategies, tool-description wording, and termination-condition thresholds. Whichever version has a higher success rate on the same task set is the one adopted. This is essentially iterating on the agent’s “program” without training the model—the cost is controllable and the effect is measurable.

8. How the 4.6MB Was Achieved

Back to the size question from the beginning. Squeezing a single executable down to 4.6MB isn’t about one trick—it’s about systematic engineering tradeoffs.

8.1 Compiler and Linker Optimization

• Link-Time Optimization (LTO): Defer cross-compilation-unit optimization to the link stage so the compiler sees the full call graph, eliminating more dead code and letting inlining and constant propagation do more.
• Size-first optimization level: Beyond speed, the compiler can be set to optimize for size, actively avoiding the code bloat introduced when chasing peak performance.
• Single codegen unit: Reducing parallel splitting during code generation gives the optimizer a larger global view and lowers the overhead of redundant generation.
• Abort over unwind: Error handling defaults to an “abort terminates” strategy, generating no full stack-unwinding metadata; only paths that genuinely need to catch exceptions and recover enable unwinding locally.
• Strip debug symbols: Release builds remove the symbol table and debug info. Build number and version info are retained to help locate production issues.

Stacked together, these typically reduce the size to a fraction of a normal debug build.

8.2 Dependency Governance

Dependencies are the biggest variable in size. The principle is “don’t use one if you can avoid it; pick a lightweight implementation if you must use one”:

• The terminal renderer implements only the necessary cursor control and incremental refresh, without pulling in a full terminal-UI framework.
• Streaming parsing only needs to search for delimiters at the byte-increment level, with no need for a full serialization ecosystem.
• If an async runtime is used only for a small number of concurrent HTTP requests and subprocess management, choose a componentized, streamlined implementation and enable features on demand.
• Use a script to periodically review the dependency tree for build-dependency management and remove libraries no longer in use.

8.3 Resource Embedding and On-Demand Loading

Text resources such as system-prompt templates, tool descriptions, and help docs are embedded in the binary at compile time and read directly at runtime. The benefit is a single file at distribution time and no file-IO overhead at startup. Optional capabilities like syntax highlighting and code completion are designed as separate extension processes, downloaded and invoked on demand, so the base package doesn’t keep growing.

9. Security and Observability: Non-Negotiable Engineering Basics

An agent can execute commands and modify files; the greater the capability, the higher the risk. Security design must be pushed forward into the tool layer, not patched on afterward.

• Path validation: All file operations are confined to the project root. User input and model-provided paths are normalized uniformly, then checked for boundary violations; symlink bypass is also guarded against.
• Command allowlist and confirmation: High-risk commands (delete, format, network push) are blocked by default and require explicit user confirmation; read-only commands (checking status, running tests) can run automatically.
• Key isolation: Model keys are read from environment variables or the system keystore, never written to config files, never entered into logs, and never placed in command-line arguments.
• Operation auditing: Every tool call records time, a parameter summary, result status, and duration for later review. Logs are output by level—concise during normal operation, with a verbose mode available when troubleshooting.
• Rollback capability: Before batch changes, a local checkpoint is created automatically so the user can restore to the pre-change state with one click.

For enterprise adoption, this can also be combined with common cloud infrastructure: keys managed by a unified credential-management service; build artifacts and the skill library archived to object storage; and runtime logs routed to a logging service and monitoring alerts for timely detection of anomalous calls.

10. Iteration Path and Lessons Learned

This project didn’t come together in one shot; it advanced in three phases:

1. Usable phase: Wire up the main “conversation—tool call—file modification” path and validate that the core loop works;
2. Good-to-use phase: Refine context compression, retrieval accuracy, interrupt/resume, and error handling to improve the success rate of long tasks;
3. Growth phase: Introduce memory and the skill library, build an offline evaluation set, and make improvements quantifiable and reproducible.

Looking back at the whole process, a few lessons are worth sharing:

• Get the closed loop working first, then talk optimization. The tool-calling loop is the skeleton; everything else is an enhancement. If the skeleton is unstable, any optimization is a castle in the air.
• Edge cases determine stability. Details like streaming parsing, incremental JSON, and interrupt handling took up a lot of development time but determine whether the product can be trusted long-term.
• Self-growth must be auditable. Memory you can’t trace is a liability, not an asset; confidence and forgetting mechanisms must be designed from the start.
• Size is a design constraint, not an afterthought. If heavyweight dependencies are allowed in from the beginning, it’s nearly impossible to squeeze the binary back down to single-digit megabytes later.
• Security is a special capability requirement for agents. When an ordinary command-line tool errs, it just reports an error; when an agent errs, it can directly wreck the codebase—so permission and rollback mechanisms must come first.

Terminal-native AI coding agents still have obvious shortcomings: their interaction information density is lower than a GUI, and their overall understanding of complex codebases is still limited by context. But their “light, fast, composable” characteristics make them a valuable complement to graphical tools. For developers who want a controllable environment and frequently work on servers and in CI, the real-world efficiency of this kind of tool is often higher than expected.