First Principles of Coding Agents: From Mechanism to Design (Part 1)
This article starts from autoregressive generation—the core mechanism of large language models—and derives three structural traits: generation is reasoning, the context window is the only memory, and outputs are probabilistic. It then shows how those traits are amplified in long, tightly constrained software tasks, and what that implies for coding-agent design.
1. Why a First-Principles View Matters
Intelligent coding tools are advancing quickly, yet much of the discussion stays at the level of “it works” or “it feels good,” without asking how the system actually operates. To understand why a coding agent succeeds on some tasks and fails repeatedly on others, we have to return to a more basic question: how does a large language model work?
A first-principles approach avoids analogy and intuition. It starts from the model’s lowest-level behavior and works upward to the phenomena we observe. Only then can we explain why “write an entire project” is so much harder than “fix one function,” and only then can we design an agent architecture that compensates for the model’s built-in limits.
2. The Core Mechanism of an LLM: Autoregressive Generation
2.1 In One Sentence
A large language model works by predicting the next token, one at a time.
Given an input, the model does not first draft a complete answer internally and then emit it all at once. The real process is: using all text already available—the user input plus everything the model has already produced—it computes a probability distribution over the next token, samples one token, appends it to the sequence, and repeats until a stop condition is reached. This is autoregressive generation.
In pseudocode:
def generate(prompt, max_tokens):sequence = tokenize(prompt)while not finished(sequence) and len(sequence) < max_tokens:probs = model(sequence) # next-token distribution over the full sequencenext_token = sample(probs) # sample according to the distributionsequence.append(next_token) # append; this becomes input for the next stepreturn detokenize(sequence)
The loop looks simple, but it shapes every form of “intelligence” the model can show.
2.2 Three Structural Consequences
Trait 1: There is no thinking process separate from the output.
For the model, “reasoning” and “writing” are the same act. It cannot fully decide what to say and then say it; it thinks by generating. That is why explicitly asking the model to “think step by step”—the chain-of-thought pattern—often improves performance on hard tasks. Intermediate text is itself the reasoning process: each generated step becomes context for the next prediction, giving the model a longer working draft. Even newer models with explicit reasoning still follow “generation is thinking”; they mainly place the intermediate steps in a more structured channel.
Trait 2: The context window is the entire memory.
The model has no separate persistent store. Everything it “knows” about the current conversation comes from the context window of this request. Anything outside that window is, for the model, nonexistent. Two implications follow:
- Anything that must be remembered—past decisions, code structure, mistakes already made—has to be managed by an external system and re-injected into the context at the right time.
- As the context grows longer, information density can drop. Attention to material in the middle is often weaker than attention to the beginning and the end, a common failure mode in long-context settings.
Trait 3: Generation is probabilistic.
Each step samples from a distribution, so the same input can yield different outputs. An agent therefore cannot treat reliability as “the model gets it right every time.” External verification has to catch mistakes.
3. Limits of Autoregression in Coding
Autoregressive generation is powerful: it is general, scalable, and able to produce arbitrarily complex structured text. Its limits come from the same mechanism, and coding agents amplify them. Software work is a long-horizon, tightly constrained, strongly verifiable task:
- Long horizon: a real task often needs dozens or hundreds of edit, run, and debug cycles.
- Tight constraints: code must compile, pass tests, and satisfy type checks. “Close enough” is not enough.
- Strong verification: correctness can be judged objectively, and an error at one step is magnified downstream.
Three problems stand out.
3.1 Local Optima: Continuing Token by Token Is Not Global Planning
Autoregressive generation is local. At each step the model optimizes the probability of the next token. There is no separate global planner coordinating the whole task. When a job requires designing an architecture, splitting modules, and then implementing them, the model may start “continuing” code before it has planned enough, following a path that is locally plausible and globally wrong.
More importantly, the model has no direct sense of whether the result will pass tests or a continuous-integration pipeline unless those signals—test output, compiler errors, and the like—are fed back into the agent loop. That is why mature agent designs insist on a verification loop: act, observe, correct—not treat a single generation as done.
3.2 Context Limits: The Codebase Is Larger Than the Window
Real projects are usually far larger than the context window. The agent cannot “see” the whole repository. It depends on retrieval, indexes, and on-demand reads to assemble what the model should look at. How well that information is organized determines whether the model’s view of the world is complete, and therefore how good its decisions are.
3.3 Error Accumulation: The Cost of Probability over a Long Chain
Probabilistic generation means every step can fail, and a long chain compounds the per-step error rate. If each stage is 90% reliable, twenty stages in series leave only about 12% end-to-end reliability. That is why conversational code writing can feel acceptable while fully automatic, end-to-end delivery of a requirement remains hard. The gap is not only single-step skill; it is the lack of a structured correction mechanism.
4. From Mechanism to Design Principles
Once these mechanisms are clear, the design stance follows: the model generates; the system handles memory, verification, and planning. In practice that means:
- Extend the boundary with tools. File read/write, command execution, and retrieval break the limit that “context is all the memory.”
- Offset locality with a feedback loop. Inject test, build, and runtime results into the loop so the model can correct course from real signals.
- Shorten the chain with structured workflow. Break large tasks into small steps that can be verified on their own, limiting error accumulation.
- Replace model memory with external state. Task lists, decision logs, and environment snapshots are persisted by the agent framework, not left to the hope that the model “remembers” everything.
5. Summary
Starting from autoregressive generation, this article derives three structural traits of large language models—generation is thinking, context is memory, and output is probabilistic—and shows how those traits are amplified in long, tightly constrained software work. Knowing what the model can and cannot do is the basis for the next question: how a reliable coding agent should design its loop, its tools, and its feedback.
