Programming Agent Architecture: Keep the Loop Simple and the Supporting Systems Strong

When teams build coding agents, they often start by designing complex state machines, directed acyclic graphs, or custom orchestration engines. The goal is to precisely control every path through “read code, edit files, run commands, inspect results, and iterate.” After deployment, however, a different class of problems often appears: long turns lose context, a single tool failure interrupts the workflow, permission boundaries remain unclear, and the original state of a session cannot be fully reconstructed.

What users experience as a lack of smoothness is usually not caused by the quality of a single generated response. It is caused by instability across the entire execution chain.

A more reliable starting point is to reduce the control flow to a loop that is easy to explain, then invest engineering effort in the systems around it. On each turn, the model performs only one task: generate the next response based on the current message history. If the response contains a tool call, the runtime executes it, appends the structured result to the history, and starts the next turn. If there is no tool call, the loop pauses and control returns to the user.

State should be represented by an immutable message sequence rather than hidden variables or an external workflow diagram. As long as the history is complete, the current progress, tools already called, failure reasons, and pending confirmations can all be reconstructed.

This design may look almost trivial. That is also why it can support so many behaviors. Programming tasks do not require every branch to be enumerated in the control flow. The model can decide whether to search further, modify a file, run validation, or ask the user a question based on tool results. The runtime only needs to execute, record, and enforce constraints faithfully.

When “what to do next” is handled by the model and the message history, while “whether an action is allowed, how its result is persisted, and how failures are exposed” is handled by the supporting systems, the responsibilities remain separate.

Why Orchestration Should Not Become Too Complex

State machines and directed acyclic graphs work well for batch jobs with fixed steps and exhaustively known branches. Coding assistance is different. It is open-ended: the same request may require inspecting a directory, reading tests, discovering a missing dependency, changing an implementation, and running checks again. Along the way, the workflow may need to backtrack because of permission or environment differences.

If every possible path is encoded in an orchestration engine, the graph quickly expands, while real sessions will still find ways to leave it. Maintainers then have to understand both model behavior and graph transition conditions. The cost of troubleshooting can exceed the benefits of explicit orchestration.

The boundaries of a simple loop should also remain clear. It does not implement business policy, assume that every operation will succeed, or hide failures from the model. Tool results—whether they indicate success, a timeout, a permission denial, or truncated output—should enter the history as ordinary messages. The next turn can then continue based on facts rather than relying on side-channel state.

If the control flow requires specialized background knowledge to understand, complexity has probably been placed at the wrong layer. The loop should fit into a short piece of pseudocode. The supporting systems are where the engineering depth belongs.

The basic relationship looks like this:

loop:
  response = model.generate(history)
  if response has no tool call:
    return response to user
  result = run_tool(response.tool_call)  # includes permission checks and error wrapping
  history.append(response)
  history.append(result)

The loop itself can remain short. Whether the experience is smooth depends on the context assembled before and after run_tool, permission decisions, result normalization, and history management.

The Supporting Layers Are the Main Engineering Work

1. Context Management

In a coding workflow, the input is more than the user’s original request. It may include the repository structure, file excerpts that have already been read, previous tool outputs, and unfinished plans.

Appending to the history indefinitely eventually consumes the useful context window. The model may start ignoring early constraints or repeat operations that have already failed. The supporting layer therefore needs an explainable trimming strategy.

It should preserve system constraints, the user’s goal, recent decisions, and unresolved errors. Long logs should be summarized or stored as external references rather than silently dropping important failure information. Compression should remain traceable so that the reason behind an earlier change can still be reconstructed.

2. Permissions and Confirmation

An agent that can read, write, and execute commands creates most of its risk through side effects. The supporting layer should classify operations as read-only, reversible modifications, or irreversible and high-impact actions.

Read-only inspection can often run automatically. Changes to the workspace and commands that may affect the environment should have a clearly defined scope, a preview, and, when necessary, explicit human confirmation.

Denials should also be recorded as structured results that explain which authorization is missing. The model should not have to guess whether it can simply try again. Permission rules belong in the runtime, not scattered across prompts and enforced through the model’s judgment alone.

3. Tool Contracts

Each tool needs a stable name, input constraints, output fields, and failure semantics. The model should see a schema that can be validated, not an ambiguous natural-language description.

Inputs should be validated before execution, and outputs should be normalized before they are returned. A successful result should provide facts that support further reasoning. A failure should include an error category, a concise explanation, and whether retrying is appropriate.

Avoid injecting very large raw dumps directly into the context. When necessary, return a summary and place the complete output somewhere it can be retrieved on demand. With clear contracts, the loop does not need to know how individual tools work internally.

4. Error Recovery

Timeouts, non-zero exit codes, missing paths, and truncated results are normal branches in a coding workflow. The supporting layer should distinguish among:

• Transient failures that can be retried
• Call errors that require different parameters
• Permission or environment problems that require the user’s input

Retries should have limits and should not repeatedly cycle through the same error without changing the approach.

Keeping errors in the message history is often more effective than writing extensive compensation branches in the orchestration layer. The model can see the failure evidence and respond by narrowing the search scope or changing its modification strategy.

5. Memory and Session Recovery

Immutable messages make “reconstructing state from history” the default capability. Long-running sessions still need layered memory, however:

• The complete trace of the current turn
• User preferences and project conventions that remain valid across turns
• Temporary outputs that can be discarded

When restoring a session, the system should be able to answer three questions:

1. What was the goal?
2. What has already been done?
3. Which conclusions were later disproved by new results?

If memory cannot be audited, outdated assumptions can easily be treated as facts.

Making the User Experience Smoother

For users, smoothness comes from predictability rather than from hiding the process. The interface or logs should show whether the agent is waiting for the model, executing a particular category of tool, or paused for permission.

Long-running tasks should be interruptible. After an interruption, the history must remain consistent rather than leaving a state in which an operation was partially executed but never recorded.

For developers, the same history should be enough to replay a problem: What context was available at the time? What did the tool receive? What did it return? What did the model do next based on that result?

Replay is often more effective for locating the root cause of an unreliable experience than adding another control-flow node.

When implementing the system, validate it layer by layer instead of focusing first on the number of available tools.

The control-flow layer should verify that:

• The loop stops when there is no tool call
• A tool call is executed and written back to history
• Exceptions do not send execution into an unobservable side path

The context layer should verify that trimming preserves the goal and the most recent failures.

The permission layer should verify that unauthorized calls are rejected and that the reason is visible.

The tool layer should verify that invalid inputs are not executed and that failure results can be understood by the next turn.

The memory layer should verify that a resumed session does not depend on hidden in-memory state.

Once these checks pass, additional domain tools can be introduced without usually changing the loop itself.

Another extreme should also be avoided: putting every policy into the prompt while leaving the runtime with almost no validation. Prompts are useful for describing goals and style, but they are not a suitable permission system or error protocol. Models can overlook constraints, while tools may still produce real side effects.

The value of the supporting layers is to place unreliable generation inside boundaries that can be validated, recorded, and recovered.

Conclusion

The core of a coding agent can be a message-history-driven loop: generate a response, execute and append any tool call result, or wait for the user when no tool call is present.

The wide range of agent behaviors comes from the model making continuous decisions based on history, not from a complex control-flow engine. What truly determines whether the experience is smooth is context management, permission and confirmation handling, tool contracts, visible errors, and reconstructable memory.

If a framework’s control flow is difficult to understand, it is often overdesigned at the wrong layer. Keep the loop simple, and invest engineering effort in the supporting systems around it. The result will be easier to extend and easier for users to trust.