Long-Horizon Coding Agents: Architecture and Engineering Practices
This article explains practical architecture patterns for long-horizon coding agents. It covers verifiable task contracts, external working memory, recoverable execution loops, inference throughput optimization, evaluation design, cost and permission controls, deliverables, and a staged rollout strategy.
The difficulty of long-horizon programming tasks is not whether a model can write a function, but that it must make dozens or even hundreds of interdependent decisions under incomplete information, changing repository state, and limited context. A failed single-turn completion usually just means rewriting a piece of code. A failed long-horizon task may instead show up as a broken environment, bypassed tests, or intermediate conclusions that cannot be reproduced. The engineering focus should therefore shift from “producing a longer single answer” to making the execution process decomposable, inspectable, and recoverable.
Turn Goals into Verifiable Contracts
Before starting a long-horizon task, natural-language requirements should be narrowed down to a small set of checkable items: which modules may be changed, which behaviors must be preserved, which commands determine completion, and which paths must not be touched. A contract does not need to be so formal that it becomes impossible to maintain, but at minimum it should allow either a person or an automated checker to answer: “Is the task done now?”
In practice, a three-layer description works well.
The first layer describes the user-visible outcome, for example: “Build a bootable desktop session and complete basic post-login operations.”
The second layer describes engineering constraints, for example: dependency sources, permission boundaries, and where logs and configuration should be stored.
The third layer defines stopping conditions: several consecutive successful verifications, no new failures in key regression tests, or reaching a budget or time limit and needing to submit the current result with unfinished items clearly stated. Without explicit stopping conditions, an agent can easily spin inside a local optimum.
A schematic task state does not need to be tied to any specific framework. Plain structured text is enough:
goal: complete a reproducible desktop environment prototypeinvariants:- do not modify locked system directories- all changes can be replayed from version recordschecks:- build- boot_smoke- regression_subsetbudget:- max_steps- max_wall_time
The value of such a contract is that it replaces “it feels finished” with “the status of the checks is known.” Whether the model, executor, or workflow changes later, the evaluation criteria remain stable.
Context Is Not a Copy of the Repository
One of the most common degradations in long-horizon execution is continuously stuffing the entire code tree, all logs, and the full conversation history into the context window. A larger window does not necessarily mean more useful information. A safer approach is to maintain an external working memory: current hypotheses, approaches already tried, reasons for failures, items still to verify, and references to artifacts, rather than repeatedly pasting the artifacts themselves into the conversation.
Working memory should distinguish facts from inferences. Command output, diff snippets, and test names are facts. A statement such as “the failure is caused by a dependency version” is an inference and must be accompanied by reviewable evidence. When an inference is later invalidated by experiments, it should be explicitly retired, so that an old conclusion is not treated as a settled premise many rounds later.
Retrieval also needs boundaries. Vector retrieval is suitable for recalling API descriptions, similar defects, and design notes, but it should not replace exact symbol lookup. For function definitions, call relationships, and configuration keys, structured indexes or direct file reads are still more appropriate. The principle of hybrid retrieval is simple: semantic recall narrows the search space, while precise reading is used to draw conclusions. If an agent relies only on similarity, it may easily modify a module with a similar name but a different responsibility.
Make the Execution Loop Recoverable
A practical execution loop usually looks like this: observe, plan, make a small change, verify, record, and decide whether to continue or roll back. Each step should be small enough that it can be undone after a failure, rather than generating an entire subsystem in one shot. Taking small steps does not mean lowering ambition; it means cutting risk into sizes that are diagnosable.
The environment must be treated as first-class state. Dependency installation, generated files, background processes, and temporary configuration can all change the premises of the next experiment. It is advisable to snapshot the workspace, dependency lockfiles, and key environment variables. Record a baseline before an experiment and compare differences afterward. A success that cannot be replayed has limited engineering value, because the next maintainer cannot distinguish luck from method.
Human intervention points should be designed in advance, rather than improvised only when the agent is already stuck in a loop. Good moments to pause include privilege escalation, irreversible data changes, repeated failures of the same type beyond a threshold, and verification commands that clearly diverge from the task goal. When pausing, the agent should not merely output “help needed”; it should provide the current hypotheses, paths already eliminated, and suggested next steps. This way, human time is spent on judgment rather than on reconstructing the scene.
Inference-Side Optimization for Stable Throughput
The inference load of a coding agent is uneven: some steps are short classification tasks, while others require reading large diffs and generating patches. If the underlying system uses sparse structures such as mixture-of-experts, the system-level concerns are not slogans about speed, but whether expert load is skewed over time, whether tail requests drag down the whole batch, and whether prefix caching can be reused across multiple tool-call turns.
Balancing within layers or across layers is essentially about preventing a few experts from staying overheated while the others remain idle. When balancing is done well, the gains usually appear in tail latency and overall throughput, rather than in a magical speedup for a single sample. When optimizing, one should look at batch composition, sequence-length distribution, expert capacity, overlap between communication and computation, and whether failed retries cause duplicate computation. Reporting only average throughput while hiding tail latency can mislead capacity planning.
Caching should distinguish what is reusable from what is not. System prompts, tool descriptions, and stable repository summaries are suitable prefixes. Content containing timestamps, random identifiers, or just-modified files should not blindly hit old caches. An incorrect cache hit is more dangerous than no cache at all, because it causes the agent to act based on an outdated world.
Deployment forms also differ. Training reproduction and online inference have different goals. Reproduction emphasizes traceability of numerical results and data pipelines. Online serving emphasizes concurrency, isolation, rate limiting, and graceful degradation. The two may share model weight management, but they should not share the same unisolated experimental environment. Container platforms are suitable for fixed runtimes, object storage for checkpoints, log bundles, and build artifacts, message queues for splitting long tasks into retryable stages, and monitoring alerts should cover queue backlog, validation failure rates, and per-task cost, not only whether an API is alive. If the system runs on a cloud environment such as Baidu AI Cloud, the same general design principles should apply; console operations should not be mistaken for the task itself.
Evaluation Should Mirror Real Agents, Not Single-Point Answers
Repository-level repair, terminal operation, and generating or modifying a repository from a description test different abilities. The first focuses on localization and patch quality. The second focuses on command planning and the use of environment feedback. The third focuses on understanding specifications and making structural tradeoffs. Comparing systems only on a single leaderboard tends to select systems that are good at answering the prompt but weak at finishing the work.
A more useful evaluation mix should include at least: reproducible tasks on fixed repositories, terminal tasks with noisy logs, and specification tasks requiring multi-file coordination. Scoring should separately record whether the patch is correct, whether unrelated changes were introduced, whether verification was actually executed, and whether the agent rolled back after failure. The agent harness itself is also a variable: the same model, paired with different tool permissions, retry strategies, and context-pruning methods, can produce results that differ by an order of magnitude. Therefore, when reporting results, one should also state the executor, permissions, time limits, and retry caps.
Self-evaluation cannot replace external testing. Having the model score its own patches is useful as a cheap filter: it can flag cases where tests were not run, where the diff is too large, or where changes are unrelated to the contract. The real gate should still be independent builds, tests, and static checks. If self-evaluation and external results diverge over a long period, the right move is to calibrate prompts and checklists, not to increase the weight of self-evaluation.
Vector indexes and retrieval strategies can also undergo small-scale self-evolution, but the scope should be narrow: adjust chunking, filtering fields, and reranking features, and compare recall and incorrect-edit rates using a fixed query set. Without a control set, “automatic improvement” often just means overfitting to the most recent few tasks.
Cost, Permissions, and Deliverables
The cost of long-horizon tasks comes from repeated reading, failed retries, and overly broad tool permissions. Possible controls include: using tiered models or tiered strategies for simple classification versus complex editing, setting retry caps for the same failure signature, and placing expensive validation after static checks. Prices and quotas can change with service-provider policies and should not be hardcoded into the workflow. The workflow should retain a budget switch and a degradation path, for example switching to outputting a diagnostic report instead of continuing automatic modifications.
Permissions should follow least privilege. Reading, searching, running tests, and writing to the workspace should be granted in tiers. The ability to modify production configuration or secrets should be off by default. All write operations should be audited: who changed which paths, when, and based on which input.
Deliverables are best fixed to four items: contract completion status, key diff explanation, verification commands and results, and known limitations. This allows a reviewer to decide whether to merge without replaying the full trajectory. The trajectory itself should be saved as an attachment for later traceability.
A Practical Rollout Order
Teams can start with narrow scenarios: a single language, a single repository, clear test commands, and a prohibition on touching release processes. First make state recording, rollback, and verification gates reliable, then expand to multiple repositories or longer time limits. Extending time limits is a result, not a starting point; when the loop is not recoverable, running for eight hours only amplifies errors.
Once the loop is stable, invest in inference-side batching, caching, and expert balancing, and use the same task set to compare tail latency, success rate, and cost per task. Finally, integrate self-evaluation into the filtering stage. This order avoids optimizing an unreliable automation pipeline before the evaluation criteria and permission model are settled.
