Long-Running Tasks in LLM Code Agents: Capability Evolution and Engineering Practice
This article traces how LLM-based code agents are moving from single-point coding help toward multi-step engineering work that can run for tens of minutes or hours. It covers three core capabilities, namely long-task context management, plan mode, and sub-agent orchestration. It also looks at how tool front-end design affects the user experience, and how developers can build a maintainable workflow on top of a stable model API.
1. From Writing Code to Handling Long Tasks: The Capability Shift in Code Agents
Early LLM coding assistants mainly solved isolated problems. They could complete a function, explain a piece of logic, or generate a utility class. As context windows grew and reasoning improved, the focus of the industry shifted to a harder problem: keeping an agent on target, keeping its context accurate, and keeping its actions under control during an engineering task that runs for tens of minutes or even hours.
This is the common backdrop for the rapid, incremental iteration that major model vendors have pursued in code agents. Take the latest iteration of a code model from one cloud vendor. Its improvements center on three areas: context management for long tasks, plan mode, and sub-agent orchestration. Together, these three form the main pillars of a long-task agent, and each deserves a closer look.
2. A Technical Breakdown of the Three Core Capabilities
2.1 Context Management for Long Tasks
The biggest threat to a long task is context decay. Halfway through a task, earlier file contents, past decisions, and resolved pitfalls can be diluted by a large volume of later tool call results, or pushed out of the window entirely. The common approaches in the industry include:
- Layered compression: Summarize the history of completed subtasks and keep only the decision conclusions, key file paths, and open items. Discard the raw intermediate output.
- External memory: Persist task state to local files, such as a task checklist or a progress log. The model reads these files back when needed instead of relying on what remains in its window.
- Anchoring key information: Explicitly mark high-value information, such as the user’s core requirements and architectural constraints, so it survives multiple rounds of compression.
In practice, an agent with good context management can still remember the original requirement boundaries after modifying a dozen or more files. An agent with poor context management starts to forget things as early as the fifth step. It asks the same questions again and repeats the same mistakes.
2.2 Plan Mode
Plan mode separates “thinking” from “execution” into two explicit phases. Before taking any action, the model produces a structured plan that covers task decomposition, dependency order, and risk points. The user confirms the plan before execution begins. This approach has three main benefits:
- Misunderstandings surface early: The user can correct a wrong interpretation at the lowest possible cost, before the model writes hundreds of lines of code in the wrong direction.
- Lower risk from destructive operations: High-risk actions such as deleting files, running large refactors, or executing shell commands can be caught and reviewed during the planning phase.
- Better control over long tasks: The plan acts as a working agreement. When execution drifts from the plan, the drift is easier to notice.
2.3 Sub-Agent Orchestration
For large tasks, a single agent working in a linear fashion is slow and places heavy pressure on its context. The sub-agent mechanism lets a main agent hand off independent subtasks, such as “find all references to a given symbol across the codebase” or “add tests for a specific module,” to separate context instances. Each sub-agent returns only a summary of its conclusions when it finishes. A simplified flow looks like this:
# Pseudocode: main agent orchestrating sub-agentsresult = main_agent.run(plan=[Task("Map the dependencies of module A", delegate="explorer-agent"),Task("Refactor function B", delegate="coder-agent"),])# Sub-agents work in isolated contexts and return only summaries.# The main context absorbs the conclusions only, so intermediate# output does not overflow it.
This divide-and-conquer pattern addresses two problems at once: context bloat and the lack of parallel execution. It is a clear direction in the ongoing evolution of code agent architectures.
3. The Overlooked Half: Usability Problems in the Tool Front End
In practice, users often attribute a “hard to use” experience to the model. A large share of these problems actually comes from the tool front end, and the model itself plays a smaller role. Typical issues include:
Heavy system prompts. Some official CLI tools inject system prompts of up to about 20,000 tokens at the start of a session. These prompts contain lengthy descriptions of built-in tools and detailed behavioral rules. For models with smaller windows, this severely reduces the working space available for the actual task. Even for models with large windows, the fixed overhead is considerable.
Repeated adjustments to interaction details. Examples include whether reasoning (thinking) output is displayed, the stability of streaming rendering, and how clearly error and retry messages are presented. Tools with active open-source communities usually fix these issues quickly. Official tools without that community engagement often iterate more slowly.
Replaceable protocols. Mainstream agents typically support standardized integration protocols or API formats. As a result, the same model can work with different clients. When one tool provides a poor experience, switching to a third-party client maintained by the community can often give an immediately better result. The tool and the model are separate layers that can be decoupled.
4. Building a Sustainable Engineering Setup
Heavy users do not have to be tied to the release schedule of a single front-end tool. A practical alternative is to build a thin wrapper of your own. A workable approach has the following elements:
- Use the model API as the only stable dependency. Connect only to general-purpose chat and tool-calling interfaces. Do not bind the setup to any specific CLI.
- Control the system prompt yourself. Keep system instructions to a few thousand tokens, and leave the remaining budget for the real code context.
- Implement your own context management and sub-agent scheduling. Apply the layered compression and task dispatch patterns described above, and adapt them to your project structure.
- Keep the model replaceable. Current domestic code models show clear strengths in cost-effectiveness, inference speed, and behavioral reliability, meaning they avoid editing unrelated code and do not overreach. Even so, the architecture should still allow switching models so the setup is not locked to a single provider.
5. Conclusion
Code agents are moving from question-answering tools toward engineering collaborators. Long-task context management, plan mode, and sub-agent orchestration form the capability foundation. The maturity of the front-end tool chain also strongly shapes the actual experience. Developers should understand the boundary between these two layers and evaluate model capability and tool design separately. With an engineering approach, they can build a long-lived AI programming workflow of their own. That approach is the most dependable way to keep pace with a fast-changing technology ecosystem.
