Running LLM Agents Autonomously for Hours: State Management, Coordination, and Reliability
Large language models can answer questions in seconds, but keeping an agent productive for hours on an execution-style task is a different engineering problem. This article draws a clear boundary between one-shot Q&A calls and long-running autonomous tasks, then walks through three pillars that determine how long an agent can run: externalizing state beyond the model's context window, building automated validation loops with checkpoints and rollback records, and coordinating multi-agent work through dependency graphs and interface contracts. It includes checkpoint schema design with optimistic locking, structured stage summaries, milestone validation checklists with concrete thresholds, contract fields for subtask handoffs, and failure propagation semantics. Aimed at teams with automated testing, CI pipelines, and sandboxed permission controls.
1. First, Draw the Capability Boundary
LLM applications generally fall into two categories. Q&A-style invocation means the user submits a single request and the model returns text or code, with the whole process finishing in seconds to a few minutes. Execution-style tasks mean the model continuously reads and writes files, runs tests, and calls external tools under a single goal, deciding the next step based on results. The engineering requirements differ:
| Dimension | Q&A-style invocation | Execution-style task |
|---|---|---|
| Typical duration | Seconds to minutes | Tens of minutes to hours, depending on the environment |
| State persistence | A single context window suffices | Must be stored outside the context |
| Permission control | Usually returns text only | Reads/writes files, calls tools; requires least privilege |
| Result verification | Human review | Automated tests, builds, and static checks |
How long an execution-style task can run depends on three conditions: whether state can be stored outside the context, whether stage results can be automatically verified, and whether the system can roll back to the last trusted checkpoint after a failure. The more of these conditions you satisfy, the longer the task can run. Durations and scales discussed in this article assume these conditions; they do not apply to environments lacking externalized state and automated verification.
The intended audience is teams with automated testing and continuous integration, and sandbox environments with constrained permissions. Not applicable: tasks that connect directly to production databases, hold high-privilege credentials, and lack human approval gates.
2. Long-Running Coding: Externalized State and Verification Loops
The main risk in long tasks is error accumulation. An early faulty assumption may only surface hours later. The countermeasure has three layers: keep state outside the model, produce structured summaries for each stage’s output, and run automated validation at every milestone.
2.1 Checkpoint Writes
Checkpoints must satisfy two requirements: repeated writes must not corrupt the result, and concurrent writers must not overwrite each other. The recommended approach is a version number for optimistic locking and step IDs for idempotent records.
The checkpoint fields are as follows:
| Field | Type | Description | Suggested value (validate in your environment) |
|---|---|---|---|
| task_id | string | Unique task identifier | Globally unique |
| version | int | Optimistic-lock version number | Increment by 1 on each successful write |
| done_steps | list | Committed step IDs | Used for idempotent skipping |
| data | object | Stage output | Cap per-checkpoint size; 256 KB is a reasonable starting point |
| checksum | string | State integrity hash | SHA-256 |
| write_retry_max | int | Max retries on write failure | Starting point: 3 |
Here is illustrative code for committing a step; store and sha256_of need to be implemented against your actual storage:
def commit_step(store, task_id, step_id, delta, max_retry=3):for _ in range(max_retry):state, version = store.load(task_id)if step_id in state["done_steps"]:return True # Idempotent: skip if already committedstate["done_steps"].append(step_id)state["data"].update(delta)state["checksum"] = sha256_of(state["data"])if store.put_if_version(task_id, state, version): # Write only if version unchangedreturn Trueraise RuntimeError("Write retries exhausted, escalate to human operator")
When resuming a task, verify the checksum first; if verification fails, roll back to the previous checkpoint. If state exceeds the single-entry size cap, split it across multiple keys by subtask.
2.2 Stage Summaries
After each sub-stage completes, have the model produce a summary in a fixed structure, then inject that summary into the next stage’s context. Even after the original conversation is truncated, the key decision chain survives. Cap the summary length — 1,000 words is a reasonable starting point — and when over the cap, prioritize keeping decisions and open issues.
The JSON summary template:
{"stage_id": "S03","completed": ["API documentation generated", "Unit tests passing"],"decisions": [{"decision": "Use SQLite for storage", "basis": "Single-machine deployment, low write concurrency"}],"open_issues": ["Coverage on new code below threshold"],"next_plan": ["Add boundary-condition tests"]}
2.3 Validation Checklist
Every milestone must pass the following checks before advancing to the next stage.
| Check | Field | Threshold (validate in your environment) | Action on failure |
|---|---|---|---|
| Unit tests | pass_rate | 100% | Roll back to previous checkpoint |
| Coverage on new code | coverage_new | Determined from team’s historical data | Do not advance to next stage |
| Static analysis | new_lint_errors | 0 | Roll back |
| Type checking | type_check | pass | Roll back |
| Build and clean-environment startup | build_ok, smoke_ok | true | Roll back |
| Same failure recurring | same_failure_count | Reaches 2 | Stop auto-repair, escalate to human |
Agree on the coverage metric in advance — line coverage or branch coverage, for example — and count only the lines changed in this round. Do not copy thresholds from other teams; set them based on your own historical data.
Every rollback should write a rollback record with fields including rollback_id, from_checkpoint, to_checkpoint, reason, and timestamp. These records make it easier to trace which step introduced an error.
3. Multi-Agent Coordination: Dependency Graphs and Interface Contracts
3.1 Dependency Graph Modeling
Decompose the task into nodes; data or artifact dependencies between nodes form edges. Follow three rules when modeling:
- The dependency graph must be acyclic. A circular dependency means the decomposition is flawed and needs to be redone.
- Every node declares its input artifacts and output artifacts.
- Parallel nodes must not write to the same path.
Failures propagate downstream along dependency edges. When an upstream node fails, downstream nodes enter a blocked state and stop executing. Unrelated branches can keep running, unless a budget circuit breaker triggers.
3.2 Interface Contract Fields
The caller and executor of every subtask must agree on the fields below.
| Field | Required | Description | Example or starting point (validate in your environment) |
|---|---|---|---|
| task_id | Yes | Subtask identifier | T-021 |
| input_schema | Yes | JSON Schema reference for input | schemas/input_v2.json |
| output_schema | Yes | JSON Schema reference for output | schemas/output_v2.json |
| timeout_s | Yes | Per-call timeout in seconds | Starting point: 300 |
| idempotency_key | Yes | Idempotency key | task_id + step_id + input hash |
| retry_max | Yes | Retry limit | Starting point: 3 |
| error_code | Yes | Failure return code | E_TIMEOUT, E_SCHEMA, E_TOOL, E_BUDGET, E_APPROVAL |
| owner_agent | No | Agent responsible for execution | Useful for accountability tracking |
3.3 Failure Propagation Semantics
Different failure types should propagate differently.
| Failure type | Propagation | Handling action |
|---|---|---|
| Retryable errors (timeouts, rate limits) | Retry within the node; do not propagate downstream until retries are exhausted | Retry with backoff parameters |
| Contract errors (input or output fails schema validation) | Fail immediately, block downstream | Return upstream to fix the contract or implementation |
| Permission errors | Stop the affected branch | Generate an approval request |
| Budget circuit breaker | Pause all branches | Save checkpoint |
