AI Coding Agents Lose Context When a Session Ends: Context Engineering and Reliability Practices

1. The Real Pain Isn’t Wrong Code — It’s a Zeroed-Out Reasoning Chain

A developer’s worst moment with an AI coding agent is usually not a buggy patch. Wrong code can be reviewed, rolled back, or fixed by the agent itself — the cost is bounded and predictable.

The harder case is different. The agent has just finished reading the repository, has just mapped the call chain, has just zeroed in on the suspicious few lines, and is about to make the edit — and a banner appears: “You’ve reached your usage limit for this session.” The context link breaks, the reasoning chain is cleared, and all the investigation accumulated so far evaporates as if it never happened. You open a new session, re-explain the background, the errors, the reproduction steps, and the hypotheses you already ruled out, and then watch it re-walk the directory structure like it has amnesia.

Many developers have described this experience repeatedly, and it has spawned a very plain idea: if it’s unstable, switch tools. That deserves a calmer look, though — switching tools only moves the risk around; it doesn’t remove the risk. Any LLM-based coding agent runs into three structural problems: finite context, volatile state, and behavior that drifts across versions. The productive move is to treat context as an engineering asset that needs explicit design, rather than hoping a model upgrade will quietly take care of it for you.

2. Turning “It Got Dumber” Into Locatable Mechanisms

What users describe as “degraded intelligence” is a vague signal. In engineering terms it is usually several mechanisms stacked on top of each other, and checking them one by one turns folklore into a debuggable checklist.

1) The default reasoning tier was changed. Most agents let you choose a thinking budget between “fast answer” and “deep reasoning.” Lowering the default from the deep tier to the balanced tier looks like faster responses and lower cost in the short term, but over time it is a behavior change: the agent stops proactively reading files, stops cross-verifying, and tends to hand back a plausible-looking patch. These changes usually aren’t in the user-facing release notes, which makes them the easiest to miss.

2) Prefix cache invalidation trims earlier reasoning. To cut cost and time-to-first-token, providers commonly cache the session prefix. When the caching policy changes or the cache misses, the system drops or compresses earlier reasoning steps to stay within the context length limit. From the user’s side it looks like “it forgot why it ruled out that approach,” so it repeats mistakes and circles back to paths already abandoned.

3) Summarization and truncation are too aggressive. Long sessions inevitably require summarization. The question is what the summary keeps. If it retains conclusions but drops evidence — which files were read, which logs were inspected, which assertions were verified — the agent keeps advancing without grounding, which shows up as editing code “from memory.”

4) Length constraints in the system prompt suppress verification. Output length limits are a common configuration, and they couple directly to coding tasks: the agent prioritizes producing the change and squeezes out the token-expensive steps, such as running the build, running the tests, and reviewing the diff. Quality loss often isn’t that the model can’t verify; it’s that the prompt never leaves room to verify.

5) Tool results get truncated or structured information is lost. If reading files, searching, or executing commands returns output cut off at a critical point, the model has to reason from incomplete information. This kind of problem is nearly impossible to spot in the final code and has to be diagnosed from trace logs.

6) Silent model version switches and rate limiting. Canary releases can send the same prompt to different model versions at different times; rate limiting causes session interruptions outright. Both make an agent feel “sometimes good, sometimes bad,” but the root causes are entirely different.

3. Start With a Baseline: Make Agent Behavior Measurable

No measurement, no governance. The practical move is to turn “agent quality” into a regression dashboard, with metric definitions that are tool-agnostic — they describe behavior, not a particular product.

Metric	Definition	What It Tells You
Read-before-edit ratio	Share of edit operations where the target file was already read	Whether it looks before it acts
Verification execution rate	Share of tasks that actually ran build/test/static checks after the change	Whether it closes the loop
First-pass rate	Share of tasks that pass acceptance criteria with no manual rework	Overall output quality
Continuation cost	Tokens and turns needed to restore the previous reasoning depth after an interruption	Context continuity
Tool call distribution	Share of read, search, write, and execute calls, and how that share shifts	Early signal of behavior drift
Cost per task	Token consumption and end-to-end latency	Economics

The key point is collect your own baseline first, then watch the variation. Absolute values differ enormously across repository sizes, language stacks, and task types, so cross-team comparisons are of limited value; what matters as a reliable signal of version-induced regression is a sudden change within the same team over time. Tool call distribution usually shifts first — a sharp drop in the share of read calls with a rise in direct edits often surfaces a problem several days before first-pass rate drops.

The read-before-edit ratio can be computed straight from the call trace:

def read_before_edit_ratio(trace: list[dict]) -> float:
    """Whether the target file was already read before an edit operation.
    Returns a value in [0, 1]."""
    READ = {"read_file", "grep", "list_dir"}
    EDIT = {"edit_file", "apply_patch", "write_file"}
    seen: set[str] = set()
    total = hits = 0
    for step in trace:
        tool, args = step["tool"], step.get("args", {})
        if tool in READ:
            seen.add(args.get("path", ""))
            seen.update(args.get("matches", []))
        elif tool in EDIT:
            targets = args.get("paths") or [args.get("path", "")]
            for t in targets:
                total += 1
                hits += t in seen
    return hits / total if total else 1.0

Wire metrics like this into CI alongside a fixed evaluation set, and run them every time you change the model version or adjust configuration. That lets you catch regressions while they are still in canary:

# agent-regression.yaml (illustrative)
cases:
  - id: fix-null-deref
    repo: fixtures/order-service
    prompt: "Fix the null pointer exception in /checkout with an empty cart"
    assert:
      read_before_edit_ratio: ">= 0.90"
      tests_pass: true
      changed_files_max: 3
  - id: add-pagination
    repo: fixtures/report-api
    prompt: "Add pagination to the report query endpoint, keeping backward compatibility"
    assert:
      tests_pass: true
      backward_compatible: true
gate:
  fail_on: [read_before_edit_ratio, tests_pass]

4. Context Layering and Explicit Budgeting

Treating context as a scarce resource with a hard capacity limit, and allocating it deliberately, is the first step toward stability. A workable layering is by rate of change:

• Stable layer: role definition, coding conventions, security rules, prohibited operations. Almost never changes; keep it resident and give it priority for the cached prefix.
• Repo profile layer: directory conventions, build and test commands, dependency management, module boundaries. Changes slowly. Maintain it as a file in the repository instead of making the agent rediscover it every time.
• Task layer: the goal for this run, acceptance criteria, hypotheses already ruled out, explicit non-goals. Rewritten for every task.
• Evidence layer: file excerpts, logs, stack traces, and failure records recalled on demand. The largest and least stable; it must be retrieved by search, never dumped in wholesale.

from dataclasses import dataclass
@dataclass(frozen=True)
class ContextBudget:
    """Context budget allocation in tokens (illustrative; adjust to the model window)."""
    total: int = 200_000
    system: int = 6_000        # Stable layer
    repo_profile: int = 10_000 # Repo profile layer
    task: int = 20_000         # Task layer
    evidence: int = 100_000    # Evidence layer
    reserve: int = 64_000      # Reserved for model output and tool results
    def validate(self) -> None:
        used = (self.system + self.repo_profile + self.task
                + self.evidence + self.reserve)
        if used > self.total:
            raise ValueError(f"Context budget exceeded: {used} > {self.total}")
    def remaining_for(self, layer: str) -> int:
        return getattr(self, layer)

Three companion rules are worth writing into team conventions:

1. Keep the stable layer early and worded consistently. Prefix caching only pays off on byte-identical text, so frequently editing the system prompt keeps invalidating the cache and drives up both cost and latency.
2. Put only the minimum sufficient signal in the evidence layer. Rather than pasting a whole 2,000-line file, give the function signatures, the line numbers of relevant call sites, and the key log excerpts.
3. Make the reserve explicit. Without headroom for output and tool results, you will eventually hit silent “read halfway, then truncated” errors.

5. Externalize Session State: Don’t Depend on Conversation History

Conversation history is the least reliable storage medium available — it expires, it gets compressed, and it breaks under rate limiting. The investigation process should land as resumable artifacts in the repository, so that any new session can restore the scene in tens of seconds.

The recommended artifact shape is an append-only ledger that records hypotheses, evidence, and conclusions:

import json, hashlib
from pathlib import Path
class SessionLedger:
    """Write the investigation process to a resumable ledger
    instead of depending on conversation context."""
    def __init__(self, root: Path):
        self.path = root / ".agent" / "ledger.jsonl"
        self.path.parent.mkdir(parents=True, exist_ok=True)
    def append(self, event: dict) -> None:
        with self.path.open("a", encoding="utf-8") as f:
            f.write(json.dumps(event, ensure_ascii=False) + "\n")
    def checkpoint(self, task_id: str, digest: str, summary: str) -> None:
        self.append({"type": "checkpoint", "task": task_id,
                     "tree_digest": digest, "summary": summary})
    def latest(self, task_id: str) -> dict | None:
        last = None
        if not self.path.exists():
            return None
        for line in self.path.read_text(encoding="utf-8").splitlines():
            e = json.loads(line)
            if e.get("task") == task_id and e.get("type") == "checkpoint":
                last = e
        return last
def tree_digest(root: Path) -> str:
    """Workspace fingerprint, used to tell whether a checkpoint is still valid."""
    h = hashlib.sha256()
    for p in sorted(root.rglob("*")):
        if p.is_file() and not {".git", ".agent"} & set(p.parts):
            h.update(str(p.relative_to(root)).encode())
            h.update(p.read_bytes())
    return h.hexdigest()[:16]

Three points matter when using it in practice:

• One ID per task, ledger archived by task. The first thing a new session does is read latest(task_id). If the tree_digest doesn’t match the current workspace, someone changed code in the meantime, and the agent should ask for review instead of continuing straight ahead.
• Record the hypotheses you ruled out. This is the most token-efficient information in the whole process: it directly stops the agent from walking back down a dead end you already verified.
• Checkpoints must be rollback-able. Tag or snapshot before each change, in combination with version control, so that after an interruption you can return cleanly to the last usable state instead of stacking more edits on top of a pile of half-finished diffs.

6. Resilience: Don’t Tie Throughput to a Single Backend

A tool-level outage hurts mainly because the workflow is welded to one vendor and one model version. The fix is to abstract model invocation behind a replaceable layer:

from typing import Protocol
class AgentBackend(Protocol):
    def complete(self, messages: list[dict], *, budget: ContextBudget) -> str: ...
    def stream(self, messages: list[dict], *, budget: ContextBudget): ...
    def health(self) -> dict: ...

With that layer in place, the governance actions have somewhere to land:

• Version pinning and change review. Model version, reasoning tier, output length limit, and summarization policy are all “configuration as code.” Put them under version control and review, and forbid silent changes to defaults.
• Canary rollout and rollback. Run a new version on a small slice of tasks against the fixed evaluation set, and roll back as soon as metrics degrade. Turn the two assertions from the previous section into a CI gate.
• Observability. Record the prompt version, model version, token distribution, tool call sequence, and latency for every session, ship them into a central logging service, and configure monitoring and alerts for signals like “read-before-edit ratio below threshold,” “verification execution rate at zero,” and “cost per task doubled.” That way regressions are discoverable rather than discovered through user complaints.
• Multi-backend failover. Keep at least two usable backends for critical paths, route by task type (complex refactors to the deep reasoning tier, formatting and unit-test completion to the low-cost tier), and attribute cost properly. On public cloud platforms, this is supported by the standard combination of general-purpose model services, log services, monitoring and alerting, and object storage — no need to bind to a specific product.

7. Implementation Checklist

If you do only one thing, externalize context first: put the repo profile and the investigation ledger under version control so that any new session can restore the scene cheaply. Smallest investment, most direct payoff.

Then work down the list in priority order:

1. Establish your own baseline for the six behavioral metrics and wire it into a CI regression gate;
2. Bring model version, reasoning tier, output length, and summarization policy under configuration review;
3. Implement context layering and explicit budgeting to eliminate silent truncation errors;
4. Add observability and alerting so capability regressions surface during canary;
5. Abstract the backend interface and keep the ability to switch and route.

Back to the opening scenario: sessions will still be cut off, quotas will still run out, and model versions will keep fluctuating. The difference is that when context lives in files, when behavior is backed by metrics you can query, and when backends are replaceable, an interruption costs tens of seconds to restore the scene instead of tearing the whole investigation down. Tools keep getting replaced; engineering capability is the part that stays with the team.