Improving AI Coding Agents: Session Management and a Five-Layer Architecture

1. Using an Agent Is Not the Same as Using It Well: Three Levels of Usage

Many developers still interact with AI coding agents in a purely question-and-answer mode. They start a new session whenever a question appears, ask what they need, close the terminal, and let the context disappear. After a year, what they have accumulated is scattered conversation experience, not a reusable engineering methodology.

The difference becomes clearer when usage is divided into three levels:

• Command memory: The user knows which commands exist and how parameters should be filled in. When facing a new scenario, they still need to search for answers.
• Mechanism understanding: The user understands what state each command changes, including the context window, session index, permission boundaries, and process lifecycle.
• System design: The user can design the agent as part of a larger system and predict failure modes in long-running tasks, concurrent tasks, and multi-agent collaboration.

The challenge raised by the technical leader in the original article is essentially a third-level capability examining a first-level capability. The key to crossing this gap is not memorizing more commands. It is building a five-layer architecture mental model. Under this model, any observation that “the agent is not working as expected” should be traceable to a specific layer rather than vaguely attributed to “the model is not smart enough.”

2. Sessions and Context: Turning Working State into a Persistent Asset

One of the most misunderstood aspects of command-line agents is what “memory” actually means.

A single session contains three types of state:

1. Conversation state: The sequence of messages exchanged between the user and the model.
2. Workspace state: Actual changes to code files, uncommitted diffs, and results of dependency installation.
3. Tool and process state: Background processes that have been started, ports that have been opened, and cached build artifacts.

The reason closing the terminal often feels like “everything is gone” is that the first type of state usually exists only in memory. A mature session management mechanism persists the first type of state into a disk index. This allows the session to be resumed across terminals, days, or machines, including restoring the conversation context along with a summary.

2.1 Three Phases of a Session: Creation, Continuation, and Archiving

• Creation: Start the session in the correct directory and with the correct permission mode. This step determines the boundaries of all subsequent operations and matters more than any prompt-engineering trick.
• Continuation: Prefer “resume” over “restart.” Even if a new session restates the background, it may lose many implicit constraints, such as which approaches have already been rejected, which files cannot be modified, and what the test command is.
• Archiving: After a task is completed, organize the conclusions, including decisions, changed points, and remaining items, and write them into long-term project documentation. Session records grow over time; conclusions do not.

2.2 Session Branching: Parallel Exploration Instead of Serial Trial and Error

When a task has two viable technical routes, it is often better to create multiple branch sessions from the current node instead of switching back and forth in the same session and polluting the context.

# Illustrative syntax only, used to explain the branching model, not the syntax of any specific product
agent-cli resume <session-id> --fork --label "Plan A - Direct Connection"
agent-cli resume <session-id> --fork --label "Plan B - Cache Layer"

The value of branching is that the mainline context remains clean and correct, while exploration noise is isolated in side branches. In long-running refactoring tasks, this can determine success or failure more strongly than model selection.

2.3 Context Budget Awareness

The context window is a limited and shared resource. It is consumed by system prompts, tool definitions, retrieval results, conversation history, and file contents. Three practical principles are useful:

• Load only the files relevant to the current subtask, and avoid “feeding everything at once.”
• For long files, first provide the path and function signatures, then expand details only when needed.
• Important constraints, such as coding standards, protected modules, and acceptance criteria, should be fixed in project-level long-term memory or configuration files rather than repeated verbally. Verbal constraints can be lost during context compaction.

3. Context Compaction: A General Design for Layered Pipelines

Any agent will eventually hit the context limit during long tasks, so a compaction strategy is necessary. The common industry approach is a layered pipeline rather than a single truncation point.

Layer	Strategy	Applicable Content
L1 Raw preservation	Keep the most recent N conversation turns without compaction	Operations currently being executed
L2 Structured summary	Merge content by goal, completed work, conclusions, and pending tasks	Middle-stage conversation history
L3 Retrieval recall	Use summaries plus retrieval to fetch original content on demand	Early discussions and reference materials
L4 External memory	Write content into project documentation or memory files	Standards, decisions, and lessons learned

The implementation skeleton can be understood as follows:

class LayeredContext:
    def __init__(self, keep_recent=8, summary_budget=4000):
        self.recent = []          # L1
        self.summaries = []       # L2
        self.archive = []         # L3, searchable
        self.keep_recent = keep_recent
    def add(self, item):
        self.recent.append(item)
    def compact(self):
        """When the budget is exceeded, demote items outside the recent window one by one."""
        while len(self.recent) > self.keep_recent:
            old = self.recent.pop(0)
            self.summaries.append(summarize(old))   # L2
            self.archive.append(old)                # L3
        # L4: Persist stable project-level constraints as long-term memory files

There are two useful indicators for judging whether compaction is healthy:

• Irrecoverable information is lost, such as a previously rejected approach being proposed again after compaction.
• The same batch of files needs to be read repeatedly, which indicates that the retrieval layer is failing and the system has degraded into full reloading.

4. Permissions and Sandboxes: Making Autonomy an Adjustable Dial

Autonomy is not a binary switch. It is a continuous spectrum. A common classification is as follows:

Level	Behavior	Recommended Scenario
Confirm each step	Every action requires manual approval	Production repositories or unfamiliar codebases
Automatic edits	File writes are executed automatically, while commands still require confirmation	Daily iterative development
Plan first	Produce a plan only, without applying changes	Solution review and impact assessment
Sandboxed autonomy	Free execution within a restricted environment	Batch processing and bulk refactoring
Silent and no prompts	No interruption, relying on post-execution review	Scenarios protected by complete CI
Full access	Execution without boundaries	One-off tasks in isolated environments

Three non-negotiable safety principles apply:

• Least privilege: Do not grant network access or write access outside the workspace by default. Elevate privileges temporarily only when required by the task.
• Rollback capability: Before any autonomous execution, the workspace should be in a clean state under version control. Changes should be concentrated in disposable worktrees or branches.
• Auditability: All tool calls should be logged, including commands, parameters, exit codes, and duration.

# Illustrative example: a minimal permission decision hook
POLICY = {
    "read":    "allow",
    "write":   "allow_in_workspace",
    "shell":   "ask",
    "network": "deny",
}
def decide(action: str, ctx) -> str:
    rule = POLICY.get(action, "deny")
    if rule == "allow_in_workspace":
        return "allow" if is_within_workspace(ctx.target) else "deny"
    return rule

The secure default should be deny, not allow. The most dangerous failure mode of an agent is not writing incorrect code. It is executing an irreversible operation without the user being aware.

5. Planning and Control: Let the Agent Think Before Acting

A single large prompt is not enough to support complex tasks. The control layer usually consists of three components:

1. Task list mechanism: Break the goal into explicit pending items and update their status as each item is completed. This serves two purposes: it prevents omissions, and it provides a stable “progress anchor” during context compaction so that the current progress is not lost when history is compressed.
2. Sub-agent delegation: Assign independently verifiable subtasks, such as “list all callers of this module,” to separate contexts and collect only the conclusions. This is one of the most effective ways to fight context expansion.
3. On-demand skill or procedure loading: Turn reusable operating procedures, such as commit standards, release checklists, or framework-specific integration steps, into documents that are loaded on demand instead of being kept in a persistent prompt.

When combined with goal-condition-driven execution, where the agent autonomously loops until acceptance conditions are met or a budget limit is triggered, the control layer becomes complete. It turns a one-time question-and-answer interaction into an autonomous loop with termination conditions.

6. Concurrency and Scheduling: From Synchronous Waiting to Asynchronous Management

Long-running tasks, such as large-scale refactoring, dependency upgrades, and batch test supplementation, should not block interaction. Mature agents support running tasks in the background and registering scheduled jobs for periodic inspection.

# Illustrative scheduling configuration, not the syntax of any specific product
jobs:
  - name: nightly-regression
    schedule: "0 2 * * *"
    prompt: "Run nightly regression; locate failing cases to the minimal reproduction and generate a fix branch"
    mode: sandboxed
    concurrency: 1          # Avoid duplicate fixes for the same failure
    notify_on: [failure]

Two additional concerns arise in concurrent scenarios:

• Workspace conflicts: If the same repository is written to by multiple tasks, use independent worktrees or independent checkouts for isolation.
• Resource contention: When concurrency is too high, builds and dependency downloads can slow each other down. Set concurrency limits and queues.

7. Multi-Agent Collaboration: From a Single Agent to an Agent Platform

When the task scale exceeds the capacity of a single context, the architecture evolves from “one agent” to “an agent platform.” This typically includes four types of infrastructure:

• Role division: Planners, executors, reviewers, and testers each hold independent contexts and prompts.
• Message bus: Agents exchange conclusions and requests through structured messages instead of sharing the same conversation.
• Workspace isolation: Each executor has an independent working copy, and the final results are integrated through a standard merge process.
• Unified external tool access protocol: Encapsulate databases, monitoring systems, and internal platforms as standard tools that agents can call, avoiding repeated integration for each agent.

A typical failure mode in collaborative architecture is the reviewer being persuaded by the executor. When the two share context, review can degenerate into agreement. Keeping independent contexts is a necessary condition for making review truly valuable.

8. Enterprise Adoption: Integrating with Engineering Systems Rather Than Using It in Isolation

Turning an agent from a personal tool into team infrastructure requires integration with existing engineering systems. This section only involves generic cloud product categories. Model inference services handle inference calls, container platforms handle isolated execution environments, logging services and monitoring alerts handle observability, object storage preserves session archives and artifacts, and message queues smooth traffic peaks and drive asynchronous tasks.

For example, on Baidu AI Cloud, these capabilities are typically provided as standard cloud product categories, while orchestration and governance remain the responsibility of the team.

There are four governance priorities:

• Observability: Record the input summary, tool call chain, duration, and result of each task so that problems can be located to a specific layer.
• Quality evaluation: Build a fixed evaluation set, such as real historical issues or pull requests. Any change to the model, prompt, or permission policy should be regression-tested against the same set of cases.
• Cost and quota: Track consumption by project or person, set per-task budget limits, and prevent autonomous loops from getting out of control.
• Code compliance: Generated code must go through existing static checks, unit tests, and human review processes. The agent does not change the quality gate.

9. Systematic Learning Path: A Three-Stage Progression

Stage 1: Official interactive guided courses. Most mainstream command-line agents include guided learning commands that demonstrate core capabilities one by one, such as permission switching, goal-condition-driven execution, parallel branching, and session restoration. The value of these courses is that they allow immediate trial and error in a real environment, which builds muscle memory faster than reading documentation alone. It is recommended to use a small project as a practice vehicle and learn while modifying it.

Stage 2: Cheat-sheet-style reference. Organize commands and CLI flags by functional domain, such as context management, session tools, configuration and diagnostics, code review, and advanced capabilities. This is useful for building a complete capability map and filling in edge cases not covered by guided courses. The point is not to memorize parameters but to know whether a corresponding mechanism exists for a certain type of need.

Stage 3: Source-level breakdown. Using open-source implementations as teaching material and breaking down the internal architecture chapter by chapter is currently one of the most cost-effective ways to understand agent principles. Start from the basic perceive-decide-act loop, add one mechanism per chapter, and eventually assemble the complete system. The recommended deliverable is a one-page five-layer architecture diagram that marks the key mechanisms and corresponding commands at each layer.

10. Layered Positioning: A Troubleshooting Decision Tree

When an agent behaves abnormally, investigate from top to bottom according to layers.

Symptom	Possible Layer	Verification Method	Fix Action
Repeatedly reads the same batch of files	Memory management	Observe summaries before and after compaction	Narrow the loading scope and fix constraints into long-term memory
Forgets an earlier rejected approach	Memory management	Check the compaction policy	Write key decisions into project documentation
Claims completion but does not apply changes	Planning and control	Check task list status	Add acceptance conditions and self-check steps
Changes pollute unrelated files	Planning and control / Permissions	Check workspace diffs	Tighten write permissions and enable worktree isolation
Commands are rejected or time out	Tools and execution	Check tool call logs	Correct parameters and sandbox configuration
Multiple tasks overwrite each other	Concurrency and scheduling	Check the task timeline	Set concurrency limits and use independent working copies
Review conclusions always agree	Multi-agent collaboration	Check whether context is shared	Isolate the reviewer context

11. Getting-Started Checklist

• Before starting, confirm the directory, permission level, and whether the workspace is clean.
• For similar tasks, prefer resuming a session rather than starting a new one.
• Write important constraints into project-level long-term memory instead of relying on repeated conversation.
• Isolate exploratory solutions in branch sessions and keep the mainline clean.
• Run long tasks in the background and set budget and concurrency limits.
• Ensure rollback capability before autonomous execution and auditability after execution.
• Maintain a fixed evaluation set for the team, and regression-test any policy change.

Returning to the original question: after using AI coding agents for a year, the real dividing line is not how familiar you are with commands. It is whether you can treat the agent as a system with clear layers and clear failure modes. Only when you can say within ten seconds that “this is a memory-layer problem,” rather than merely saying “it messed up again,” have you truly crossed that threshold.