AI Agent Self-Iteration: From Passive Memory to Active Skill Consolidation

1. A Recurring Confusion: It Remembers, but It Does Not Use It

Many teams first notice this problem not during an impressive demo, but during a routine, repetitive task.

A typical scenario is like this: you ask an agent to process a data file, such as cleaning fields, standardizing encoding, removing duplicates, and writing the result into a database. It does the job smoothly, and you even praise it. Some time later, you need to do almost the same task again, and you find that it starts from scratch: reading the file again, guessing the meaning of the fields again, and asking you again which deduplication strategy to use.

If you ask it about this, it may admit that “we discussed a similar problem before.” But that is the end of it. It does not proactively retrieve the previous processing path, nor does it say, “Last time I validated this three-step process; should we reuse it directly?” You have to translate that memory into natural-language instructions yourself and restate it step by step.

The root cause is not that the model cannot remember. The root cause is the form of the memory. Most agent frameworks handle “memory” as a form of passive retrieval: historical conversations, memory configuration files, and project documentation are assembled into the context window at the appropriate time. This kind of memory is narrative knowledge. It records what happened, but it does not express how something should be done.

From this comes a highly engineering-oriented implementation path, which can be summarized as “configuration as behavior”: you write preferences, constraints, and environment facts into memory files and project documentation, and the agent follows them. The more detailed you write, the more accurate it becomes. If you do not write them, it has to derive everything from scratch each time.

This approach is not wrong in itself. The problem lies in the cost structure. It transfers the long-term cost of capability accumulation to the user, and it does so in the form of prose. Prose-based memory has three inherent limitations:

1. Not executable: a paragraph describing cleaning rules needs to be re-interpreted by the model at runtime, and each interpretation may introduce deviation;
2. Not verifiable: no one can write unit tests for a paragraph;
3. Not reusable: when a person or project changes, all implicit assumptions in the paragraph are lost.

2. Two Paradigms: Passive Retrieval vs. Proactive Consolidation

Compared with this, the key difference in another approach is a single action: at the moment a task is successfully completed, proactively ask whether to solidify this operation path into a named, callable capability.

Passive retrieval memory                         Proactive consolidated capability
─────────────────────────────                 ─────────────────────────────
Historical conversations + memory prose        Named capability (Skill / Procedure)
Runtime ad-hoc interpretation                  Deterministic invocation at runtime
Results vary from run to run                   Results remain stable across runs
No version control                             Can be committed to a repository and reviewed
Lost when people or projects change            Distributed with the codebase

This is not a difference in wording. It is a shift in the type of knowledge: from declarative knowledge to procedural knowledge. Procedural knowledge can be executed, validated, and reused, which is why it deserves the term “self-iteration.” True self-iteration is not “it becomes smarter after more chatting.” True self-iteration means consolidating validated successful paths into executable assets and deterministically recalling them during the next similar task.

It is also worth noting that the proactive inquiry interaction point is itself a key design element. It turns an implicit process, one that depends on the user remembering to maintain it, into an explicit process initiated by the system at the completion moment. In psychological terms, this is reducing friction at the right moment: the user has just seen the result, the context is still complete, and the cost of judgment is lowest. This is the best window for consolidating experience.

3. What a Capability Unit Should Look Like

If the goal is to turn experience into an asset, then the output of a consolidation must contain five types of information. None can be missing.

1) Trigger conditions: when should this be used?
This is the basis of the recall mechanism. The description should be based on task intent and data characteristics, not on the user’s verbal naming.

2) Input/output contract: what does it consume and what does it produce?
Fields, types, encoding, and required items should be explicitly declared so that failures surface at the entry point rather than exploding in the middle of the workflow.

3) Step orchestration: how is it done?
Each step should have clear inputs, outputs, and decision conditions. Vague instructions such as “just process it roughly” should be avoided.

4) Validation and rollback: what counts as success, and how do you recover from failure?
For data-related tasks, row-count reconciliation, primary-key uniqueness, and aggregate-value sampling comparisons are especially important. Irreversible operations must have a rollback path or require a backup first.

5) Applicability and known boundaries: when should this not be used?
Declaring unsupported scenarios is often more effective than declaring supported scenarios, because it helps prevent misuse.

A schematic capability definition can be organized like this:

name: csv-to-table-import
description: >
  Cleans a UTF-8 CSV file and writes it into a target table.
  Suitable when headers are stable, a single file has fewer than 5 million rows, and no cross-file joins are required.
triggers:
  - User mentions importing CSV / loading data / cleaning before database ingestion
  - Input file contains headers and is UTF-8 encoded
inputs:
  source_path: { type: string, required: true }
  target_table: { type: string, required: true }
  dedup_keys: { type: array, default: ["id"] }
  batch_size: { type: integer, default: 2000 }
steps:
  - id: inspect
    run: Read the header and sample 100 rows; output inferred field types
    assert: Required-column non-null rate > 0.99
  - id: normalize
    run: Standardize date formats, trim spaces, convert empty values to NULL, and normalize encoding
  - id: dedup
    run: Deduplicate by dedup_keys, keep the last row, and record the number of conflicts
  - id: dry_run
    run: Import the full dataset into a temporary table and compare row counts and aggregate values
    gate: Requires manual confirmation before continuing      # Mandatory gate before an irreversible operation
  - id: commit
    run: Write to the target table in batches and record the batch cursor
on_failure:
  - Roll back the current batch, keep the original file, and output sample failed rows
observability:
  log: [input row count, discarded row count, deduplication conflict count, elapsed time, per-batch latency]

Combined with a directory structure, it becomes an engineering object that can be code-reviewed, version-controlled, and reused:

skills/
  csv-to-table-import/
    skill.yaml          # definitions and contracts
    steps/              # scripts for each step
    tests/fixtures/     # sample inputs and expected outputs
    CHANGELOG.md        # iteration log

4. What Should and Should Not Be Written in Memory Files

Consolidating capabilities does not mean abandoning memory. The two are complementary: memory handles “who I am and what my environment looks like,” while capabilities handle “how this task should be done.” It is recommended to split them into three layers and define expiration rules for each.

Layer	Example content	Lifecycle
Global memory	Language preferences, code style, commit-message conventions, prohibited operation types	Long-term, manually maintained
Project memory	Technology stack versions, directory conventions, data dictionaries, known pitfalls	Evolves with the repository, updated through review
Task memory	Current input paths, temporary parameters, intermediate conclusions	Expires when the task ends; should not enter persistent context

Several practical principles apply:

• Do not write volatile facts. Ports, temporary paths, and currently active flags become expiration mines if written into memory.
• Do not write one-off context. It is a major source of context pollution and squeezes out truly useful information.
• Do not let memory grow without limit. Context expansion increases both cost and latency, and it dilutes the attention weight of key constraints. Summarization compression and eviction strategies are required.
• Handle conflicts. When global memory and project memory give different requirements for the same item, priority must be explicit. In general, more local and more recent information should take precedence; however, safety and compliance constraints should have the highest priority and should not be overridden.

5. The Complete Closed Loop for Self-Iteration

A trustworthy iteration requires at least five stages. If any stage is missing, the closed loop degrades into “self-suggestion.”

collect -> evaluate -> consolidate -> recall -> feedback
  ^                          |
  +--------------------------+

Collect: record the task’s input characteristics, execution path, elapsed time, number of human interventions, and failure points. Without measurement, there is no way to judge whether a consolidation actually brings benefit.

Evaluate: decide whether this execution is worth consolidating. The criteria should be quantifiable, such as: the operation has recurred at least twice, the execution succeeded, and there was no major manual rework. An operation performed only once does not have enough sample data to be called a pattern.

Consolidate: generate the capability definition, place it under version control, and pass it through a review process. Automatically generated drafts should be explicitly marked as “pending review,” not immediately active.

Recall: perform intent matching at the start of a task. A common mistake is to make recall a keyword-matching problem. A more robust approach is to combine the declared trigger conditions, similarity to historical tasks, and invocation-frequency weighting, while allowing the user to choose among multiple candidates.

Feedback: record invocation success rate, failure types, and average human interventions, and use them as the basis for the next revision. The CHANGELOG in a capability file is not document decoration; it is the evidence chain of iteration.

6. Safety Boundaries: Which Experiences Should Not Be Automatically Consolidated

Proactive consolidation must have brakes. Otherwise, it can turn a one-off lucky success into a systemic risk. The following types of operations should not be automatically consolidated by default, or must only take effect after mandatory manual review:

• Irreversible operations: deleting data, overwriting production tables, force pushing, and bulk permission changes;
• Credential-related operations: any attempt to write keys, tokens, or connection strings into a capability definition should be blocked; credentials should only be obtained through external secret management and runtime injection;
• Permission changes and network exposure: opening ports, modifying access policies, and changing authentication methods;
• Compliance-sensitive data processing: processes involving personal information or regulated fields should go through compliance review first.

From an engineering perspective, three things are recommended:

1. Add a “risk level” field to capability definitions, and make high-risk capabilities manually triggered only;
2. Keep execution traces for all operations: inputs, parameters, output summaries, and executor information should be collected in a centralized logging service with monitoring and alerting configured;
3. Isolate execution environments from development environments: put the actual task-running process in a restricted sandbox on a container platform, and limit the scope of filesystem access and network egress.

7. Usability: Capabilities Are for People, Not Just for Terminals

There is another dimension that technical teams often overlook: no matter how strong the self-iteration capability is, if it can only be exposed through layered command-line arguments and steep configuration syntax, ordinary users will not be able to adopt it.

For agents whose primary interaction mode is the command line, at least the following design elements should be added:

1. Task templates. Converge high-frequency tasks into named entry points, provide as many default values as possible, and let users only supply the one item that actually changes.
2. Pre-execution parameter validation. Check required fields, types, and value ranges before actual execution, and report all problems at once instead of reporting one error, making the user fix it, and then reporting another.
3. Explainable plan confirmation. Before execution, restate in human language what will be done, which data will be affected, and whether the operation is reversible, so the user has a clear point at which to stop.
4. Rollback and replay support. On failure, provide recovery steps; on success, retain parameter snapshots sufficient for reproduction.
5. Progressive disclosure. Show a concise result by default, and expand details on demand, avoiding a one-time dump of all logs.
6. Team-oriented asset management. Put the capability repository under version control, require review, maintain change records, and keep regression test cases. It is a team asset, not a local file on someone’s terminal.

8. Implementation Checklist

• Distinguish memory from capability: narrative information goes into memory, while procedural workflows go into capability definitions;
• Write all five elements for every capability: trigger conditions, input/output contract, steps, validation and rollback, and boundaries;
• Establish a three-layer memory structure and conflict-priority rules, and define periodic cleanup and compression strategies;
• Set a consolidation entry point at the successful completion of a task, and require sample-size and success-rate thresholds;
• Mark capability definitions with a risk level, and require manual confirmation for high-risk items;
• Isolate execution environments, collect logs centrally, and connect key metrics to monitoring and alerting;
• Provide template entry points with default values and pre-execution parameter validation for common tasks;
• Use invocation success rate, average human interventions, and average elapsed time as iteration metrics.

9. Conclusion

“It remembers but does not use it” is not fundamentally a model-capability problem. It is a problem of the form of knowledge consolidation. If experience remains in conversation history and prose paragraphs, it will always be a temporary conclusion that must be re-derived every time. Only when it is solidified into a capability unit with contracts, validation, and version control can it become an organizational asset that grows through compounding.

Therefore, the real “change of track” is often not a change of tool, but a change of working method: from “re-teaching every time” to “validate once, reuse over the long term, and revise continuously.” Whether this path can succeed depends on three things being in place at the same time: whether the consolidation mechanism is proactively initiated by the system at the appropriate moment, whether the safety boundaries are clear, and whether the interaction is accessible to ordinary users. If any one of these is missing, self-iteration remains a concept rather than engineering.