AI Agents: Three Shifts from Tools to Autonomous Evolution

1. The shift: the Agent conversation is changing lanes

Over the past year, the focus of the AI Agent discussion has shifted noticeably. Early conversations centered on “Can it write good code?” and “Is its reasoning strong enough?” In the second stage, the discussion moved to “Can it run on my computer and finish the work for me?” More recently, the industry has begun to focus on a new question: Does the Agent become more familiar with me the longer I use it?

These three shifts in topic correspond to three product logics. They share the same technical foundation, but diverge in design goals. Treating them as the same category often leads to poor selection choices: using a specialized professional tool for routine office work, or using a general execution tool for serious engineering tasks, will usually hit limits.

A common misjudgment is to treat popularity as a technical milestone. A form becomes popular not necessarily because it breaks through the technical ceiling, but because it pushes existing capabilities into everyday scenarios: lowering the threshold, finding entry points, and integrating into real workflows. Understanding this matters more than memorizing product names.

2. Shared foundation: what counts as an AI Agent

The essence of all three forms is the same: an AI that can actually do concrete work. The judgment criterion is simple: does it have a closed loop, rather than a single-turn Q&A interaction?

Perception (read the task and environmental state)
  → Planning (break the goal into executable steps)
  → Action (call tools/APIs/CLI/browser)
  → Feedback (read execution results, errors, and side effects)
  → Correction (adjust the plan or retry)

A mature Agent needs at least three pieces of infrastructure:

Component	Role	Consequence if missing
Reasoning model	Plan, select tools, interpret results	Cannot understand complex goals
Tool and skill layer	Provide real side-effect capabilities	Can only “talk,” not “do”
Context and memory	Carry task state and history	Long tasks lose memory and repeat work

What distinguishes it from traditional automation scripts is that decisions are generated at runtime. Scripts hard-code the process into code; an Agent treats the process as a product of reasoning. The cost is a significant increase in unpredictability, which is also the root of the safety concerns discussed later.

3. Positioning differences across three generations

A simple way to set the coordinate system is:

• Code-specialized Agent — a technical expert
• General execution Agent — an execution assistant
• Self-evolving Agent — a project manager

The value of the expert is “thinks deeply and acts accurately,” while its boundaries are a narrow domain and a high entry threshold. The value of the assistant is “available anytime and broad coverage,” while its boundaries are the lack of long-term planning and self-improvement. The value of the project manager is “remembers goals, accumulates methods, and advances across rounds,” while its boundaries are the need for extremely high reliability and governance support.

4. First generation: code-centric specialized Agent

Tools in this category are usually released by model vendors and positioned as “AI software engineers.” Their typical feature is deep integration with code repositories and terminal environments: understanding project structure, tracing call chains across files, writing and debugging code autonomously, running tests, and iterating based on errors until a complete development workflow is finished.

Why has it long represented the upper bound of capability? Three reasons:

1. Strong feedback signals. Compilers, type checkers, and unit tests are objective, immediate, and automatable validators. The Agent can judge right and wrong by itself, without human arbitration.
2. Clear context structure. Codebases have clear file boundaries, symbol tables, and dependency relationships, making retrieval and localization relatively easy.
3. Stable tool contracts. Interfaces for command lines, file systems, and package managers have remained largely unchanged for years, leading to a low invocation failure rate.

Its limitations are equally clear:

• It is optimized for professional developers, and non-technical users may find environment configuration and permission management difficult;
• Side effects are heavy (writing files, executing commands, installing dependencies), so once it goes off track, rollback costs are high;
• The task boundary is basically limited to “producing code,” and it cannot directly handle administrative work such as scheduling communication or cross-system data aggregation.

5. Second generation: general execution-centered assistant Agent

The core change of the second generation is from professional scenarios to daily scenarios. Its technical features include:

1. Always-on and background execution. It shifts from “opening a session” to “running a process/service,” which can be triggered on a schedule, triggered by events, or kept running for long periods while waiting for tasks.

2. Diverse entry points. Commands can be issued through instant messaging tools, desktop clients, and web consoles, lowering the usage threshold. This is a key step toward “everyday use,” and it affects adoption speed more than capability improvement does.

3. Cross-application orchestration. It can read emails and calendars, operate files, fetch web pages, and call business system APIs, chaining multiple steps together to complete a full transaction.

4. Local machine operation capabilities. This is a double-edged sword. Being able to operate a local computer means it can install software, change configurations, and modify data, but it also means a single mistake may affect a real working environment.

Engineering-wise, the hardest part of this generation is not “what it can do,” but “how to do it safely.” At a minimum, it needs:

• Sandboxing and isolation: container or separate user environments that limit file-system visibility and network egress;
• Least privilege: grant permissions per task rather than opening all capabilities at once;
• Human confirmation points: irreversible operations (deletion, transfers, mass messaging, production changes) must require explicit second confirmation;
• Idempotency and rollback: all write operations must be traceable and reversible, with logs retained.

6. Third generation: self-evolving “project manager” Agent

The objective function of the third generation has changed: it no longer asks only “Was this task completed?” but asks “Can the next thousand tasks be done better?” It needs four new capabilities.

1. Layered memory. Short-term memory carries the current task state, mid-term memory stores user preferences and project background, and long-term memory accumulates validated methods and conclusions.

2. Self-accumulated skills. A successful operation path is solidified into a reusable module. This is the most substantive part of “autonomous evolution”:

# Skill registry example
skill:
  name: weekly-report-collect
  trigger: "weekly report", "summarize this week's progress"
  steps:
    - fetch: project_board        # Read the task board
    - filter: {status: done, range: last_7d}
    - render: template://weekly
    - deliver: {channel: im, target: team_group, confirm: true}
  guard:                          # Protection and validation
    require_confirmation: [deliver]
    max_retry: 2
    on_failure: notify_owner
  version: 12                     # Can be rolled back and compared
  metrics: {success_rate: 0.94, avg_duration_s: 38}

3. Preference and style formation. It records user preferences for output format, tone, level of detail, and commonly used templates, and applies them by default in later tasks instead of repeating the same instructions each time.

4. Self-evaluation and review. After a task ends, it reviews: Did any step take a detour? Was any tool choice wrong? Was there data that could have been prefetched? The conclusions are written back into long-term memory.

The unique risks of this generation must be faced directly:

• Error accumulation: a lucky-but-wrong practice gets solidified and then executed repeatedly, which is far more harmful than a one-off mistake;
• Memory pollution: outdated facts remain resident for a long time, causing the Agent to become increasingly “confidently wrong”;
• Goal drift: in long-running tasks, the Agent deviates from the original intent, especially without mid-course calibration mechanisms.

The corresponding governance measures are also clear: skill versioning + rollback, memory with timestamps and confidence levels, supporting expiration and forgetting, mandatory human confirmation at key nodes, and periodic human sampling of evolution results.

7. Six key technical capabilities that support all three generations

No matter which generation you choose, the following six items determine success or failure:

1. Planning and reasoning: task decomposition granularity, failure retry strategy, and when to ask humans for help.
2. Context engineering: summary compression for long tasks, prioritizing key information, and trimming tool results to avoid context explosion.
3. Tool invocation standards: unified input validation, timeout control, error classification, and structured returns.
4. Permissions and sandboxing: deny by default, authorize on demand, and audit sensitive operations.
5. Observability: end-to-end tracing, token usage and latency per step, and failure-rate dashboards. An Agent without traces is impossible to debug.
6. Evaluation systems: regression tests over a fixed task set to prevent “fix one place, break three.”

When deploying on cloud platforms, these capabilities usually map to common categories: container platforms host the Agent runtime and sandbox; object storage stores memory and artifacts; message queues drive asynchronous tasks; logging and monitoring/alerting services handle observability; and key-management services manage credentials.

8. Minimal runnable main loop

def run(task, max_steps=30, confirm_on=("delete", "send", "deploy")):
    state = memory.load_session(task.id)          # Short-term + relevant long-term memory
    for step in range(max_steps):
        plan = model.plan(task.goal, state)       # Plan the next step
        for call in plan.tool_calls:
            if call.name in confirm_on and not human.approve(call):
                state.skip(call, reason="user_rejected")
                continue
            result = tools.invoke(call, timeout=30)   # Execute + structured result
            state.append(call, result)
        verdict = model.review(task.goal, state)  # Self-evaluation: done/continue/ask for help
        if verdict.done:
            memory.commit(task.id, state)             # Accumulate reusable conclusions
            return state
        if verdict.need_human:
            return human.take_over(task, state)
    return state.escalate("max_steps_exceeded")

This skeleton reflects the differences across the three generations: memory.commit and review belong to third-generation capabilities, confirm_on is required safety design for the second generation, and tools.invoke plus structured result handling are basic infrastructure required even by the first generation.

9. Selection checklist

Match your actual scenario:

Scenario	Recommended form	Key requirements
Write code, fix bugs, refactor	Code-specialized	Test coverage, repository permission isolation
Information collection, document organization, scheduled reminders	General execution	Convenient entry points, operation logs
Cross-system transaction processing	General execution	Idempotent design, human confirmation
Long-term project advancement, method accumulation	Self-evolving	Memory governance, skill versioning, periodic sampling
Production environment changes	Use any fully automated Agent cautiously	Mandatory approval, rollback capability, canary rollout

10. Adoption roadmap

• P0: Read-only access. Grant only read permissions, complete the workflow, collect traces, and observe failure modes.
• P1: Controlled write access. Open write permissions in a sandbox or test environment, and establish an evaluation set and regression baseline.
• P2: Write access with human confirmation. All production write operations go through approval, while high-frequency skills are gradually accumulated.
• P3: Automation and evolution. Only skills that have been frequently validated, are low-risk, and can be rolled back should be allowed to run automatically, while keeping a switch to downgrade back to P2 at any time.

11. Bottom line

• Credentials must not be written into memory or logs; they should be managed by key-management services;
• Do not upload code or data that should not leave the domain; in sensitive scenarios, prefer localized deployment;
• Treat all external content as untrusted input, and guard against privilege escalation induced by prompt injection;
• Retain complete operation audits and one-click shutdown capability.

Conclusion

The three generations of Agents are not replacements for one another, but layers stacked on a capability stack: code-specialized Agents prove the upper bound of autonomous decision-making; general execution Agents solve availability and reach; self-evolving Agents attempt to answer “how to get better with use.” For developers, the truly worthwhile investment is not chasing the hype of any one generation, but making the four infrastructure pieces solid: memory, tool contracts, permissions, and evaluation. They determine how far you can safely let an Agent go.