AI Coding Agents: From Code Completion to Autonomous Code Refactoring

1. Starting From “Zero-Code Delivery”

One engineer set himself a challenge: build a reinforcement learning training environment from scratch without writing a single line of code by hand, relying only on AI coding tools. It took dozens of hours of back-and-forth prompting, adjustment, and continuous conversation with the model. The environment was eventually built, and he came away completely hooked on the tooling.

The significance of that experience has little to do with how complex the task was. It marks a change in how coding tools divide labor:

• The old assistant-style tools: the developer owned every line of code, and the tool completed snippets or answered questions. The human was always the author.
• Today’s agent form: the developer defines goals and constraints, while the agent plans steps, reads and writes files, executes commands, and verifies its own work. The human becomes the goal-setter and the reviewer.

Two or three years ago, typical usage stopped at “have AI explain a block of code, write a unit test, complete a function.” Today these tools can carry out large-scale code migrations and even participate in building new products from zero. That jump in capability brings efficiency, but also new problems: when an agent can modify code and run commands on its own, how do you ensure it does the right thing, does it safely, and can be rolled back when something goes wrong? The answer is not simply a stronger model, but an engineering system designed around the agent.

2. Three Levels of Capability: Completion, Conversation, and Autonomous Execution

Classified by degree of autonomy and method of verification, AI coding tools fall into roughly three levels:

Capability level	Source of context	Typical interaction	Output form	Main risks
Completion	Current file and code near the cursor	Line-level and block-level suggestions	A single code fragment	Syntax errors, inconsistent style
Conversational	Selected snippet, conversation history	Q&A, explanation, test generation, local refactoring	A code patch to paste in	Misunderstood requirements, hallucinated APIs
Agentic	Repository structure, dependency relationships, execution results	Long tasks, multi-step planning, iterative self-correction	Coordinated cross-file changes	Wrong edits, overreach, poor traceability

Each level introduces a new class of risk:

• From completion to conversation: the tool starts to “understand intent.” Once that understanding is off, the developer may adopt a solution that looks plausible but is actually wrong, such as calling an API that does not exist.
• From conversation to autonomous execution: the tool starts to “take action.” It can create files, modify configuration, install dependencies, and run commands, expanding the blast radius from a single function to the whole repository and even the runtime environment.

The governance focus for AI coding tools therefore has to move from “reviewing generated content” to “constraining execution behavior.” The three elements discussed next, namely permissions, verification, and context, all answer the same underlying question: how to keep the human in control of the whole picture.

3. Controlled Autonomy: Permission Models and Sandbox Design

3.1 From Step-by-Step Approval to Tiered Authorization

Early agents required human approval for nearly every operation: confirm before reading a file, confirm before editing one, and certainly confirm before running a command. That approach is safe, but it creates two problems:

1. Developer attention is interrupted constantly, and the efficiency gains are offset by approval overhead.
2. Facing dozens of authorization requests in a row, people tend to click “approve” out of habit, so the approval step becomes a formality and the safety boundary actually weakens.

A more sensible approach is tiered authorization based on least privilege: read-only by default; routine code edits allowed inside the workspace; builds and tests run in an isolated environment; high-risk operations always require explicit confirmation.

Operation type	Risk profile	Recommended policy
Reading files, searching code	Low, but may touch secrets or configuration files	Allow by default; mask or block sensitive paths
Editing workspace source	Medium, trackable and reversible under version control	Restrict to specific directories, keep a change list
Installing dependencies, running scripts	Medium to high, may pull in malicious code or pollute the environment	Run in an isolated container with pinned sources
Accessing external networks	High, may leak code or pull in untrusted content	Deny by default, open selectively via a domain allowlist
Pushing branches, releasing or deploying	High, impact extends beyond the local machine	Require manual confirmation and keep a full audit record

The following is an illustrative permission policy. The fields are examples meant to show the design approach:

# Permission policy for an AI coding assistant (illustrative)
permissions:
  read:
    allow: ["src/**", "tests/**", "docs/**"]
    deny:  ["**/.env*", "**/secrets/**", "**/*.pem"]   # Block access to sensitive files
  write:
    allow: ["src/**", "tests/**"]
    deny:  ["**/migrations/**", "**/*.tfstate"]          # Never auto-modify high-risk files
  exec:
    allow: ["make build", "make test", "make lint"]      # Only allow controlled commands
    deny:  ["git push", "kubectl *", "rm -rf *"]
  network:
    default: "deny"
    allowlist: ["registry.internal.example"]             # Example domain, internal dependency source only

3.2 Sandbox Isolation: Drawing a Physical Boundary Around Execution

No matter how complete the policy is, an agent that runs commands needs an environment where it is allowed to make mistakes. Common isolation measures include:

• Dedicated containers or virtual machines: each task runs in a disposable environment that is destroyed afterward, preventing state pollution and lateral spread.
• Read-only filesystem mounts: source code is mounted read-only, agent changes are written to a separate workspace, and merged only after verification.
• Least-privilege credentials: temporary, tightly scoped credentials are injected only when a task needs them, and are automatically masked in logs and error messages.
• Outbound traffic control: external network access is denied by default, with only the dependency sources and endpoints a task requires allowed through.

3.3 Traceability and Rollback: Changes Must Be Reviewable and Reversible

Every agent modification should carry the three traceability elements:

1. Who changed it: identify the requesting user, plus the model and policy versions used.
2. Why it was changed: record the task description, planned steps, and rationale.
3. What was changed: produce a structured change list so code review can focus on what matters.

Here is a minimal change list structure:

{
  "task_id": "MIG-2025-0142",
  "operator": "user:zhangsan",
  "agent_version": "coding-agent@1.8",
  "target_repo": "order-service",
  "files_changed": [
    {"path": "src/repository/legacy_dao.py", "action": "modified", "additions": 42, "deletions": 88},
    {"path": "tests/test_legacy_dao.py", "action": "added", "additions": 120, "deletions": 0}
  ],
  "verification": {
    "compile": "passed",
    "unit_tests": "passed (312/312)",
    "static_check": "passed"
  },
  "rollback_ref": "a1b2c3d",
  "created_at": "2025-11-10T09:30:00Z"
}

With a change list and a rollback reference, even if the agent introduces a problem, it can be reverted in minutes instead of costing hours of investigation.

4. The Self-Verification Loop: Getting It Right After Getting It Done

An agent’s autonomous cycle boils down to four phases: plan, execute, verify, and correct. The most underrated one is verification:

• A loop without verification just generates wrong code faster.
• With sufficient verification, errors stay local and the agent can fix them in the next iteration.

A typical autonomous loop can be described in pseudocode:

def autonomous_task(goal: str, verify: VerifySuite, max_iterations: int = 10):
    plan = planner.make_plan(goal)          # 1. Break down into verifiable subtasks
    for step in plan.steps:
        for i in range(max_iterations):
            patch = coder.apply(step)       # 2. Apply the change
            report = verify.run(patch)      # 3. Machine verification: compile, test, static check
            if report.passed:
                break
            if not report.actionable:       # 4. Escalate to a human if it can't self-correct
                return escalate(step, report)
            patch = coder.fix(step, report) # 5. Fix based on failure info, next iteration
        else:
            return escalate(step, reason="max_iterations_exceeded")
    return summarize(plan, verify.latest_report())

The verification suite should be layered, forming a progression from fast feedback to medium checks to deep verification:

Verification layer	What it checks	Feedback speed	Role in the loop
Syntax and type checks	Compile errors, type mismatches	Fastest	Run every iteration
Unit tests	Whether function and module behavior matches expectations	Fast	Run every iteration
Static analysis	Potential defects, code smells, convention deviations	Medium	Run before commit
Integration and contract tests	Consistency of inter-service interfaces and data contracts	Slower	Run when the task wraps up
Performance and security checks	Latency on critical paths, dependency vulnerabilities	Slow	Run before merge or periodically

The loop also needs “stop conditions” to keep the agent from cycling through pointless revisions:

• Iteration cap: if the same subtask fails several rounds in a row, pause and hand it to a human.
• Regression detection: if a new round passes fewer checks than the previous one, the direction is wrong and the code should revert to the last working version.
• Cost budget: set limits on API calls and resource consumption per task or per unit of time to prevent runaway loops.

5. Context Engineering: Helping the Agent Understand the Whole Project

If permissions govern what the agent is allowed to do, context engineering determines whether it does the right thing. Failures in autonomous coding usually do not come from the model being unable to write code, but from it failing to see key information, for example:

• It does not know which dependency versions the project actually uses, so it calls an API that does not exist.
• It does not know the team’s coding conventions, so the generated code clashes with the rest of the codebase.
• It misses every caller of a given interface, and the change causes cascading failures.

Engineering practice usually builds context from four sources:

1. Structured repository knowledge: automatically generate descriptions of directories, module responsibilities, and dependency relationships, updating them incrementally as code changes, so the agent gets a “map” first and then looks up specifics.
2. Task-level retrieval: retrieve code, historical commits, and defect records relevant to the current task, feeding only the small set of most relevant snippets into context rather than stuffing in the entire repository.
3. Project convention files: write the tech stack, directory conventions, testing requirements, and prohibited actions as a machine-readable document. A simplified example:

<!-- Project conventions (illustrative) -->
## Technical conventions
- Server side uses Python 3.11; dependencies are declared in pyproject.toml; never modify the lock file directly
- All database access must go through the Repository layer; never assemble SQL in the interface layer
- Every new public function must come with a unit test
## Prohibited
- Do not introduce third-party libraries not declared in the dependency manifest
- Do not modify already-committed historical migration files under migrations
- Do not commit configuration files containing keys or tokens

1. Execution feedback: return test output, error stack traces, and log fragments in structured form so the agent can fix code based on observed failures instead of guessing at causes.

Context is not simply “more is better.” Irrelevant information dilutes attention and raises the chance of misreading. A practical rule: give a global overview during planning, local snippets during implementation, and failure information only during verification.

6. A High-Value Scenario for Coding Agents: Large-Scale Code Migration

Among engineering tasks, large-scale code migration aligns best with agent capabilities:

• The goal is clearly defined, for example replacing every call to a deprecated interface with the new one.
• The change pattern is highly repetitive and suits batch processing.
• Results can be machine-verified: whether the build and tests pass is obvious.
• Doing it manually is expensive, tedious, and error-prone.

6.1 Why Pure Scripts and Pure Agents Both Fall Short

• Pure scripts can perform deterministic text replacement but struggle with semantic differences, such as changes in error handling, default values, or async behavior between the old and new APIs.
• Pure agents understand semantics, but across tens of thousands of files the cost is high, the pace is slow, and individual judgments may not be consistent.

The more reliable option is a hybrid model of deterministic rewriting plus agent-handled semantics: let scripts do the mechanical parts, let the agent handle the parts that need judgment, and back both with a unified verification gate.

6.2 The Five-Step Migration Method

1. Scope the blast radius: use static search to count files to change, the number and distribution of call sites, and produce a migration list.
2. Pin down the rules: define the mapping between old and new APIs, edge cases, and prohibitions, written as migration guidance for the agent.
3. Pilot on a small set: migrate a few representative files, review the results and verification output manually, and refine the rules.
4. Execute in batches: proceed by module or directory, running the build, tests, and static checks after each batch, and letting only batches that pass the gate move on.
5. Spot-check and wrap up: manually sample a proportion of the agent’s changes, focusing on changes that touch edge cases and concurrency behavior. When everything is done, update the documentation and turn the migration into reusable rules.

6.3 Risk Control Checklist

Risk	How it shows up	Mitigation
Wrong replacement	Methods with the same name get rewritten indiscriminately	Use semantic analysis to locate call sites instead of pure text replacement
Missed changes	Dynamic dispatch and reflection call sites go undetected	Add a static search inventory and cross-check with runtime coverage
Behavior change	Default values or exception handling differ between old and new APIs	List the differences in the migration guidance and add targeted tests
Dependency conflicts	Old and new dependency versions are incompatible	Verify dependency resolution and build in an isolated environment first
Oversized changes	A single commit contains hundreds or thousands of edits, hard to review	Split commits by module, with a change list and verification report per batch

7. Evaluation and Observability: Judging Reliability With Data

Agent performance cannot be judged by subjective impression alone. It needs a measurement system combining offline evaluation with online monitoring.

7.1 Offline Evaluation Metrics

Metric	Definition	What it tells you
First-pass rate	Share of tasks that pass all verification on the first submission	Degree of out-of-the-box usability
Average iterations	Average number of corrections before a task passes verification	Self-correction efficiency
Human intervention rate	Share of tasks that need human help to finish	Controllability and team workload
Regression defect rate	Share of agent changes that later cause defects	True quality of the generated code
Cost per task	Model calls and compute consumed per task	Feasibility at scale

The evaluation set should come from the team’s real historical tasks, such as merged defect fixes and refactoring requests, and be refreshed regularly to prevent tuning against the benchmark.

7.2 Online Observability

In production, use general capabilities such as logging services, monitoring and alerting, and distributed tracing to observe agent behavior continuously:

• Record each task’s goal, steps, duration, verification results, and whether the output was ultimately adopted.
• Set threshold alerts on failure rate, average duration, and resource consumption to catch runaway loops or permission misuse early.
• Break down metrics by model version, policy version, repository, and language to pinpoint where problems concentrate.

Evaluation is not meant to prove “the agent is strong.” It is meant to identify which task types it handles reliably and which scenarios still require a human safety net.

8. Common Pitfalls and How to Avoid Them

1. Treating the agent as “faster autocomplete”: what it really changes is the division of labor. If you keep reviewing line by line, the gains are eaten up by review cost and you lose the benefit of autonomous planning.
2. Granting full permissions once and using them forever: permissions should narrow dynamically by task type, with re-confirmation before every high-risk operation.
3. Merging changes without verification: agent-generated code must go through compile, test, and review like any human commit. Being AI-written is not a reason to lower the bar.
4. Piling on context: dumping large amounts of irrelevant code and documentation degrades output quality. The right approach is layered, on-demand retrieval.
5. No rollback plan: any workflow that lets the agent make bulk changes needs a predefined rollback path and impact assessment.
6. Leaving compliance to the end: code exfiltration, secret leakage, and open-source license risk should be part of workflow design from the start, not patched afterward.

9. Conclusion: Humans Stay in Control, Agents Do the Execution

From early code completion, to building a whole training environment without writing a line of code, to completing large-scale code migrations independently, the capability boundary of AI coding tools keeps expanding fast. But what determines their value is not only how much they can do autonomously, but how much control the human retains.

A pragmatic adoption path comes down to four statements:

• Humans define the goal: clarify task boundaries, acceptance criteria, and prohibitions.
• The agent does the execution: let it handle search, modification, and self-verification, the repetitive work.
• Machines run the verification: compile, test, and static checks form an automatic gate.
• Humans remain the safety net: keep manual approval for high-risk operations and always retain rollback capability.

For teams looking to adopt agentic coding, start with low-risk read-only assistance, then gradually open up controlled code editing and execution permissions, while building up project conventions, verification suites, and evaluation metrics along the way. General capabilities offered by cloud services such as Baidu AI Cloud, including container platforms, object storage, logging services, and monitoring and alerting, can provide elastic isolated runtime environments, unified change archiving, and continuous observability support for this kind of system. With clear permission boundaries and verification standards in place, agents can become capable executors for the team, while developers focus on what matters more: defining problems, designing systems, and spending the time they save on the creative work they actually care about.