Practical Lessons from Six Months of Building with AI Coding Agents: Boundaries, Methods, and Production Adoption

The short version is this: AI coding agents are powerful, but no AI programming tool is a silver bullet. An agent is better understood as a fast, broadly knowledgeable engineering collaborator that still needs clearly defined task boundaries and acceptance criteria.

Used in the right situations, it can significantly reduce the time required for boilerplate code, test completion, localized refactoring, and documentation maintenance. Used in the wrong situations, it can produce seemingly reasonable explanations that create additional verification work and pull the team into an inefficient “trust first, verify later” cycle.

Two common types of community feedback illustrate this well. In one case, an agent identifies what appears to be a serious database concurrency issue. After spending 30 minutes investigating, the developer discovers that the issue was fabricated. In another case, a developer tries several tools and eventually realizes that flashy features are less important than having a reliable agent and a well-written project instruction file.

Together, these experiences describe the real nature of AI coding agents: extremely effective in the right context, but capable of wasting substantial time in the wrong one.

This article does not focus on a specific product. Instead, it addresses general engineering questions: what coding agents are good at, what they should not be trusted to do, how instruction files, task decomposition, and validation loops improve results, and where agents belong in the broader software development toolchain.

1. Capability Boundaries: Strong at Pattern-Based Production, Weak at Factual Guarantees

The core capability of an AI coding agent is pattern recognition and generation across code, documentation, logs, and task descriptions. Agents are particularly effective at:

• Applying existing conventions to new files
• Turning repetitive logic into reusable templates
• Generating tests from interfaces
• Suggesting investigation paths based on error logs
• Renaming symbols and restructuring localized code
• Drafting interface documentation
• Explaining configuration options
• Creating repetitive data transformation scripts
• Performing preliminary static issue scans before code review

These tasks share several characteristics: their context is relatively controllable, their acceptance criteria are reasonably clear, and their outputs can be verified quickly through compilation, tests, static analysis, or human review.

The limitations are equally important.

First, factual accuracy is not guaranteed. An agent may produce a detailed analysis of concurrency, performance, or security that is not actually grounded in the real call chain, lock granularity, transaction isolation level, or business constraints.

Second, implicit business rules are easy to miss. Many system behaviors are not documented in code. They may be embedded in historical compatibility requirements, operational policies, data definitions, or established exception-handling practices.

Third, causal reasoning across files, services, and time remains fragile. A change that appears correct locally may break global consistency.

Fourth, verification costs are often underestimated. A plausible bug report without an evidence chain can take significant developer time to disprove. That investigation is itself a cost.

The correct mindset is therefore to treat an agent’s output as a candidate solution that must be verified, not as an established fact. For topics such as database concurrency, transaction boundaries, authorization checks, billing logic, data migrations, production changes, and high-risk security policies, the agent should be required to provide verifiable support, including:

• Relevant code locations
• Assumptions about the call chain
• Reproduction steps
• Test cases
• A rollback plan

2. A Use-Case Matrix: High Value, Cautious Use, and Avoiding Direct Delegation

Tasks can be divided into three categories to help teams decide what to delegate.

High-value use cases

These include:

• Boilerplate code and scaffolding
• Unit test completion and boundary-case expansion
• Initial classification of logs and error messages
• Localized refactoring and naming standardization
• Draft interface documentation
• Configuration explanations
• Repetitive data conversion scripts
• Preliminary static issue scanning before code review

These tasks generally have clear inputs and outputs, and their results can be checked through automation.

Use with caution

These areas may benefit from an agent, but require smaller task boundaries, explicit evidence, and review by experienced developers:

• Cross-module architectural changes
• Database concurrency and transaction design
• Performance diagnosis
• Security vulnerability assessment
• Root-cause analysis for production incidents
• Complex dependency upgrades
• Migration plans involving historical compatibility

The issue is not that agents can never help with these tasks. Rather, they must not be allowed to make unsupported conclusions or apply broad changes without review.

Do not delegate directly

Avoid giving an agent direct responsibility for:

• Vague requirements without acceptance criteria
• Context containing sensitive data that has not been sanitized
• Irreversible production operations
• Final decisions involving compliance obligations
• Domains where the team itself cannot clearly explain the business rules

In these situations, the more confident the agent sounds, the greater the risk may be.

A practical rule is this: if the result can be verified within a few minutes using tests, compilation, static checks, or a small-scale reproduction, the task is usually a good candidate for agent assistance. If verification takes more than 30 minutes and depends on substantial implicit knowledge, ask the agent to produce an investigation plan and an evidence checklist instead of directly modifying the code.

3. Engineering a Reliable Agent: Tasks, Permissions, Context, and Rollback

Turning an agent from an occasionally impressive tool into a consistently useful one requires engineering constraints, not simply more features. At minimum, a reliable setup should include the following.

1. Define task boundaries clearly

The task description should specify:

• The objective
• What is out of scope
• Directories that may be modified
• Directories that must not be modified
• Validation commands
• What to do when validation fails

Vague tasks give the agent too much freedom and increase review costs.

2. Use least privilege and ensure rollback

The agent should operate in an isolated branch or temporary workspace and have only the read and write permissions required for the task.

For databases, deployments, secrets, and external services, access should be read-only or disabled by default. Every change should be reviewable and reversible.

3. Manage context deliberately

Provide the project structure, key module descriptions, dependency relationships, build and test procedures, and coding conventions. This is generally more effective than supplying a large amount of unrelated code.

More context is not always better. Relevant context is better.

4. Use staged planning and validation

For complex work, first ask the agent to produce a plan. Then proceed through a cycle of small changes, validation, and result recording.

Each step should be independently reviewable. This reduces the risk of receiving a large, difficult-to-control change in a single pass.

5. Require an evidence chain

For defect reports and performance conclusions, require the agent to identify:

• Relevant files
• Functions
• Call paths
• Assumptions
• Reproduction methods
• Test results

A conclusion without supporting evidence should be treated only as an investigation lead.

Project instruction files

A project instruction file is one of the lowest-cost, highest-value ways to improve agent behavior. It does not need a complicated format. It only needs to be stable, concise, and executable.

# Project Instructions
## Objective
Refactor the specified module locally without changing external interfaces.
## Build and Validation
- Use the project’s standard build command.
- After making changes, run the relevant unit tests and static checks.
- If validation fails, explain the failure before expanding the scope of the changes.
## Coding Constraints
- Follow the existing directory structure and naming conventions.
- Do not introduce new dependencies unless explicitly required.
- Do not modify database migration scripts, authorization settings, or production configuration.
- For concurrency, transaction, or security logic, document assumptions and risks separately.
## Output Requirements
- Provide a plan before making changes.
- List modified files, validation results, and uncovered risks.
- Mark uncertain items as "Needs confirmation."
- Do not invent interfaces, parameters, or performance data.

The value of this file is that it moves team conventions to the front of the process, makes review standards explicit, and limits the agent’s freedom to improvise. Much of the difference between a frustrating and a productive agent experience comes down to the clarity of the instructions and whether validation is built into the workflow.

4. A Practical Workflow: From “Make It Write” to “Make It Verifiably Write”

A reusable workflow consists of task decomposition, context preparation, plan review, incremental execution, automated validation, human review, and rule refinement.

During task decomposition, replace “optimize system performance” with a narrower objective such as “identify candidate slow-query locations for this endpoint under a specific input and provide a reproducible validation plan.”

During context preparation, provide only the relevant modules, log excerpts, interface definitions, and test entry points. During plan review, check whether the agent understands the boundaries and has listed its assumptions.

During incremental execution, make only one independently verifiable change at a time. During automated validation, run tests, static checks, and formatting checks. During human review, focus on business constraints, exception paths, authorization, and data consistency.

Finally, turn recurring problems into project rules and add them to the instruction file. This converts individual lessons into team assets.

The most important principle is to validate assumptions before accepting conclusions.

For example, if an agent reports a concurrency problem, do not immediately change the locking strategy. First ask it to identify the call chain, shared state, critical section, and a reproducible test. If it cannot provide these, downgrade the report to an investigation lead.

This approach preserves the agent’s ability to generate useful hypotheses without allowing unsupported claims to dictate engineering decisions.

5. Toolchain Positioning: A Productivity Orchestration Layer, Not a Replacement for Engineering

An AI coding agent should not be positioned as a replacement for developers. It is better understood as an orchestratable productivity layer within the engineering system.

Above it are requirements, tasks, and project standards. Below it are version control, code review, static analysis, testing, builds, deployment, logging, and monitoring.

Its value does not come from generating impressive code in isolation. It comes from fitting into the existing engineering system and participating in a complete validation loop.

For example, on Baidu AI Cloud, a team can reuse general-purpose services such as container platforms, logging, and monitoring to build environments for isolated execution, change observation, and rollback validation. The agent itself should still follow least-privilege, auditability, and human-approval principles.

The correct division of responsibility is:

• The agent proposes candidate changes.
• Tests and static analysis attempt to disprove them quickly.
• Code review handles business judgment.
• Monitoring and alerting provide production safeguards.

If any of these links is missing, faster generation can simply become a larger review burden.

6. Team Adoption: Use Metrics, Not Enthusiasm, to Drive Adoption

Teams should begin with a limited pilot and expand gradually. Useful metrics may include:

• Agent suggestion acceptance rate
• Rework rate
• Average verification time
• Escaped defects
• Changes in test coverage
• Distribution of code review comments

Do not focus only on how much code the agent generates. Measure how much effective verification time it saves and whether it increases the review burden.

Teams should also establish three guardrails:

1. High-risk directories must not be modified automatically.
2. Changes involving data, authorization, billing, or security logic require human review.
3. Every agent-generated change must include validation results and a risk description.

Project instruction files should be version-controlled and updated regularly based on rework cases.

For developers, the most important skill is not writing sophisticated prompts. It is breaking work into small, verifiable tasks and requiring the agent to provide evidence.

So, does this technology actually work?

As a general-purpose AI coding agent, it is highly effective for pattern-based production, test completion, localized refactoring, and documentation maintenance. It cannot replace engineering validation or human judgment when factual guarantees, complex causal reasoning, or high-risk decisions are involved.

The results depend less on the product name than on scenario selection, instruction quality, permission boundaries, and the completeness of the validation loop.

Place the agent in the right role, and it becomes a reliable accelerator. Treat it as a silver bullet, and it will waste your time with content that merely sounds reasonable.