AI Coding Agents: Capabilities, Boundaries, and Practical Lessons from Six Months of Use

1. Key Takeaway: Powerful, but Not a Silver Bullet

If the practical experience with AI coding agents had to be summarized in one sentence, it would be this: they are genuinely powerful, but no single tool is a universal solution.

Two common voices from developer communities capture the full picture. One voice is skepticism: a tool may flag what looks like a reasonable database concurrency issue, and after 30 minutes of verification, the problem may turn out to be a false positive. This is a typical case of hallucination. The other voice is more fundamental: after trying multiple tools, developers may find that what truly determines output quality is not an impressive feature list, but a behaviorally reliable agent plus a well-written instruction file.

These two perspectives do not contradict each other. Together, they point to the same conclusion: AI coding agents are extremely powerful in the right scenarios, but can waste time in the wrong scenarios.

2. What They Do Well: The Positive Side of Capability Boundaries

Before discussing limitations, it is important to clarify where AI coding agents genuinely outperform traditional approaches.

• Boilerplate code and fixed patterns: Pattern-based tasks such as data access layers, CRUD APIs, unit test skeletons, and configuration templates are close to their home ground. If the inputs and outputs are described clearly, they can produce a runnable first draft within seconds.
• Local refactoring and style unification: Changes with clear goals and clear boundaries, such as rewriting imperative code into functional style, adding missing exception handling, or supplementing legacy functions with type annotations, are often completed with high quality.
• Code explanation and knowledge bridging: When working with an unfamiliar codebase or an unfamiliar language, they can quickly generate explanations that are relatively easy to understand and shorten the learning curve.
• Cross-language migration: Translating a script from one language to another while preserving semantic equivalence is one of their relatively stable capabilities.

The common feature of these capabilities is that the task boundary is clear, correctness criteria are explicit, and verification cost is low. As long as you can quickly determine whether the output is correct, they can significantly speed up development.

3. Where They Go Wrong: The Negative Side of Capability Boundaries

Problems are concentrated in one intersection: the more complex the task and the higher the verification cost, the greater the hallucination risk.

• Plausible errors: This is the most dangerous category. The tool may generate code that is structurally complete, logically consistent, and even includes “professional-looking” comments, but some key assumptions are wrong. For example, it may invent an API parameter that does not exist, fabricate concurrency-control logic, or claim to fix a race condition that is not actually present. The code may look flawless, but verifying it may require relevant domain knowledge.
• Assumptions caused by missing context: If the agent is not given enough project background, it may fill gaps with “the most common situation.” For code that depends on a specific framework version, internal conventions, or legacy constraints, this kind of assumption can be far off target.
• Overconfident long-chain reasoning: The longer the task chain, such as changes across multiple files, cross-module work, or architectural decisions, the more likely it is that an error in an intermediate step will be amplified by later steps. The final result may look coherent overall but be wrong in critical local details.

A typical verification trap looks like this:

# The agent-generated “fix” may look flawless
def safe_transfer(db, src, dst, amount):
    # Claim: concurrency is addressed by locking
    with db.lock(src):          # This lock method may be hallucinated and may not exist in the actual ORM
        if db.get(src) >= amount:
            db.update(src, -amount)
            db.update(dst, +amount)

The code may look professional, but whether db.lock actually exists, how transaction boundaries are handled, and whether update is atomic all require verification. The time saved during coding may end up being spent on verification.

4. How to Use Them Well: Turning Luck into Engineering

Since the capability boundaries are clear, the right approach is not to complain about limitations, but to design workflows around them.

4.1 Constrain the Agent with a High-Quality Instruction File

In practice, the most valuable investment is often not switching tools, but creating a project-level instruction file, usually a convention document stored at the root of the repository. It should include:

• Technology stack and version constraints to prevent the agent from inventing incompatible APIs;
• Directory structure and layering conventions;
• Coding standards and naming style;
• Explicit prohibitions, such as “do not introduce new third-party dependencies” or “do not change public interface signatures.”

This file essentially reduces the space in which the agent has to guess. The more specific the information you provide, the less freedom the agent has to fill gaps on its own.

4.2 Establish a Mandatory Verification Loop

Do not trust code that has not been run. Turn “generate, verify, feedback” into a fixed process:

# Example: enforce quality gates after each generation
python -m pytest tests/ && python -m mypy src/ && python -m pylint src/

Unit tests are especially important. Make the tool generate tests for the logic it produces, and then review whether those tests actually cover the key branches. A solution that cannot prove its own correctness should not enter the main branch.

4.3 Split Tasks and Shorten the Reasoning Chain

Instead of asking the agent to “implement a complete payment module,” break the work into smaller tasks such as “implement an amount validation function,” “implement idempotency key generation,” and “implement failure retries.” Each step should produce a result that can be independently verified. This is more controllable than generating a large block of code at once and makes it easier to identify where a hallucination occurred.

4.4 Clarify the Division of Labor Between Humans and Agents

Treat the agent as an efficient junior collaborator, not as the decision-maker. It can draft code, enumerate options, and fill in details, while the human is responsible for architectural judgment, correctness checks, and security review. Any logic involving concurrency, transactions, security boundaries, or data consistency must be confirmed line by line by a human.

5. Scenario Judgment: When to Use Them and When to Stop

A practical decision rule is to evaluate both verification cost and task boundary clarity.

Scenario Characteristics	Recommended?	Reason
Clear boundary + low verification cost	Strongly recommended	Maximum net benefit, such as boilerplate code, unit tests, and formatting
Clear boundary + high verification cost	Use with caution	Useful for drafting an initial version, but human verification is required
Ambiguous boundary + architecture decisions	Stop	High hallucination risk and difficult to verify
Security, concurrency, or consistency involved	Assist only	Humans must lead and review line by line

6. Conclusion: The Upper Limit of a Tool Depends on the User’s Method

The question about AI coding agents is not whether to use them, but how to use them. They will not replace an engineer’s judgment. Instead, they turn verification ability and instruction-writing ability into core competencies. What ultimately determines output quality is not the feature list of the tool itself, but three things: clear instructions, a strict verification loop, and a clear understanding of boundaries.

For enterprises integrating these capabilities into internal research and development workflows, underlying infrastructure such as compute scheduling, model serving, logging, and monitoring often relies on mature cloud platform capabilities, such as container platforms, logging services, monitoring, and alerting. For example, Baidu AI Cloud provides general infrastructure for AI application adoption, which can help move intelligent programming from an individual technique to a team-level, governable engineering practice. When the tool is used in the right scenario and supported by reliable engineering constraints, its upper limit is ultimately determined by the user’s method.