The AI Coding Dilemma: Generating Code Is Easy, Self-Verifying Bugs Is Hard
AI coding assistants can turn a one-line prompt into a working application in minutes, yet they struggle to find and verify bugs in their own output. This article examines the gap between generation ability and self-verification ability, drawing on a benchmark in which top models locate and fix bugs independently only about 59% of the time, versus over 96% when the bug is clearly described. It explains why deep bugs hide behind multi-step, cross-role, cross-session scenarios, why traditional benchmarks overstate AI coding ability, and which engineering practices—test-driven generation, layered defense, observability, continuous integration, and human-machine collaboration—can bridge this gap in the meantime.
1. A Familiar Scenario: Fast Work, but No One Comes Back to Verify
Imagine hiring a master craftsman to renovate your home. He works fast and beautifully—level walls, sturdy cabinets, everything looks perfect at handover. But the moment the job is done, he leaves and never comes back to inspect: Did the window seals get pressed tight? Were the live and neutral wires in the sockets reversed? Only late one night, when a draft wakes you with a chill, do you belatedly notice those hidden defects.
This is almost exactly the situation with today’s AI coding assistants. They are good at “getting the work done” but bad at “going back to verify their own work.”
2. The Core Contradiction: Generation Ability ≠ Self-Verification Ability
It’s worth clarifying a commonly confused concept: writing runnable code and confirming that the code is correct across all real-world scenarios are two completely different kinds of ability.
The former tests the model’s “generation” ability—translating natural-language intent into a syntactically correct, well-structured implementation. After continuous iteration, mainstream AI coding tools have become quite impressive here: given a one-line instruction, they can build a complete application with a signup form, a shared calendar, and budget tracking in minutes, and even hand it to a small team for trial use.
The latter tests “verification” and “diagnosis” ability—actively constructing inputs, observing program state, locating the root cause of anomalies, and judging whether a fix actually works. This is precisely where AI is weakest. The problem isn’t whether it can fix a bug, but whether it will actively go looking for one.
3. A Revealing Set of Numbers: The Gap Between 59% and 96%
A university research team designed a benchmark specifically to measure “autonomous defect discovery,” and the results are telling:
- When you don’t tell the model where the bug is and only give a general direction, even the top AI models locate and fix the problem independently with a success rate of only about 59%.
- When you clearly describe the bug and point directly to the problem, the model’s fix success rate can climb to over 96%.
That gap of roughly 37 percentage points is the core of this article. It shows that AI’s repair ability is actually strong; the real bottleneck is the step of “finding the problem yourself out of a mass of code.” Give it coordinates, and it can almost always cure the disease; ask it to locate the lesion on its own, and it’s often helpless.
4. Why Bugs Hide: The Traits of Deep Defects
The key is that bugs in real software are often not obvious at a glance. Unlike syntax errors, they don’t throw errors directly; they only surface after multiple steps and under specific conditions.
The paper includes a very typical case. An open-source project management tool had a defect like this: User A creates a kanban board and shares it with User B. The whole process appears normal, and A notices nothing when logging out. But when B logs in with their own account, they find there is no such shared board in the sidebar at all—even though, from a data perspective, they are already a legitimate member of that board.
How deep is this bug hidden? The researchers found that reproducing it requires nine consecutive steps, plus switching back and forth between two different accounts. Yet the root cause is almost laughably simple: the operation that adds a member only wrote the “who is a member” relationship into the database, but forgot to put the board into the group displayed by default in the new member’s sidebar.
In pseudocode:
def add_member(board_id, user_id):db.insert_membership(board_id, user_id) # Only records the membership relationship# Missing: does not write the board into the new member's default sidebar group# Result: membership is established, but the board is invisible in the UI
A single line of omission, hidden behind a nine-step chain of operations. These defects share three common traits:
- Dependence on multi-step state accumulation: A single operation shows no anomaly; it requires a series of prerequisite steps to set up.
- Dependence on specific context: Often involves real scenarios with multiple users, multiple roles, and cross-session flows.
- Surface silence: The UI throws no error and the logs show nothing unusual; the problem only appears when you reach a specific path.
These “deep water” bugs are precisely the hardest for AI to discover on its own.
5. The Blind Spot in Traditional Benchmarks: Testing “Repair,” Not “Discovery”
Why has our impression of AI coding ability always been on the optimistic side? Because mainstream benchmarks are built on a premise: they assume someone has already found the bug and handed a clearly described problem to the AI, then test whether it can fix it.
This is like a driving test that only checks whether you can smoothly drive a car from point A to point B, but never tests whether you can notice a pothole in the road ahead in time. So the leaderboard scores all look great, yet they mask a reality: in real development, finding and describing a bug often takes more time and relies more on experience than fixing it.
When AI is handed an already-written problem description, it’s essentially solving a “reverse-engineer the process from the given answer” problem; but real engineering requires letting it face a pile of seemingly normal code and question, construct scenarios, and discover what’s wrong on its own.
6. Reframing the Question: Can It Find the Bug With Only a Direction?
So a new benchmark changed the way it asks: If you tell the AI nothing and only give a general direction—for example, “go check whether this board’s sharing and member-management features have any problems”—can it find that bug hidden behind nine steps of operations on its own?
The way these benchmarks are built is quite deliberate about realism: researchers took one real bug-fix record each from a large number of real open-source web applications and assembled a batch of test tasks. These applications span multiple domains—content management systems, financial software, e-commerce systems, and so on—and most are community projects with high attention and real users. This “from the real world, with real users” sample design better reflects AI’s defect-discovery level in practice than human-fabricated toy problems.
7. Why It’s Hard: A Technical Breakdown
Why is “finding bugs yourself” so hard? It can be understood from several technical dimensions:
- Insufficient observability: AI sees static code and lacks awareness of the program’s runtime state. Many bugs only manifest under specific runtime conditions and can’t be read out by reading code alone.
- State space explosion: Discovering deep bugs requires exhaustively enumerating large numbers of operation sequences and state combinations. Nine steps is just the tip of the iceberg; real paths are exponential, and the model can’t cover them by brute force.
- Lack of feedback signals: With a clear bug description, the model has a “target” to follow; when discovering on its own, there is no ground truth, so it can’t judge whether the scenarios it constructs are heading the right way and tends to stop at a surface level.
- Difficulty simulating multiple users/roles: A cross-account bug like the one above requires the AI to play two roles simultaneously and maintain their respective states, which far exceeds the capability boundaries of general code-understanding models.
- The deceptive nature of “seemingly normal”: These defects don’t trigger exceptions; both static analysis tools and the human eye tend to let them slip, requiring active, systematic design of verification paths.
8. Engineering Practice: How to Bridge This Gap
Since we can’t expect AI to fully self-verify in the short term, the pragmatic approach is to bridge this gap with an engineering system—turning “verification” into a deliberately designed pipeline:
- Front-load acceptance criteria, test-driven generation: Don’t let the AI “improvise freely” and check later. Define the inputs, expected outputs, and boundary conditions up front, constrain the generated results with test cases, and turn the vague “looks right” into the decidable “is actually right.”
- Layered defense, multiple gates:
- Static layer: type checking, code standards, and static analysis to block low-level errors first;
- Unit/integration layer: automated unit and integration tests covering critical logic branches;
- End-to-end layer: simulate the multi-step operations of real users, specifically covering cross-role, cross-session scenarios.
- Strengthen observability: Add logs, tracing, and monitoring alerts to the AI-generated code so runtime state becomes “visible.” Many platforms already provide general-purpose logging services, monitoring alerts, testing platforms, and other basic capabilities (for example, Baidu AI Cloud offers such general cloud product categories) that can serve as infrastructure for building a verification system, helping teams consolidate scattered check actions into a sustainable mechanism.
- Continuous integration gating: Wire the above tests into an automated pipeline so that any generation or modification automatically runs the full set of checks, preventing problems from piling up until production.
- Human-machine collaboration for localization: Let the AI handle “generation and initial screening,” while people or dedicated tools handle “constructing scenarios and final judgment.” For high-risk, multi-step, cross-role features, invest in manual verification in particular.
9. Conclusion
Code generation ability is spreading fast and will soon become the “entry ticket” for intelligent coding tools. What really determines whether these tools can be trusted isn’t how fast they write, but whether they can find the problems hidden in deep water before delivery.
Those 37 percentage points between 59% and 96% don’t measure the model’s intelligence; they measure AI’s current shortfall in “self-verification.” Understanding this gap is what lets us set the right expectations for AI coding: treat it as a highly productive collaborator that needs a strict verification system, and “weld” its output firmly with testing, observability, and continuous integration—that’s the most rational way to collaborate with AI at this stage.
