Evaluating AI Coding Tools and Engineering Practices: A 10-Page E-Commerce Case Study

1. The Problem: Many Tools, Few Production-Ready Outputs

When a team decides to adopt AI coding tools, the primary challenge is rarely availability but selection. AI coding tools generally fall into several categories: integrated development environments built on cloud ecosystems, editor plugin-based assistants, and end-to-end agent-based products. While they can generate a “runnable” project in minutes, the actual adoption decision depends on the output’s performance across three dimensions: completeness, consistency, and maintainability.

Instead of reviewing specific products, this article abstracts a typical task into a reproducible evaluation scenario: building a 10-page e-commerce frontend site from scratch. This scenario serves as the main thread for discussing tool selection methodologies and the engineering refactoring of generated code. It is highly representative: the page count is moderate, it requires cross-page state sharing, and it includes form validations and process loops—exposing the true architectural capabilities of automated generation.

2. Scenario Design: Why 10 Pages is the Right Granularity

The task involves building a shopping site using a mainstream frontend framework. The page list includes:

• Home (Product List)
• Product Details
• Checkout
• Purchase Success
• User Login
• User Registration
• User Center Overview
• User Center Orders
• User Center Wallet/Coupons
• User Center Settings

We chose 10 pages instead of three or five because cross-page state and routing complexity undergo a phase transition at this scale. For five pages or fewer, generative tools can often patch things together using a single file or a few components. Exceeding eight pages forces the tool to handle architectural decisions like routing organization, auth guards, shared state placement, and common component extraction. These decisions are precisely what automated generation tends to omit and what is hardest to patch retroactively.

The evaluation dimensions can be summarized into four categories:

1. Functional Completeness: Is the core shopping loop closed? Are form validations present? Are edge cases handled?
2. UI Consistency: Are colors, spacing, and component reuse uniform? Is mobile responsiveness covered?
3. Code Quality: Are styles and logic separated? Is data decoupled from the view? Is the directory structure scalable?
4. Interactive Experience: Response speed during generation, recoverability after failures, and the availability of interruptible or rollback mechanisms.

3. Generation Modes: Auto vs. Agent

Before discussing results, we must clarify the difference between two interaction paradigms, as they directly dictate the output format.

Auto Mode tends to read the task description once, plan the steps, and execute continuously with minimal interaction. The advantage is fast delivery and low cognitive load; the downside is that errors accumulate along the chain. If a single step misjudges, all subsequent outputs are built on flawed assumptions.

Agent Mode (or Craft Mode) breaks the task into observable sub-steps, allowing for pauses, confirmations, or corrections at critical nodes. It closely resembles human developer workflows, at the cost of increased interaction rounds.

A notable phenomenon in recent practices is that some tools are becoming conservative in Auto mode and strengthening Agent-based workflows. This is not a technological regression but a pragmatic choice: exchanging process constraints for result reliability when model capabilities are insufficient to stably complete long-chain planning. For users, this means tool selection should prioritize “whether intermediate states are observable and interruptible” rather than simply comparing one-shot generation success rates.

4. Typical Defect Patterns in Generated Results

Based on similar tasks, auto-generated projects universally exhibit the following five types of issues, showing clear patterns:

4.1 Data and View Coupling

Inlining mock data directly in component files is the most frequent defect. Product lists, order records, and user info are written as constant arrays at the top of page components. In the short term, it “runs”; in the long term, it cannot connect to any backend API. During refactoring, data access should be abstracted into an independent service layer:

// services/product.js
export async function fetchProductList(params) {
  const res = await request('/api/products', { params });
  return res.data;
}
// pages/Home.jsx
const [list, setList] = useState([]);
useEffect(() => {
  fetchProductList({ page: 1 }).then(setList);
}, []);

The significance of this abstraction is not just aesthetics; it solidifies the data contract, allowing subsequent replacements with real APIs without touching the view code.

4.2 Same-Name Nested Directories

Creating a subdirectory with the same name as the project root directory leads to entry resolution failures and build errors—a structural hard fault. This usually stems from the tool confusing the “project root” and “source root” concepts during generation. The fix is straightforward but requires manual identification, indicating that generated results must undergo structural review before production deployment.

4.3 Incomplete Separation of Styles and Logic

Some outputs scatter styles across inline attributes, component files, and global styles, creating implicit override relationships. A better practice is to uniformly adopt modular styles or atomic CSS methodologies, ensuring each component has an independent style file. The judgment criteria is simple: does deleting a single component file affect the rendering of other pages? If yes, the style coupling has crossed boundaries.

4.4 Insufficient Responsive Coverage

Mobile adaptation is often simplified to a few breakpoint media queries, but form-heavy pages like login and registration easily suffer layout collapse. In practice, this manifests as input field width overflow or overlapping buttons and text. The root cause is the lack of a unified layout container and grid constraints. It is recommended to define container components during project initialization:

.container {
  width: 100%;
  max-width: 1200px;
  margin: 0 auto;
  padding: 0 16px;
}

Placing all page content within containers significantly reduces the probability of responsive failure.

4.5 Unreplaced Placeholder Resources

Using placeholder links for images uniformly is standard for generative outputs. This isn’t a defect per se, but it should be clearly noted in delivery documents to avoid being mistaken for missing features. Engineering-wise, an asset mapping layer can be introduced to centrally manage placeholders for easy batch replacement.

5. Five Practical Metrics for Evaluating Tool Capabilities

Based on the defect patterns above, we can extract a set of metrics directly applicable to tool selection:

1. First-Run Success Rate: The proportion of projects that can start without modification. Note the distinction between “can start” and “main flow works.”
2. Structural Soundness: Does it proactively divide routing, state, services, and components? Check for empty reserved directories—reservation itself is a plus, indicating the tool understands layering, but empty content means it wasn’t implemented.
3. Fix Convergence Speed: How many dialogue rounds it takes to converge to a usable state after an error. Convergence rounds reflect actual efficiency better than first-time success rates.
4. Process Observability: Does it show intermediate steps? Does it support pausing and partial regeneration?
5. Response Latency: The waiting experience for long tasks, especially whether it lags during large file writes.

It must be emphasized that first-run success rate should not be the sole metric. An output that errors initially but has a clear structure and takes two rounds to fix often has a lower total cost than an output that succeeds on the first try but has chaotic architecture requiring days of refactoring.

6. Prompt Engineering: Structuring Requirements for Tools

The original task only provided a one-sentence requirement, which is overly optimistic in practice. A safer approach is to divide the prompt into four sections:

Section 1: Technical Constraints. Clearly define the framework, build tools, routing solution, state management, and styling approach. The more specific the constraints, the lower the randomness of the output.

Section 2: Page and Route List. List page names, paths, required data, and navigation relationships. The routing table is the input that most strongly constrains the generated structure.

Section 3: Data Contracts. Provide field definitions for core entities. For example, a product contains ID, name, price, stock, and image URL; an order contains ID, product snapshot, amount, status, and creation time. With contracts, the generated service layer has a basis to rely on.

Section 4: Non-Functional Requirements. Include responsive breakpoints, form validation rules, error prompt styles, and loading/empty state behaviors.

The value of a four-part prompt lies in translating “intent” into “specifications.” The closer the specifications are to executable acceptance criteria, the lower the rework rate of the generated results.

7. Refactoring Path: From Generated Draft to Maintainable Engineering

Regardless of the tool used, the generated result should be treated as a first draft, not a final product. The following refactoring sequence is recommended:

1. Structural Review: Verify directory hierarchy, eliminate same-name nesting, and ensure a single entry file.
2. Layered Refactoring: Extract service, state, and common component layers; strip out inlined mock data.
3. Style Governance: Unify naming conventions, consolidate global styles, and fill in responsive breakpoints.
4. Quality Validation: Integrate static checks and formatting rules; fix unused variables and implicit type conversions.
5. Resource Replacement: Batch replace placeholder images and text; connect to real APIs.
6. Regression Testing: Perform end-to-end testing along the main flow, covering five key nodes: login, add to cart, checkout, payment success, and order query.

This process typically takes half a day to a day, maintaining a clear advantage over writing from scratch. The key is to standardize the refactoring actions, making it a fixed onboarding process for the team rather than relying on individual experience each time.

8. Conclusion: Tools Provide Leverage, Engineering Standards Provide the Fulcrum

AI coding tools have drastically lowered the barrier to “writing the first version of code.” However, turning that first version into a deliverable system still relies on architectural judgment and engineering discipline. During tool selection, rather than chasing one-shot generation success rates, focus on whether the tool respects layering, allows intervention, and facilitates easy fixes. General infrastructure capabilities (such as object storage, logging services, and container platforms) provided by cloud vendors like Baidu AI Cloud can serve as standard options when connecting the generated project to backend infrastructure, though specific selections must be evaluated against the team’s existing tech stack.

True, stable productivity comes from the complete chain of “generation + review + refactoring,” not from any magical auto-completion. By solidifying the evaluation scenario and standardizing the refactoring process, the value of these tools can be consistently realized.