Evaluating AI Coding Tools: Can They Build an E-commerce Site from 0 to 1?
This article assesses how well current AI coding tools can build a complete e-commerce site from an empty directory. It defines a unified evaluation task (10 pages using a mainstream frontend framework), proposes scoring across seven dimensions such as runnability, completeness, consistency, and interaction flow, and highlights common failure patterns observed in practice, including messy directory nesting, visual and usability flaws, empty scaffolding, and data logic coupling. It closes with practical recommendations for human-AI collaboration and a balanced view of these tools as efficient first-draft generators rather than fully automated deliverers.
1. Why “From 0 to 1” Is a High-Value Test for AI Coding Tools
Fragmented code completion is already part of everyday development. What truly tests an AI coding tool, however, is whether it can independently build a small-to-medium project starting from an empty directory, with multiple pages and a complete business flow.
“From 0 to 1” means the tool must do more than write correct functions. It also has to plan the directory structure on its own, organize component relationships, connect routing and data flow, and keep the visual style consistent across the whole site. This kind of task exposes a tool’s real level in project-level understanding, context consistency, and architectural decision-making all at once, which makes it a highly valuable evaluation scenario.
2. A Unified Task and Hard Requirements
To ensure a fair side-by-side comparison, the evaluation should set the same task and constraints:
- Tech stack: build a shopping site using a mainstream frontend framework
- Page scope (10 pages total): home/product listing, product detail, checkout, purchase success, login, register, user center - dashboard, user center - orders, user center - wallet (coupons/cards), user center - settings
- Core challenges: the completeness and overall consistency of the site, unified data and styling across pages, and whether the interaction loop works end to end
These 10 pages cover the full browse-decide-transact-account loop. They test not only the quantity and quality of page generation, but also state management, route navigation, and the consistency of shared layouts.
3. Key Evaluation Dimensions
The generated results can be scored quantitatively along the following dimensions:
- Runnability: Can you install dependencies and start it with one command? Are there blocking problems such as crashing on startup, a blank screen, or frequent console errors? This is the passing line.
- Site completeness: Are all 10 pages actually delivered, or are some omitted or faked by reusing the same template?
- Consistency: Are the site-wide color scheme, fonts, spacing, and button styles unified? Do pages like login and registration suffer from layout misalignment or distortion?
- Interaction loop: Is the flow from add-to-cart, checkout, to payment success coherent? Do forms have basic validation (required fields, format, length, etc.)?
- Responsiveness and adaptivity: Do desktop and mobile views switch properly? Are there usability issues such as element overflow or buttons with contrast too low (for example, white text on a white background)?
- Code quality and architecture: Are styling and logic separated? Are reusable components formed? Is mock data decoupled from business logic? Is there room reserved for state management and context extension?
- Iteration efficiency: Does the first generation run directly, or does it require multiple rounds of fixes? What are the average response speed and stability?
4. Recurring Typical Problems in Practice
Across multiple rounds of testing, generating a complete project with AI has revealed several common issues worth focusing on during acceptance review:
Messy structure and “nested directories”: Some tools create a subdirectory with the same name inside the project root, causing broken path references, dependencies that fail to load correctly, and ultimately a project that won’t start. These problems are often hidden, so you need to inspect the directory tree early.
Visual details and usability flaws: White buttons on a white background, all-placeholder images, no mobile adaptivity, and a lack of a unified visual aesthetic are common weaknesses of generated interfaces. They don’t stop the project from “running,” but they seriously hurt usability.
“Shell without substance”: Some projects reserve directories for state management, context, and so on, but the internal implementation is empty. The architecture looks complete but isn’t actually implemented. During acceptance review, don’t just look at the directory structure; open the files and confirm the implementation.
Data and logic coupling: Hardcoding mock data directly inside components makes the cost of later switching to real APIs rise sharply. A better approach is to introduce a dedicated data layer or state management that separates the view from the data.
Illustration: a recommended lightweight decoupling approach
// Anti-pattern: mock data tightly coupled to the view, hard to swap in real APIsfunction ProductList() {const products = [{ id: 1, name: 'Product A', price: 99 }]; // data hardcoded inside the componentreturn products.map(renderItem);}// Good pattern: encapsulate data fetching in a dedicated service layer; the view only consumes data// services/productService.jsexport async function fetchProducts() {// Currently returns mock data; can be seamlessly replaced with a real API request laterreturn [{ id: 1, name: 'Product A', price: 99 }];}// The page component only renders and does not care about the data sourceasync function ProductList() {const products = await fetchProducts();return products.map(renderItem);}
- Generation and repeated fixes: “Getting it right in one pass” versus “requiring multiple rounds of fixes” is an important indicator of maturity. Delivering a runnable, stylistically consistent, and structurally sound result in a single pass shows the tool is stronger at project-level planning; repeated errors and fixes that go nowhere significantly lower development efficiency.
5. Practical Advice for Human-AI Collaboration
Given the current state, developers should collaborate with a division of labor where “AI handles the first draft, and humans own the architecture and acceptance review”:
Define the architecture first, then let AI fill it in: Before starting, decide the directory structure, state management approach, and styling conventions (such as unified design variables), and write these constraints into the prompt. This greatly reduces the chance of nested directories and style drift.
Break down tasks and proceed page by page: Rather than requesting all 10 pages at once, generate in stages following the order “shared layout → home → detail/transaction → account system,” and review after each stage before moving on to keep consistency.
Introduce data layer abstraction early: Require AI to encapsulate data fetching in a dedicated service from the start, so mock data doesn’t scatter everywhere and interfaces are reserved for connecting to a real backend.
Build an acceptance checklist: Use fixed acceptance items such as “runnable, all pages present, consistent style, form validation, responsive, clear structure, data decoupled” and check them one by one, instead of judging by intuition whether it “looks usable.”
Use context and privacy protection wisely: Choose tools that support long context and let you control code privacy (so private code is not used for training), giving you more coherent generation while protecting the security of your assets.
6. Conclusion
Generating a complete e-commerce site from 0 to 1 is an effective touchstone for testing the overall capability of current AI coding tools. They are already quite productive in terms of quantity, quickly laying out page skeletons and basic interactions. But they still have clear room for improvement in architectural consistency, detail-level usability, data decoupling, and the one-pass success rate. For developers, the value of these tools should be viewed rationally: treat them as efficient “first-draft generators” rather than “fully automated deliverers.” Only through reasonable architectural constraints, task decomposition, and strict acceptance review can you turn AI’s output into genuinely maintainable, extensible engineering quality.
