Evaluating Intelligent Coding Assistants: An E-Commerce Implementation Practice

    This article explores how to evaluate intelligent coding assistants by building a demo e-commerce site. It covers defining acceptance criteria, breaking down tasks, selecting generation modes, establishing a minimum maintainable structure, and implementing a structured error-fixing loop to turn AI coding tools into reliable delivery assistants.

    Network accessibility and local-first deployment are often the primary constraints for teams adopting intelligent coding assistants. When selecting a tool, if the focus remains solely on whether the UI resembles certain overseas tools or if it supports autopilot-style one-click generation, teams easily overlook the factors that truly determine delivery quality: whether the task is completed in one go, whether the generated code runs, whether styles and data are decoupled, and whether errors can be fixed within the same context. Below, we abstract practical observations into reusable engineering methods using “building a demo e-commerce site from scratch” as an example. The site scope is limited to three pages: a product purchase page, a payment/order success page, and an order list in the user center. A smaller scope makes it easier to horizontally compare different interaction modes rather than competing on who generates more irrelevant pages.

    1. Define Acceptance Criteria Before Discussing UX

    A friendly interface lowers the psychological barrier to entry, but being “approachable” cannot replace being executable. It is recommended to write down four acceptance criteria before starting, to avoid subjective arguments after generation is complete.

    First, three user paths must be connected: from product information to purchase confirmation, submission to the success page with an order identifier, and viewing the order status in the user center using the same identifier. Second, demo data must be decoupled from the UI; product name, price, stock, and order status should come from an independent data module or local API stubs, not scattered within button click handlers. Third, styles and structure must be separated; at a minimum, define style boundaries by page or component to avoid piling inline styles and business logic into a single file. Fourth, failures must be locatable: when build or runtime errors occur, logs should point to the file and cause, not just pop up a generic “generation failed” message.

    At the interface level, three things can be observed: whether the task description supports segmented constraint additions; whether the generation process exposes the current step (reading requirements, creating directories, writing pages, installing dependencies, starting preview); and whether the privacy or local-first mode clearly states if code and prompts leave the local machine. These are experience and compliance expectations and should not be packaged as a unique selling point of any specific product. If the tool offers multiple agents or “modes,” first check if the mode descriptions correspond to different risks: rapidly outputting a skeleton, generating based on architectural constraints, or merely making local patches. The name itself is not important; the input-output contract is.

    2. Break Down the E-Commerce Site into Task Packages

    Feeding “make an e-commerce website” to an assistant in one shot often results in viewable directories and clickable pages, but with hardcoded data, idle state, and the success page and order page using separate sets of fake order numbers. A more reliable approach is to break the task down into clearly constrained work packages, allowing the assistant to complete them continuously within a single context, rather than switching to a memoryless session for each step.

    Work packages can be written as follows:

    1. Information Architecture: Two core objects (Product, Order); only three pages; routes and page titles are fixed.
    2. Data Contract: Product contains ID, name, price, stock; Order contains ID, product ID, quantity, amount, status (pending, completed, closed). Amount is calculated by price multiplied by quantity in one place; duplicate formulas within pages are prohibited.
    3. Interaction: The purchase page validates that quantity does not exceed stock; the success page displays the current order in read-only mode; the order page supports filtering by status. Real payment is not required.
    4. Engineering Constraints: Styles are independent; data access goes through a single module; directories for state management and custom logic are reserved, but empty directories must be marked as “TODO” in the documentation, not pretending to be integrated.
    5. Definition of Done: Can be started with a single local command; three paths are manually clickable; intentionally changing stock to 0 should show “unavailable to purchase” rather than failing silently.

    This writing style directly targets common failure points: nested directories with the same name causing entry file pointing errors; hooks and context directories created but empty, misleading readers into thinking state has been elevated; mock data coupled inside components, requiring a full-repo string search when switching to real APIs later.

    3. Mode Selection: One-Shot, Step-by-Step, or Architecture-First

    In practice, you will roughly encounter three working methods; there is no need to blindly trust any specific name.

    One-Shot Completion is suitable for small sites with a frozen scope. Provide the entire work package above and require the output of the directory tree and data contract first, then the code. The advantage is context continuity, reducing the risk of pages writing their own order identifiers. The risk is a flat architecture: all logic squeezed into a few files, making it look “done,” but causing a steep cost increase when adding promotions, addresses, or after-sales features later. After one-shot completion, a structural review should be done immediately, rather than just checking if the pages look good.

    Step-by-Step Build is suitable for making changes while observing. Start with the data contract and a static purchase page, then the success and order pages, and finally extract styles and data access. Each step must be runnable. The advantage is a smaller error surface; the disadvantage is that if the fake data from the previous step is not contained, it will be copied in the next step. When stepping, explicitly “prohibit adding a second product list.”

    Architecture-First is suitable for cases with many constraints, few pages, but complex rules, such as inventory, state machines, and amount precision. Generate module boundaries and interface specifications first, then fill in the implementation. Slower response times are usually acceptable, but a time box must be set: if the planning phase repeatedly changes directory names or produces structures like “parent directory and subdirectory with the same name,” it should be aborted and directories manually specified, rather than continuing automatic retries. If multiple fixes still fail to start, keep the logs and go back to the contract layer to redo it; this saves more time than patching over an incorrect entry point.

    Slow speed does not automatically equal high quality. If priority can only be improved through payment, the “wait” should be factored into the trial cost, and the first-time success rate under the free quota should be compared using the same task package, rather than just comparing first-screen generation time. If the locally available version uses different models or routing strategies, the conclusions cannot be extrapolated to other network environments or versions.

    Below is a framework-agnostic layered approach. Use it as a reference after the assistant generates the code, rather than requiring the assistant to memorize a specific scaffold.

    • Page Layer: Three entry points (Purchase, Success, Orders), responsible only for layout and event forwarding.
    • Component Layer: Quantity selector, amount display, order status label. Presentation components do not directly read mock data.
    • Data Layer: Reading products and orders, creating orders, querying by ID. In the demo phase, returning in-memory data is sufficient.
    • Domain Rules: Stock validation, amount calculation, whether a status is allowed to appear in the “completed” filter. Rules are centralized; pages are prohibited from copying them.
    • Style Layer: Split by page or component; design variables (spacing, primary color, danger color) are stored in one place.
    • Reserved Layer: If directories for state sharing or reuse logic are generated, either write real calls into them or mark them as unintegrated in the README to avoid “fake architecture.”

    The schematic data entry point should look like this (pseudo-code, illustrating boundaries only, not a specific tool’s API):

    1. catalog.listProducts() -> Product[]
    2. orders.create({ productId, qty }) -> Order | { error }
    3. orders.get(id) -> Order | null
    4. orders.list({ status? }) -> Order[]

    The purchase page only calls create; the success page only uses the order identifier from the route to call get; the order page only calls list. This way, checking “whether mock data is coupled” only requires searching if any page hardcodes a price array.

    It is recommended to use explicit enums for order statuses, rather than scattered strings. Use integer cents or a clear decimal strategy for amounts, and format them into display strings in one place. The success page and order page should use the same formatting function to avoid one page displaying a currency symbol while the other displays a bare number.

    5. Post-Generation Fixing Loop

    It is common for the first generation to “start but report console errors.” The key is whether the fixes converge. It is recommended to fix a maximum of three rounds. If the three paths cannot be clicked through after three rounds, the mode is deemed a failure; switch to a smaller work package or manually specify the entry file.

    Round 1 handles only startup errors: missing dependencies, incorrect entry paths, module resolution failures caused by duplicate directories. Do not change visuals at the same time. Round 2 handles contract errors: the success page cannot retrieve the order, filter statuses do not match, quantity can exceed stock. Require the assistant to reference data layer function names to explain the modifications. Round 3 handles structural debt: extract styles, delete the second product data within the page, change empty reserved files to export empty implementations with comments, or simply delete unused directories.

    Fix prompts should include evidence: the original error text, expected behavior, and files that cannot be modified (e.g., the data contract). A vague “optimize it a bit” easily causes the assistant to rewrite the entire project, introducing new duplicate directories or a different set of routes.

    If the tool provides multiple agents simultaneously, do not run multiple agents in parallel to modify the same directory. First, select one responsible for generation, and use another for read-only review only: list gaps against the acceptance checklist. Multiple agents writing files simultaneously is a high-frequency cause of structural chaos and entry points overwriting each other. The review checklist can consist of only five questions: Are there redundant directory nestings? Is data single-sourced? Do the three paths use the same order identifier? Are styles independent? Are reserved directories actually referenced?

    6. Quality Benchmarking and Team Adoption

    At the end of the trial, do not use “friendlier interface” or “faster generation” as the sole conclusions. Use the same scoring sheet:

    • Does the one-time task run successfully within the agreed number of rounds (0 or 1).
    • Number of data coupling points (occurrences of product or order literals within pages, fewer is better).
    • Are style files separated by page, with no large blocks of duplication.
    • Number of empty implementation directories.
    • Number of blocking-level errors remaining after three rounds of fixes.
    • Are the manual steps from purchase to order list consistent with the requirements.

    When scores are close, compare privacy statements, offline or intranet availability, and whether context can be exported during collaboration. When a cloud preview or demo environment is needed, general capabilities suffice: put static assets in object storage or host the frontend via a container platform; if orders are still demo data, do not pretend payment is integrated; logs and monitoring alerts should only collect startup failures and API stub errors, avoiding writing full prompts into logs. If your team is already using container, storage, and observability services on Baidu AI Cloud, simply verify the network and permission boundaries without fabricating a separate set of APIs or performance metrics for the coding assistant.

    The e-commerce demo site looks simple, which makes it perfectly suited to expose the assistant’s true capability level: will it weld fake data into the UI just to “complete the task”; will it use empty directories to create an illusion of architecture; will it drift further off course after an error. By writing acceptance criteria into prompts, consolidating data access into a single module, and limiting fixes to three rounds, intelligent coding tools can evolve from chat toys into auditable delivery assistants. Mode names may differ, but engineering constraints should remain the same.