Testing Open-Source AI Agents: Building Full-Stack Websites from Scratch

1. Beyond Benchmark Scores: What Truly Matters

The release pace of foundation models has accelerated significantly recently. Leaderboards are constantly refreshed, with scores rising rapidly across comprehensive knowledge QA, mathematical reasoning, code completion, and software engineering tasks. While benchmark scores are valuable references, what truly determines whether a technology enters a developer’s daily workflow is its stable performance on concrete tasks.

Therefore, this article observes the capabilities of current open-source models from a different perspective: instead of debating metric improvements, we task the model with completing an assignment that mimics a real-world ticket—generating multiple brand websites with complete frontend and backend logic from scratch.

This task is representative because it simultaneously tests several core capabilities of the model:

• Long-range planning and task decomposition: Generating a website involves multiple stages, including information architecture, visual style, interaction design, and data persistence.
• Frontend engineering implementation: It requires not only visual completeness but also handling details like responsive layouts, animations, SVG interactions, and form validation.
• Backend logic implementation: Bookings, inventory, state management, and persistent storage must function rather than being static displays.
• Cross-file collaboration and self-correction: The model must maintain consistency across multiple code files and correct errors based on runtime feedback.

In other words, this is not a demo of “generating a pretty page,” but an observation of engineering capabilities focused on delivery quality.

2. Task Design: Building a Deliverable Website

To make the results evaluable, we set a clear requirement for the model: build a brand website for a low-Earth orbit travel company, which must include the following features:

1. Space capsule seat selection;
2. Flight date selection;
3. Passenger information input and booking submission;
4. Persistent storage of booking data;
5. A limit of 6 seats per flight, automatically locking once fully booked.

This requirement avoids the common issue of appearing complete but actually being static. A runnable booking backend forces the model to handle data structures, state synchronization, concurrency constraints, and user feedback.

Regarding the prompt structure, we used a layered description approach: first explaining the brand positioning and visual tone, then listing the page modules and interaction requirements, and finally adding backend functional constraints. This approach ensures the model does not ignore business logic while generating the frontend presentation layer.

3. Frontend Generation: From Visual System to Interaction Details

3.1 Hero Section: Establishing Brand Identity

The model’s output is first reflected in the hero section design. The page uses a full-screen video background showing the Earth’s curvature slowly rotating from the perspective of a space capsule window, with the edge of the atmosphere glowing blue. This treatment is not simply stacking assets but aligns closely with the “low-Earth orbit travel” theme.

A flight data panel is placed in the top right corner, using a monospace font to display key parameters such as orbital altitude, speed, and cabin pressure. In the top left, there is a breathing fluorescent cyan dot simulating an online system status. These small elements collectively form a credible aerospace mission interface.

3.2 Vertical Timeline: Organizing Information via Narrative

Below the hero section, the model generated a vertical timeline to organize the flight experience information:

• T-60 Changi Spaceport, changing into custom pressurized suits;
• T+03 Propulsion phase, enduring G-forces;
• T+08 Entering zero gravity;
• T+92 Touchdown, capsule splashes down in the ocean, recovery team on standby.

Each stage is accompanied by key orange data points and corresponding scene images. The benefit of this structure is that users can quickly understand the temporal logic and key milestones of the entire journey without reading large blocks of text.

3.3 Seat Selection: Interactive SVG

Instead of using static images, the seat selection section employs an interactive SVG graphic. Users can click on different cabins to view location, perspective, and pricing information. When generating this part, the model had to simultaneously handle graphic coordinates, mouse events, and selected state styling, adding a degree of frontend complexity.

4. Backend Logic: A Runnable Booking System

More noteworthy than the frontend visual effects is the backend implementation.

The model generated a complete logic chain for the booking system:

• After the user selects a date, the system queries the remaining seats for that flight;
• The user selects a seat and fills in passenger information;
• Upon submission, the data is written to persistent storage;
• When bookings for the same flight reach 6, the frontend automatically locks the options.

This chain includes several engineering details, such as:

• Date and flight number association: Flights on different dates require independent counting;
• Seat deduplication: Seats already occupied on the same flight cannot be selected again;
• Data validation: Fields like passenger name and contact information require non-empty validation;
• State feedback: A clear prompt is needed upon successful submission, and understandable error messages should be returned on failure.

Small but fully functional. Although this small-scale booking system cannot compare to large-scale order systems in production environments, it already possesses the fundamental characteristics of a minimum viable product.

5. Model Generalization: Same Structure, Different Domains

If the model merely memorized a single page template, its value would be significantly diminished. We further observed its generalization performance across different themes.

5.1 Deep-Sea Research Institute Website

Using the same prompt structure but changing the theme to a deep-sea exploration research institute, the model generated an entirely different visual and content system.

The hero section uses a vertical descending camera movement: from the sea surface into the water, passing the waterline, the screen splits into two shades of blue, with the warm yellow lights of a distant submersible becoming the visual focal point. The main body of the page contains parameter cards for three submersibles, detailing dimensions like propulsion methods and observation window sizes. Additionally, there are independent display pages for six deep-sea creatures, along with field notes signed by a chief scientist, a marine biologist, and a documentary director.

This result indicates that the model does not simply replace image assets but reconstructs the information architecture and content style.

5.2 Luxury Perfume Brand Website

In the scenario of a luxury perfume brand that emphasizes aesthetic expression, the model demonstrated a high level of completion in typography, color, and whitespace control.

The page features an amber perfume bottle as the core visual element, with the liquid inside rotating slowly and light passing through the glass to create caustic patterns. The brand name uses the Cormorant Garamond serif typeface overlaid on the bottle, giving the overall vibe a resemblance to commercial fashion brand pages.

Such scenarios require a high level of aesthetic consistency from the model: font selection, color matching, animation rhythm, and spatial whitespace all need to be coordinated. From the results, the model’s generation capability in the direction of “extreme aesthetics” shows a certain degree of stability.

6. Practical Advice for Developers

6.1 Prompts Must Cover Both Presentation and Logic Layers

If you want the model’s output to be more than just a static page, the prompt needs to explicitly state backend constraints and interaction rules. For example, “a limit of 6 seats per flight,” “write to persistent storage upon submission,” and “seats cannot be selected twice.” The more specific the requirement description, the lower the subsequent repair costs.

6.2 Treat Generated Results as Initial Drafts, Not Final Deliverables

Although the model showed strong completeness in this task, real-world projects still require engineering effort for code review, security hardening, exception handling, and performance optimization. Especially in scenarios involving user data, payments, and concurrency, the prototype code generated by the model cannot be directly equated to production code.

6.3 Leverage Agent Capabilities Instead of Single-Shot Completions

There is a significant difference between simply asking a model to complete a function and letting it act as an agent to complete a full-stack task. In agent mode, the model can independently plan file structures, generate multi-file code, provide execution instructions, and adjust based on error messages when necessary. This mode is much closer to the role of a “junior engineer” in real-world development collaboration.

7. Conclusion: Signals of Capability Improvement Have Emerged

Based on the results of this task, the usability of current open-source models in complex agent scenarios is improving. They are not just making progress on benchmarks but are beginning to possess a sense of “deliverability”—generated pages have complete visual systems, backends have runnable business logic, and cross-theme generalization is relatively robust.

For developers, this means general-purpose LLM agents can be more deeply integrated into actual workflows such as prototype development, marketing page construction, and data visualization. They can handle a large amount of repetitive implementation work, allowing developers to focus more on architecture design, business boundaries, and quality assurance.

Of course, the model’s generated results still need to be treated with caution. It is better at quickly scaffolding a “good enough system” rather than directly replacing senior engineers for the production-level implementation of critical systems. Understanding this distinction is key to maximizing its value.