Can Open-Source Agents Deliver Full-Stack Results? From Benchmarks to Working Systems

Over the past year, LLM evaluation has been dominated by benchmark scores. When an open-source model posts strong results across difficult benchmarks, it does tell us something about improvements in reasoning and code generation. But for developers in the trenches, scores do not answer a far more practical question: if I plug this model into my development workflow, what can it actually ship?

This article sets benchmarks aside and focuses on a single question: when a model with agentic orchestration capabilities receives a complete product requirement, can it go from brand visuals all the way to a runnable backend? Three real tasks of increasing complexity are used below to map the model’s capability boundaries and engineering usability.

1. Task Design: Why Use Full-Site Generation as a Test

Single-point code generation is no longer difficult. What actually separates capability tiers is stability across long-horizon tasks: requirements understanding, information architecture, visual systems, frontend interaction, graphic assets, and backend data modeling. A break in any link produces an unusable deliverable.

Three task groups were designed with progressively increasing complexity:

• Task A (full-stack): A brand website for a low-Earth-orbit travel company, including cabin selection and flight booking, with persistent booking data.
• Task B (content-driven): A website for a deep-sea exploration research institute, emphasizing narrative structure, spec cards, and multi-author attributed content.
• Task C (aesthetics-focused): A luxury perfume brand website, emphasizing typography, material rendering, and dynamic visuals.

All three tasks shared the same prompt structure: brand positioning, information architecture requirements, interactive feature points, and visual direction. The only differences were domain and delivery focus. This makes it possible to observe whether the model can reliably reuse the same workflow across different contexts.

2. Task A: From Visual System to a Working Booking Backend

2.1 A Cohesive Brand Visual on the First Pass

The hero section uses a full-screen video background viewed through a spacecraft window, with Earth’s curved surface slowly rotating and a blue glow along the atmosphere’s edge. A monospace flight data panel sits in the lower right corner, while a fluorescent cyan dot pulses slowly in the upper left to simulate system status. These visual elements are not individually complex, but they require the model to balance premium commercial spaceflight aesthetics with information density, without drifting into cheap sci-fi.

In the actual output, the color palette, typography hierarchy, and motion timing arrived as a coherent set rather than scattered pieces. This suggests the model maintained an implicit design token system during generation, keeping colors, spacing, and font sizes consistent across sections.

2.2 A Vertical Timeline Narrative

Scrolling is driven by a vertical timeline that breaks the flight into phases:

• T-60:00 Suit up in a custom pressure suit at the spaceport
• T+03:00 Endure G-forces during ascent
• T+08:00 Zero gravity begins
• T+92:00 Splashdown, recovery divers approaching

Each phase is paired with an orange key metric and a scene image. The critical requirement here is the binding between data and narrative. Time codes, phase names, and key figures must correspond precisely without narrative drift. In testing, the model maintained consistency between phases and their data, with no contradictory fields.

2.3 An Interactive Cabin Map and a Real Backend

Cabin selection is rendered in SVG, with available, occupied, and locked seat states distinguished through styling. More importantly, the backend is functional: after selecting a date, choosing a seat, and entering passenger information, the booking data is written to persistent storage. Each flight has a six-seat capacity, and flights lock automatically when full.

This is where it becomes clear whether the output is just a shell. A frontend-only mockup typically returns a fake success message after submission. This deliverable includes a complete data model and constraint logic. The core constraints can be expressed in pseudocode as follows:

def reserve(flight_id, seat_id, passenger):
    with transaction():
        flight = load_flight(flight_id, for_update=True)
        if flight.reserved_count >= flight.capacity:   # capacity limit
            raise CapacityFull(flight_id)
        if seat_id in flight.locked_seats:             # seat mutual exclusion
            raise SeatUnavailable(seat_id)
        persist_reservation(flight_id, seat_id, passenger)
        if flight.reserved_count + 1 >= flight.capacity:
            lock_flight(flight_id)                     # auto-lock when full

Two engineering details are worth noting. Capacity checks must happen inside a transaction; otherwise, concurrent requests can oversell. And full-flight locking should be a state transition rather than data deletion, so that cancellations and audits remain possible. The implementation covers both points, indicating that the model derived the logic from constraints rather than copying a template.

3. Task B: Structured Content and Multi-Role Narrative

Switching to a deep-sea exploration research institute, the same prompt structure produced a completely different information architecture.

The hero shot descends vertically from the ocean surface. At the moment of crossing the waterline, the screen splits in two, with blues shifting from light to dark and two warm yellow submersible lights glowing in the distant deep. This split-waterline effect places real demands on CSS gradients and layer composition. In testing, it was implemented through layered backgrounds and clip paths without introducing heavy rendering dependencies.

Three content-layer features stand out:

1. Spec cards include details down to propulsion method and observation window dimensions, indicating that the model can build field-structured data for each entity rather than offering vague descriptions.
2. Six deep-sea species each occupy one full screen, creating an even narrative rhythm and avoiding density imbalance.
3. Three field notes are attributed to a chief scientist, a marine biologist, and a documentary director respectively, meaning the model understands that different roles within the same institution write from different perspectives, and adjusts tone accordingly.

The third point is especially significant. Multi-role attribution is not simply swapping names; it requires tone, terminology density, and focus to shift with the role. This moves beyond text generation and into simulating division of labor within an organization.

4. Task C: A Stress Test for Aesthetic Consistency

The luxury perfume task places all its weight on visual execution: an amber perfume bottle resting on a marble surface, liquid rotating slowly inside, light passing through the glass and casting caustic patterns. The brand name is overlaid on the bottle in a serif typeface.

Caustics are notoriously difficult to render, usually requiring light simulation. In a pure frontend environment, the viable path is approximation through layered semi-transparent gradients and blend modes rather than true ray tracing. The output adopted an approximation approach, with acceptable visual results and preserved page performance. This reflects an important engineering judgment: choose a deliverable approximation within constraints rather than chasing an unattainable optimum.

In typography, the overlap between the serif type and the bottle, along with letter spacing and line height adjustments, all point to a single aesthetic standard. Comparing the three tasks side by side, the model swaps its design language wholesale when switching domains, rather than patching things locally.

5. An Engineering View of Capability

Looking at the three tasks together, several conclusions matter for real development workflows.

First, long-horizon planning is usable. From information architecture to visual systems to backend constraints, the model maintained cross-stage consistency. This determines whether it can be placed into a requirements-to-prototype workflow rather than being limited to code completion.

Second, deliverables are runnable. The booking system’s capacity constraints, seat mutual exclusion, and full-flight locking are real logic, not placeholders. The output can serve as a working prototype that moves directly into integration, significantly reducing early-stage build cost.

Third, asset generation and code generation are connected. The SVG seat map, scene images, gradient backgrounds, and component code were produced in a single flow, reducing the translation loss between design and implementation.

Fourth, the boundaries are equally clear. Video backgrounds, complex lighting, and high-fidelity materials still rely on approximation. Generated results require human review of data models and concurrency safety. Brand-level visuals still need a designer’s direction. Positioning the model as a high-throughput prototype engine is more accurate than positioning it as a replacement for developers.

6. Practical Adoption

If you plan to introduce this kind of agent capability into your team, the following steps are recommended:

• Replace open-ended descriptions with constraint-driven prompt structures. Explicit information architecture, feature points, and data constraints are more effective than asking for something that looks good.
• Write backend constraints into the requirements. Rules like capacity limits, concurrency exclusion, and state transitions should be stated explicitly; otherwise the model may omit them.
• Build a deliverable review checklist. Focus on transaction boundaries, state machine completeness, accessibility, and performance budgets.
• Start from the prototyping stage. Use it first for requirements validation, solution demos, and integration scaffolding before gradually expanding into production code.

Benchmark scores describe a model’s upper bound. Full-site delivery describes its lower bound, and the lower bound determines whether it can truly enter engineering workflows. Based on this test, open-source agents already offer practical value at the level of single-pass generation with full-stack usability. What remains is to manage that capability with disciplined engineering constraints.