Beyond Benchmark Scores: Evaluating General Agents on Full-Stack Engineering Delivery

Moving Beyond Benchmark Scores and Back to Engineering Reality

When evaluating新一代general-purpose AI agents, the industry often falls into a familiar pattern: check the benchmark results, review the leaderboard, then conclude whether the model is “strong” or not. However, benchmark scores only reflect statistical performance on a fixed set of tasks and do not answer a more critical question:

If a developer hands the agent a real-world requirement, can it independently complete the full chain—from understanding, design, and implementation to final delivery?

To address this, this article focuses on a reproducible engineering scenario: asking an agent to independently build a complete website that includes front-end visuals, interaction logic, backend APIs, and data persistence. Although this scenario appears common, it places high demands on the agent’s multidimensional capabilities, including:

• Structured decomposition of complex requirements;
• Understanding of visual language and brand tone;
• Management of front-end interaction states;
• Design of backend business rules;
• Assurance of code consistency, runnability, and maintainability.

Only when an agent satisfies all these conditions simultaneously can it evolve from “writing code” to “delivering engineering.”

Task Design: Why a “Near-Earth Orbit Travel Website”

To keep the evaluation close to real development, the task avoids low-complexity projects such as personal blogs or static landing pages. Instead, it is set as a fictional commercial space tourism brand that requires the following core capabilities:

• Space capsule seat selection;
• Flight date selection;
• Passenger information entry;
• Seat capacity limits and locking;
• Persistent storage of booking data.

This setup offers several distinct advantages.

First, the visual requirements are high. Commercial space travel carries a premium, high-tech aesthetic, making it an ideal test of the agent’s ability to handle color, typography, motion effects, and spatial storytelling.

Second, the business rules are explicit. Each flight has only six seats, and once full, the flight is automatically locked. This constraint verifies whether the agent can translate natural language requirements into executable business logic.

Third, front-end and back-end must be integrated. If the agent only generates static pages, the booking feature cannot actually function; if it only generates backend APIs, users cannot perceive the complete flow. Only by fully integrating the full stack can delivery capability be proven.

Four Key Layers of Full-Stack Delivery

1. Visual System and Brand Narrative

The agent’s generated hero section goes beyond simply arranging information on a page—it constructs a visual system with complete narrative logic.

The full-screen video background uses a spacecraft window perspective, with the Earth’s curvature slowly rotating and the atmosphere’s edge glowing in blue. The flight data panel on the right uses monospaced fonts to display simulated parameters such as altitude, velocity, and cabin pressure. The status indicator in the upper-left corner pulses slowly in fluorescent cyan, conveying a “system online” state.

More deeply, the entire site is organized along a vertical timeline, from T-minus 60 minutes to splashdown recovery, divided into multiple phases. Each phase includes not only descriptive text but also a key data point and a scene image. This approach transforms the website from a mere pile of features into a complete brand experience.

2. Front-End Components and Interaction States

In the seat selection module, the agent must handle complex front-end states:

• Visual distinction between selected and available seats;
• Definitions for selected, disabled, and hover states;
• State preservation across a multi-step booking flow;
• Loading and error handling during back-end synchronization.

Looking at the implementation, the seat SVG diagram is not a static decoration but a clickable interactive component. Users can select or deselect seats, and the system provides real-time feedback for unavailable seats. This demonstrates that the agent understands not only visual design but also state management and event binding.

3. Backend Logic and Business Constraints

The backend of the booking system is equally noteworthy. The agent translates the natural language constraint “six seats per flight, auto-lock when full” into backend validation rules, rather than merely disabling clicks on the front end.

This means:

• The backend API queries the current number of booked seats for a given flight;
• When remaining seats reach zero, the API returns a “not bookable” status;
• The front end updates the interface based on the API response;
• Concurrent booking requests cannot exceed the seat limit.

This layer of logic is often a weak point in low-quality implementations. Many implementations hard-code restrictions only on the front end, allowing overbooking once the interface is bypassed and direct API requests are made. In this task, the agent correctly places the business rules on the backend, demonstrating an understanding of server-side consistency.

4. Data Persistence

Booking data must be genuinely saved rather than disappearing on page refresh. The agent selects an appropriate persistence mechanism to store passenger information, flight dates, and seat numbers in an associated manner. On subsequent visits, previously booked seats remain locked.

From an engineering perspective, this design satisfies at least two requirements:

• The data structure supports querying by flight and deduplication;
• Read and write paths remain consistent across multiple requests.

Although the project scale is small, it covers the core data flow of a typical web application.

Transfer Performance Across Different Domains

To further validate the model’s generalization capability, the evaluators used the same prompt structure to ask the agent to generate websites on completely different topics: a deep-sea exploration research institute and a premium perfume brand.

In the deep-sea theme, the agent produced a narrative with a vertical camera descent from the ocean surface, splitting the frame at the horizon with colors transitioning from shallow blue to deep blue. Parameter cards for three submersibles included precise details such as propulsion methods and observation window dimensions, while the field notes section was written from the perspectives of a chief scientist, a marine biologist, and a documentary director.

In the perfume theme, the visual focus shifted to an amber bottle with light refraction from the rotating liquid, with serif typography overlaid on the bottle, conveying a quiet yet intense sense of luxury.

This cross-domain transfer demonstrates that the agent is not copying a template but instead understanding the visual metaphors, content structures, and audience expectations of different industries before reorganizing its output. This is a critical distinction between “using tools” and “understanding tasks.”

Practical Implications for Developers

From a practical standpoint, the agent’s performance in complex engineering tasks offers several actionable insights for developers.

First, treat natural language as the entry point for requirements.
In traditional development workflows, requirements pass through multiple layers of translation—product, design, front end, and back end. The agent’s involvement can compress parts of this pipeline, but only if the requirement description is sufficiently clear. It is recommended to specify functional boundaries, business rules, and acceptance criteria directly in the prompt.

Second, focus on the runnability of generated output, not just the visual effect.
Static pages are easily overestimated. What truly matters is an engineering artifact that can be launched, accessed, integrated, and persisted. When evaluating agent delivery, check backend validation, error handling, and state consistency.

Third, push business rules to the server side.
Front-end restrictions are only an experience layer; server-side validation is the true security boundary. The agent correctly applied this principle in this task, and developers should maintain this engineering habit when reusing generated code.

Fourth, cross-domain transfer is a strong capability signal.
If a model only performs well on a single topic, it may simply reflect training data coverage. Only when the same prompt structure can consistently produce high-quality output across completely different scenarios—aerospace, deep sea, luxury goods—does it demonstrate that the capability is transferable.

Boundaries and Risks

It is important to emphasize that this type of capability does not mean the agent can replace a full development team. In real production environments, the following challenges remain:

• Architectural design and module decomposition for complex systems still require human decisions;
• Generated code requires security and maintainability reviews;
• Integration with existing systems depends on organizational knowledge;
• Requirements for performance, permissions, auditing, and compliance far exceed what a demonstration project covers.

Therefore, a more pragmatic positioning is to use the agent as an efficient tool for requirement validation, prototype construction, and initial implementation—not as a replacement for delivering production-grade systems.

Conclusion

The degree of completion in complex engineering tasks is a key benchmark for whether a general-purpose agent possesses true “engineering delivery capability.” A complete full-stack generation must simultaneously satisfy four layers: visual expression, front-end interaction, backend business logic, and data persistence. When an agent can stably complete such tasks and transfer its capabilities across different domains, it signals a shift from “answering questions” to “delivering results.”

For developers, the focus should likewise move from leaderboard numbers to runnable, verifiable, and reusable engineering output. After all, what truly determines value is not how many points the model scores on an exam, but whether it can take a real requirement and carry it through to completion.