Complex Frontend Agent Benchmark: Balancing Speed, Retries, and Reliability
A controlled comparison of frontend AI agents shows why response speed alone is a poor measure of delivery efficiency. By holding the agent framework, task, assets, browser, and acceptance criteria constant while changing only the underlying model, this study examines requirement understanding, implementation, state management, browser automation, and failure recovery. The results highlight the importance of first-pass success rate, useful action ratio, retry cost, and final delivery quality alongside execution time.
In complex frontend tasks, a model’s response speed does not necessarily translate into faster delivery. This article presents a controlled benchmarking method that keeps the agent framework, task description, assets, browser, and acceptance criteria fixed while changing only the underlying model. It compares how different models handle requirement analysis, page implementation, state management, automated operations, and failure recovery.
The fastest model completed the task in approximately 17 minutes. Other models took close to or more than an hour because of initial failures, automatic retries, or backoff strategies. For development teams, model evaluation should therefore include first-pass success rate, useful action ratio, retry cost, and final delivery quality—not just response latency.
Why Model Response Speed Is Not Enough
In a conventional question-and-answer setting, the time required to generate a response is often a useful indicator of user experience. In an agent-based development workflow, however, the model must continuously perform a series of actions:
- Read project files
- Analyze requirements
- Write code
- Start the application
- Operate the browser
- Inspect rendered results
- Fix errors
- Revalidate functionality
As a result, total delivery time is not equal to the model’s response time for a single step. It can be represented as:
Total delivery time = planning time + coding time + tool-call time + page validation time + failure recovery time + retry wait time
Failure recovery and retry wait time are frequently underestimated. A model may respond quickly at each individual step, but if it repeatedly generates invalid actions, calls the wrong tools, or modifies the same file without making progress, its overall efficiency may be worse than that of a slower model with a higher first-pass success rate.
For enterprise development, the following metrics are more meaningful:
- First-pass success rate: Whether the first complete execution passes the core acceptance criteria.
- Useful action ratio: The proportion of model actions that actually advance the task.
- Tool-call accuracy: Whether the model selects and uses browser, filesystem, and testing tools correctly.
- Repair convergence speed: How quickly the model identifies and fixes a problem after it appears.
- Context utilization: Whether the model consistently retains task constraints, page state, and previous decisions.
- Stability: Whether repeated executions of the same task produce consistent results.
Controlled Experiment Design
The benchmark used a fixed agent execution framework and changed only the underlying model. The task was a complex frontend prototype called Creative Studio Collaboration System. It emphasized information architecture, interaction state, and multi-role collaboration rather than simple static page generation.
The task input included six openly licensed city museum images, project member information, asset data, version data, and seeded comments. No existing page was provided for direct imitation. Each model had to determine the page structure, visual style, and interaction logic independently.
The system included at least five core areas:
- Project overview: Displays the project name, current progress, members, and delivery status.
- Inspiration asset wall: Supports asset arrangement, filtering, and viewing.
- Asset comments: Associates comments with specific assets or page locations.
- Version approval: Supports version switching, approval, rejection, and status display.
- Release preview: Presents the final result in a form close to a real delivery preview.
The application also defined three roles: creative director, designer, and client. Each role had different permissions. For example, some members could submit versions, while others could only comment. Approval actions had to respect role constraints.
After a page refresh, the project state, comments, selected version, and approval results still had to remain available. This prevented the system from being merely a visual demo that appeared complete but was not functionally usable.
To reduce environmental variation, all tests used the same context size, task budget, browser automation capabilities, and acceptance process. Desktop checks used a viewport of 1440 × 1000, while mobile checks used 390 × 844. Each run preserved the initial screenshot, final screenshot, source code, runtime logs, review records, and metric data. Only one consolidated repair cycle was allowed.
Execution Results Across Four Model Groups
All four test groups completed the task, but their execution paths differed substantially.
Model Group 1: Fastest Completion
The first model group completed the task in approximately 17 minutes with a relatively short action chain. It quickly established the task boundaries, built the page skeleton, and then added state logic and responsive layout.
The final interface used a light editorial-studio style. Project status, assets, and member relationships were relatively clear. Although the visual design was not especially elaborate, the structure was lightweight, allowing users to find the primary functions quickly.
Model Group 2: High Design Quality, Unstable First Run
The second model group failed during its initial execution. After a rerun, its cumulative execution time reached approximately 1 hour and 12 minutes.
It invested heavily in visual design, combining dark and off-white tones with editorial styling. Serif headings, a permission matrix, and approval states created a consistent visual language. Its strengths included a strong brand identity and relatively complete mobile layouts.
However, the model did not validate key interactions early enough during the first run. Errors accumulated and only became visible later. This demonstrates that strong design quality does not necessarily indicate a stable execution process.
Model Group 3: Multiple Internal Retries
The third model group did not perform a complete rerun. Instead, it triggered three internal automatic retries during a single execution, bringing the cumulative time to approximately 88 minutes.
It selected a dark professional-workbench style and organized navigation, project progress, roles, delivery, approvals, and the asset canvas into a unified system. The result had high information density and a relatively complete engineering structure.
The trade-off was a heavier page architecture. The model repeatedly adjusted multiple modules, which significantly increased the number of actions. Automatic retries prevented a full-run failure, but they also extended execution time considerably.
Model Group 4: Repeated Failures and Extended Backoff
Under the default retry strategy, the fourth model group failed twice in succession. It then expanded its backoff strategy and eventually completed the task, with a cumulative first-success time of approximately 1 hour and 4 minutes.
Its visual direction resembled an art museum archive system. Black serif headings, an off-white background, fine-line tables, and artwork thumbnails formed the primary visual system.
This approach was particularly strong in content presentation and archival character. However, tool-call failures early in the process created a substantial waiting cost.
These timings describe observations under a specific task, environment, and execution strategy. They should not be interpreted as a general ranking of model performance. A more reliable approach is to treat them as engineering evaluation samples and combine repeated experiments with failure logs and final acceptance results.
Action Strategy Creates the Real Performance Gap
Complex agent tasks are rarely completed by generating all code in a single step. Instead, they consist of many small actions. Throughout the process, the model must repeatedly answer three questions:
- What is the most important unfinished item?
- Should it be addressed through a code change, browser inspection, or data validation?
- How can the model confirm that the result satisfies the requirement?
Efficient models often build a minimum runnable version first and then add functionality incrementally. They do not attempt to optimize visual details, complex interactions, and complete data structures simultaneously from the beginning.
This approach makes it possible to detect basic issues—such as environment configuration, routing, and state persistence—at an early stage.
By contrast, if a model implements extensive animation, filtering, drag-and-drop behavior, and permission logic before confirming that the page can run, later repair becomes much more difficult. The model may continue adding code on top of a flawed foundation, creating a cycle of local patches, repeated failures, and additional retries.
The agent workflow can therefore be divided into four stages:
- Requirement modeling: Extract roles, pages, states, data, and acceptance criteria.
- Skeleton implementation: Build routing, basic components, and the minimum data flow first.
- Interaction completion: Add comments, version switching, approvals, drag-and-drop behavior, and other interactions.
- Automated acceptance testing: Validate visual presentation, interactions, and refresh persistence on both desktop and mobile devices.
Designing a More Reliable Agent Workflow
Development teams can improve their execution frameworks in several ways.
Structure the Task into Layers
Do not combine every requirement into one long block of text. Separate functional requirements, visual requirements, data requirements, device sizes, and acceptance conditions.
A structured task helps the model create a more stable execution plan.
Add Milestone Checkpoints
Run page-start and basic interaction checks after each core module is completed. For example, after implementing the project overview, immediately verify routing and data display. After implementing comments, refresh the page and confirm that the data remains available.
Limit Ineffective Retries
More retries do not necessarily produce better results. If the same error occurs repeatedly, the agent should pause its current strategy and reassess the cause instead of sending the same tool call again.
A reasonable retry mechanism should distinguish between:
- Temporary failures
- Invalid parameters
- Logical errors
Preserve Structured Logs
Logs should record at least the following information:
- Action timestamp
- Tool name
- Input parameters
- Return value
- Error type
- Retry count
- Final status
Only with this information can a team determine whether time was spent on model reasoning, environment response, or error recovery.
Evaluate Quality and Cost Together
Final quality and execution cost should both contribute to the score. One possible composite metric is:
Overall efficiency = functional pass rate × visual pass rate × state reliability ÷ total execution cost
Total execution cost should include more than runtime. It may also include tool-call count, retry count, manual repair count, and resource consumption.
Conclusion
The most important conclusion from this comparison is not which model completed one task the fastest. The broader lesson is that complex agent efficiency depends on multiple stages working together.
Parameter scale, single-response latency, and final screenshot quality explain only part of the outcome. The delivery experience is determined more directly by whether the model can decompose the task correctly, use tools appropriately, validate results in time, and converge quickly after failure.
In real projects, teams should avoid running only one comparison and judging only the final page. A more valuable evaluation method is to fix the task and environment, execute multiple rounds, and record first-pass success rate, average action count, tool-call accuracy, retry cost, mobile adaptation, and state persistence.
For frontend agents, the ability to generate a page is only an entry-level capability. The more practical production standard is the ability to complete complex tasks reliably and deliver usable results at a predictable cost.
