Frontend Agent Benchmark: Speed, Retries, and Stability in Complex Tasks
This article presents a controlled comparison of four models on a complex frontend agent task: building a creative studio collaboration system from scratch. Using the same orchestration framework, task budget, browser automation, and acceptance rules, it examines completion time, action rounds and tool calls, failure recovery, and delivery quality. The results show that agent efficiency depends on model capability, execution framework, toolchain, and retry and context management together, so model selection should be evaluated across the full task chain rather than by response speed alone.
1. Why Compare Models Within the Same Execution Framework
In complex frontend development tasks, the model itself is only one variable affecting the result. Task decomposition, tool-calling protocols, browser environment, context injection strategy, retry mechanisms, and final acceptance rules can also significantly change execution efficiency.
If both the model and the execution framework are changed at the same time, the final result often cannot show which factor caused the difference. Therefore, this experiment used a fixed execution environment and replaced only the model for side-by-side observation:
- The same agent orchestration framework was fixed.
- Task descriptions, initial materials, and seed data were fixed.
- Context capacity and task budget were fixed.
- The browser automation environment was fixed.
- Desktop and mobile inspection sizes were fixed.
- Retention requirements for screenshots, code, logs, review records, and metrics were fixed.
- Only one formal repair round was allowed per execution.
This design is closer to a model selection problem in engineering practice: once the application framework, toolchain, and delivery requirements are determined, how much do different models actually differ in performance?
2. Test Task: Building a Creative Collaboration System from Scratch
The test subject was not a simple static landing page, but a complex product interface called the Creative Studio Collaboration System. The system provided only a small amount of open image, project member, asset, version, and comment data. The model could not directly copy an existing page; it had to design the product structure, interaction flow, and visual style on its own.
The task included five core states:
- Project overview: display project progress, members, current stage, and key deliverables.
- Inspiration board: support centralized browsing, arrangement, and organization of image assets.
- Asset comments: allow users to leave comments on specific assets or local positions.
- Version approval: support viewing historical versions and performing approve or return actions.
- Release preview: simulate the inspection and preview process before final delivery.
The system also included three roles: creative lead, designer, and client. Different roles did not see exactly the same scope of operations. The page needed to reflect permission differences visually while ensuring that the main flows could be completed smoothly.
From an interaction perspective, the system needed to cover at least the following behaviors:
- Dragging and rearranging assets.
- Linking comments to asset positions.
- Switching between versions.
- Approval, return, and status changes.
- Preserving key state after page refresh.
- Responsive layout for desktop and mobile.
- Sharing project data across multiple pages.
Therefore, this task tested the model’s product understanding, information architecture, visual design, frontend implementation, state management, and automated validation capabilities at the same time.
3. Common Experiment Conditions and Evaluation Method
All four models used the same context capacity, task budget, and browser automation tools. Checks were divided into desktop and mobile parts, with the former using a large-screen size and the latter using a common phone size.
Evaluation did not rely only on final screenshots. It also covered four dimensions.
3.1 Completion Time
This is the time from task start to first meeting the acceptance criteria, reflecting the model’s basic execution efficiency. However, comparing duration alone is not sufficient. If a model fails and does not retry, its time may look short, but it has not produced a valid delivery.
3.2 Action Rounds and Tool Calls
Complex tasks usually go through multiple stages such as requirements analysis, file reading, page generation, service startup, browser inspection, screenshot comparison, and issue repair. More action rounds do not necessarily mean weaker capability, but excessive invalid calls usually indicate insufficient planning or low efficiency in using feedback.
3.3 Failure Recovery Capability
Whether a model can understand error messages, locate problems, and continue completing the task is an important indicator that distinguishes an agent system from ordinary code generation. Key records included:
- Whether it could identify build errors.
- Whether it could distinguish code problems from environment problems.
- Whether it repeated the same erroneous action.
- Whether it damaged existing work during retries.
- Whether it could complete a degraded handling within a limited budget.
3.4 Delivery Quality
Final delivery quality includes visual consistency, interaction completeness, responsive behavior, data persistence, role permissions, and engineering maintainability. Even if a page looks good in a screenshot, the task cannot be considered truly complete if state is lost after refresh or the mobile layout is unusable.
4. Execution Performance of the Four Models
In this experiment, all four models eventually completed the task, but the process differences were very clear.
The first model completed in about 17 minutes and had the shortest overall path. It understood the task goals relatively quickly, built the page skeleton first, then added key interactions, and finally performed targeted checks through the browser. Its implementation style was lightweight, with a clear page structure, and project state, assets, and member relationships were easy to understand. Its advantage was a stable rhythm and fewer redundant operations, making it suitable for scenarios with high requirements for response speed and basic delivery efficiency.
The second model failed on its first run and, after a rerun, took about 1 hour 12 minutes in cumulative time. It performed strongly in visual design, using an editorial style combining dark and warm tones, and created a strong brand feel through serif headings, approval status, and permission information. Mobile page consistency was also relatively good. However, the first run did not complete smoothly, showing that visual understanding and product expression do not necessarily equal engineering execution stability. For this type of model, teams need to focus on improving error detection, build validation, and post-failure recovery strategies.
The third model did not fully restart the task. Instead, it performed three internal retries during execution and ultimately took about 88 minutes. The system it built had the highest information density, with navigation, project progress, roles, deliverables, approvals, and the asset canvas organized into a relatively complete workbench, showing high engineering completeness. The problem was that the complex information structure also increased page implementation and validation costs. The model had to handle more states and components, making it prone to local interaction inconsistencies, missed layout details, or duplicate checks.
The fourth model failed twice consecutively under the default retry policy, then completed the task by extending the backoff time, taking about 1 hour 4 minutes cumulatively from first start to success. It ultimately formed a restrained archival system style, using dark headings, a light background, thin-line tables, and work thumbnails to present the creative workbench. This case shows that model failure is not necessarily equivalent to insufficient capability; it may also be related to service fluctuation, request timeout, context congestion, or an unreasonable retry policy.
5. What the Results Show
5.1 Model Speed Is Not the Same as Agent Efficiency
Model generation speed is only a local metric. What truly affects delivery efficiency is:
Effective completion time = generation time + tool execution time + error repair time + retry wait time
If a model responds quickly every time but frequently calls invalid tools, or needs multiple repair rounds to pass acceptance, the total time may still be long. Conversely, a model that thinks slightly longer per turn but plans completely may finish the entire task faster.
5.2 Retry Mechanisms Are Part of the Execution System
Retry should not simply be designed as “send the same request again after failure.” A more reasonable strategy should take different actions based on the error type:
- For network timeouts, use exponential backoff.
- For service rate limiting, reduce request frequency.
- For build failures, read the logs before modifying code.
- For page interaction failures, relocate elements and check state.
- For overly long context, compress historical information and retain key constraints.
Without error classification, blind retries only repeat failures, waste budget, and increase the risk of state contamination.
5.3 Context Management Determines the Upper Limit of Complex Tasks
The creative collaboration system includes multiple pages, roles, states, and interactions. If the model carries the full history in every round, the context will expand quickly and key information will be buried.
In engineering practice, context can be divided into four layers:
- Immutable constraints: task goals, acceptance rules, and size requirements.
- Current state: completed pages, pending issues, and run results.
- Historical summary: attempted approaches and their failure reasons.
- Immediate feedback: latest logs, screenshots, and browser check results.
This reduces invalid information and helps the model quickly resume work during retries.
5.4 Visual Quality and Interaction Quality Need Separate Acceptance
Final screenshots can easily hide functional problems. It is advisable to split acceptance into two chains:
Visual acceptance focuses on hierarchy, spacing, color, typography, responsive layout, and information density. Functional acceptance focuses on dragging, comment positioning, version switching, approval status, refresh persistence, and role permissions.
Only when both chains pass is delivery complete. For automated systems, page screenshots, operation logs, run metrics, and review results should also be retained for later review.
6. Evaluation Metrics Suitable for Enterprise Practice
If a team needs to evaluate models over the long term, it is not advisable to record only “completed or not” and “time spent.” The following metrics can be established:
- First-pass success rate.
- Average effective completion time.
- Number of tool calls per task.
- Rate of invalid retries.
- Number of repairs after first generation.
- Desktop and mobile pass rate.
- State persistence pass rate.
- Key interaction coverage.
- Size of code changes.
- Post-failure recovery success rate.
At the same time, failure causes should be recorded, not just failure results. Only by dividing failures into model understanding errors, tool-call errors, environment exceptions, context problems, and acceptance omissions can evaluation data have improvement value.
7. Conclusion
The most important conclusion of this comparative experiment is not that one model is absolutely ahead, but that the results of complex agent tasks are jointly determined by model capability, execution framework, toolchain, and recovery mechanisms.
Fast models are suitable for tasks with clear structure and timely feedback. Models with strong visual expression are suitable for scenarios that emphasize interface style and product presentation. Engineering-oriented models are better for information-dense workbenches with complex states. In the face of service fluctuations or long-flow tasks, stable retries and context management are often more important than single-generation quality.
Therefore, in actual model selection, it is recommended to evaluate the complete task chain rather than compare only model response speed or single-round code quality. Only by including planning, execution, validation, repair, and delivery in the test can a team truly judge whether a model is suitable for a production-grade agent system.
