Intelligent Agent Benchmarking for Complex Creative Systems: Efficiency, Reliability, and Delivery Quality

    This benchmark compares four large general-purpose models on the same browser-based development workflow for building a stateful creative collaboration system. It examines not only final visual quality, but also execution time, tool-use stability, action count, failure recovery, responsive behavior, state persistence, and overall engineering completeness. The results show why agent evaluation should measure the full delivery process rather than a single screenshot or first-response speed.

    1. Why Agent Delivery Capability Matters More Than Model Capability Alone

    For complex front-end tasks, generating a page that looks complete is only part of the evaluation. Development efficiency depends on whether an agent can continuously understand requirements, call tools correctly, detect errors, and complete repairs within a limited budget.

    This benchmark used the same agent workflow for all tests and changed only the underlying model. The task, asset data, context capacity, browser environment, acceptance criteria, and repair limits remained consistent. The goal was to reduce variable interference and observe how different models perform in a realistic development process.

    All four models were general-purpose models with relatively large parameter scales. They were tested with:

    • Approximately 262K tokens of context capacity
    • A task budget of 200K
    • Browser automation tools
    • Desktop and mobile viewport checks
    • Archived first-version output, final screenshots, execution logs, source code, and review records
    • A maximum of one explicit repair cycle

    All four models eventually completed the task, but their execution paths were substantially different:

    • Model A finished in approximately 17 minutes.
    • Model B failed during its first execution and took approximately 72 minutes after rerunning the task.
    • Model C triggered multiple internal automatic retries during the same task and finished in approximately 88 minutes.
    • Model D failed twice, then expanded its backoff strategy and achieved its first successful completion after approximately 64 minutes.

    These results show that comparing model price, response speed, or final screenshots alone does not fully represent an AI agent’s engineering value. In practical development, the number of actions, failure-recovery capability, tool-calling stability, and context-management efficiency are equally important.

    2. The Benchmark Task: Building a Complete Creative Collaboration System

    The task was not to create a static showcase page. It required a functional creative-studio collaboration system with a business workflow.

    The system started with only a small amount of data, including several open images, project members, assets, version records, and seeded comments. The models could not simply reproduce an existing page. They had to design the information architecture, visual direction, and interaction logic themselves.

    The system needed to support at least five major states.

    1. Project Overview

    The overview needed to display basic project information, current progress, participating members, asset counts, version status, and recent activity. Users should be able to quickly determine whether the project was in the concept, production, review, or publishing stage.

    2. Inspiration Board and Asset Organization

    Users needed to browse image assets, drag them onto an inspiration board, and organize them by theme or visual direction. This tested both layout design and the stability of drag-and-drop interactions across different screen sizes.

    3. Asset-Anchored Comments

    Comments could not remain ordinary text entries. They needed to be associated with a specific asset or position on the canvas. Users had to be able to identify the source of a comment, reply to the discussion, and see the existing comment state after refreshing the page.

    4. Version Approval

    The system needed to support version switching, version comparison, approval, and rejection. Permissions had to vary by role. For example, a creative lead could approve a version, a designer could submit it, and a client could view or provide feedback.

    5. Publishing Preview

    After a version was approved, the system needed to provide a preview state for pre-publication checks. Users had to be able to distinguish between draft, pending review, approved, and published content.

    The system also included three roles: creative lead, designer, and client. The differences could not be limited to role names. They had to affect available actions, visible information, and workflow paths.

    After a page refresh, the project state, comments, current version, and approval result also needed to remain available.

    3. Standardized Test Environment and Acceptance Criteria

    To make the comparison repeatable, all tests used the same execution framework and browser automation environment. The desktop viewport was 1440 × 1000, and the mobile viewport was 390 × 844.

    The acceptance criteria covered five main areas.

    1. Functional Correctness

    The test verified whether navigation, asset loading, drag-and-drop, comments, version switching, approval, and preview actions could be completed.

    For a stateful application, it was not enough to confirm that a button existed. The test also needed to verify that the underlying data changed correctly before and after each operation.

    2. State Persistence

    Page refreshes were an important part of the benchmark. The system needed to preserve at least the following:

    • Current project
    • User role
    • Selected version
    • Comment content and its associated target
    • Approval result
    • Asset positions on the inspiration board
    • Current page or workflow stage

    If a refresh returned the page to its initial state, the delivery could not be considered complete, even if the visual design was strong.

    3. Responsive Adaptation

    Desktop and mobile layouts require different design logic. Desktop screens are suitable for displaying information across multiple columns, while mobile layouts must prioritize access to core actions.

    The test focused on:

    • Whether navigation collapsed appropriately
    • Whether tables created uncontrolled horizontal overflow
    • Whether images and canvases were clipped
    • Whether action buttons were easy to tap
    • Whether comment and approval workflows remained coherent

    4. Visual Design and Information Hierarchy

    Visual evaluation covered more than colors and typography. It also examined whether the page provided a clear content hierarchy.

    A complex system needs whitespace, cards, labels, status indicators, and section structures to help users understand the current context quickly.

    5. Engineering Delivery Completeness

    The final output had to preserve the source code, screenshots, logs, review information, and metric records. This made it possible to trace what the agent did, where it failed, how many times it retried, and whether the final repair actually solved the problem.

    4. Differences in Execution Across the Four Models

    1. Model A: Short Execution Path and Fastest Overall Convergence

    Model A completed the task in approximately 17 minutes, making it the fastest model in the benchmark.

    Its primary strength was establishing a clear information architecture before implementing the core states. It avoided repeatedly revisiting visual details too early in the process.

    The final page used a light editorial-style interface. Project status, asset relationships, and member information were easy to read. The layout was relatively restrained and did not overload the interface with decorative elements, allowing users to quickly locate the current project state and the next action.

    This approach benefited from a short action chain and clearly targeted tool calls. It rarely entered a cycle of making a change and immediately discarding it.

    For an AI agent, effective prior planning can significantly reduce the number of browser operations and code modifications.

    2. Model B: Strong Visual Design but High First-Failure Cost

    Model B did not complete its first execution successfully. After rerunning the task, its total time was approximately 72 minutes.

    The final interface combined dark tones, gold, and off-white to create a strong branded visual direction. Its title treatment, approval states, and permission matrix produced a distinctive design language.

    The model performed particularly well in maintaining consistency on mobile devices, suggesting a solid understanding of layout transformation across screen sizes.

    However, the initial failure indicates that the model may not have validated critical interactions early enough, or may have proceeded to later operations before the required state dependencies had been established.

    This demonstrates that a strong final result does not necessarily mean high delivery efficiency. In environments with strict time limits, the cost of a failed rerun can offset the benefits of better visual quality.

    3. Model C: High Engineering Completeness but Excessive Automatic Retries

    Model C did not rerun the entire task. Instead, it triggered multiple internal automatic retries during a single execution and completed the task in approximately 88 minutes.

    It built a dark professional workspace that integrated navigation, project progress, role permissions, delivery status, approval workflows, and the asset canvas into one complete system.

    The result had high information density and strong engineering completeness, making it suitable for scenarios in which users need to view several business dimensions at the same time.

    However, the large number of automatic retries exposed a potential weakness. When an agent lacks a clear failure-classification mechanism, repeated attempts may simply repeat the same operation instead of changing the problem-solving strategy.

    A retry mechanism should therefore do more than limit the number of attempts. It should identify the type of failure. For example:

    • If an element does not exist, the agent should reevaluate the page structure and locate it again.
    • If state has not changed, the agent should inspect the data-writing logic.
    • If a browser action times out, the agent should reduce the complexity of the individual operation.
    • If the visual result is incorrect, the agent should enter screenshot review rather than repeat the same code execution.

    4. Model D: Restrained Style with Recovery Strategy as the Main Cost Driver

    Model D used the visual language of an art museum archive. Black serif headings, an off-white background, fine-line tables, and artwork thumbnails created a consistent and restrained interface.

    The page had a unified atmosphere and carefully organized content.

    However, its default retry strategy failed twice before the model completed the task by increasing the backoff interval. The total time to first success was approximately 64 minutes.

    This process shows how tool stability and waiting strategies can significantly affect agent performance.

    In browser automation, waiting too briefly can cause an agent to misinterpret a page as failed. Waiting too long can slow the entire workflow. A more appropriate strategy is to use staged waits based on page behavior:

    1. Wait for the required structure to appear.
    2. Wait for data loading to complete.
    3. Wait for the interaction state to stabilize.
    4. Execute the screenshot or assertion only after the previous stages are complete.

    5. Four Core Factors That Affect Agent Efficiency

    1. Number of Actions

    Every code modification, page visit, or screenshot inspection increases the probability of failure and consumes context.

    High-quality agents generally plan the data structure and page states first, then implement related changes in batches.

    2. Tool-Calling Quality

    Browser automation is not simply a matter of clicking buttons. An agent must understand page structure, determine whether an element is interactive, and confirm that the operation actually took effect.

    Without precondition checks, tool calls can result in ineffective clicks, obstructed elements, or unsynchronized state.

    3. Failure-Recovery Capability

    Whether an agent restarts, performs a local repair, or changes its execution strategy after a failure reflects its engineering maturity.

    A localized failure should preferably receive a localized repair. A small error should not force the entire task to restart.

    4. Context Management

    Complex tasks include requirements, source code, screenshots, error messages, and operation history. If an agent cannot compress irrelevant information in time, later decisions may continue to be influenced by an incorrect early path.

    A better strategy is to retain the current state, critical constraints, and incomplete tasks while removing intermediate steps that have already been verified.

    6. Building a More Reliable Evaluation System

    Evaluation should be divided into four metric groups instead of recording only the final completion time.

    Outcome metrics include functional pass rate, visual completeness, mobile adaptation quality, and state-persistence success rate.

    Process metrics include total execution time, number of tool calls, number of code modifications, number of screenshots, retry count, and failure-recovery time.

    Stability metrics include first-attempt success rate, local-repair success rate, browser-operation timeout rate, and the rate of state loss after refresh.

    Cost metrics include context consumption, task-budget usage, and the additional cost caused by rerunning the task.

    A simplified scoring model could be written as follows:

    1. Overall score =
    2. Functional pass rate × 40%
    3. + State reliability × 20%
    4. + Visual and responsive quality × 20%
    5. + Execution efficiency × 10%
    6. + Recovery stability × 10%

    The weights can be adjusted according to the nature of the project.

    For internal tools, functionality and state reliability should have the highest priority. For presentation-oriented products, visual quality and responsive behavior may receive greater weight.

    7. Conclusion: Do Not Choose a Model Based Only on Its Fastest Run

    The most important conclusion from this benchmark is that an AI agent’s real capability cannot be determined by response speed or a final screenshot alone.

    Model A’s strength was rapid convergence. Model B stood out for visual consistency and mobile performance. Model C delivered the most complete system, while Model D demonstrated the importance of restrained design and recovery strategy. Each represented a different engineering trade-off.

    When a task emphasizes rapid delivery, teams should prioritize first-attempt success rate and action count.

    When a task involves complex state management, evaluation should focus on refresh persistence, permission logic, and local-repair capability.

    When a task emphasizes cross-device experience, mobile acceptance testing should be part of the formal metrics rather than an optional check.

    A more reliable evaluation method is to standardize the test environment, record the complete execution trace, distinguish first-attempt success from retry-based success, and compare outcome quality, process cost, and recovery capability at the same time.

    Only then is it possible to determine whether a model is truly suitable for a development workflow, rather than merely capable of generating a page that appears complete.