Multi-Model Agent Execution Efficiency in Complex Collaborative Systems: A Cross-Model Analysis
This article compares four models under a unified agent workflow for building a creative studio collaboration system. It examines not only final page quality but also execution paths, tool calls, failure recovery, retry behavior, context management, and total time. Readers will learn how to break down agent efficiency, interpret completion-time differences, and apply layered acceptance checks, clearer state modeling, explicit retry strategies, and process metrics in complex agent tasks.
1. Why Compare Models Instead of Only Final Pages
In agent development practice, the final deliverable is often just a webpage, a backend system, or a set of interactive prototypes. Judging model quality only by final screenshots can easily overlook key differences in the execution process.
For complex tasks, a model must not only generate code. It also needs to understand requirements, plan page structure, read assets, call browser tools, validate interaction logic, handle runtime errors, and complete modifications within a limited context and task budget. Therefore, what really deserves measurement is not simply “whether it can be built,” but the following questions:
- Can it accurately break down complex requirements?
- Can it reduce unnecessary tool calls?
- Can it quickly locate the cause after an error occurs?
- Does it fall into repeated attempts?
- Can it preserve business state across pages and roles?
- Can it fix critical issues within a single limited repair opportunity?
- Does the final result balance visual quality, interaction completeness, and mobile adaptation?
This comparative experiment fixes the runtime framework, task assets, context capacity, browser environment, and acceptance rules, replacing only the underlying model. The purpose is to isolate variables as much as possible so that the observations are closer to “the model’s actual execution capability within a unified agent workflow.”
2. Unified Experimental Task: Building a Creative Studio Collaboration System
The experimental task is to build a collaboration system for creative teams. The system needs to form a complete loop around projects, assets, versions, and comments.
The input data is relatively limited, mainly including:
- A set of openly licensed visual assets;
- Project members and their role information;
- Initial asset data;
- Version records;
- Seed data for comments and approvals.
The system cannot directly imitate an existing page. The model must design the information architecture, visual language, and interaction flow on its own.
The task includes at least five core states:
1. Project Overview
The project overview needs to show basic project information, current progress, participating members, phase status, and recent activity. It is both the homepage users enter after accessing the system and the navigation center for subsequent operations.
2. Moodboard and Asset Organization
Users need to browse assets, view asset details, and add assets to a creative board through drag-and-drop or another clear interaction. The system must not only display images but also reflect the relationship between assets and projects.
3. Asset-Specific Comments
Comments cannot be just a simple text list. They should be connected to specific assets or canvas positions as much as possible. This allows designers and clients to know which object an opinion refers to, reducing communication ambiguity.
4. Version Approval
The system needs to support version switching, viewing version information, submitting for approval, approving, and returning. Users with different identities should have different permissions and operation scopes.
5. Release Preview
Release preview is used to check the current project state and let users view results in a way close to final delivery. The preview page must remain consistent with the project, version, and approval status rather than becoming an isolated static page.
In addition, the system defines three identities: creative director, designer, and client. Role differences should not only appear in page copy but also be reflected in operation permissions, button states, and process paths. After refreshing the page, project status, comments, version selection, and approval results must not be lost. This places higher demands on frontend state management and data persistence.
3. Experimental Environment and Acceptance Constraints
To make the comparison referenceable, all four experiments use the same execution conditions:
- The same agent runtime framework;
- The same task description and assets;
- The same context capacity;
- The same task budget;
- The same browser automation tool;
- Desktop check size of 1440×1000;
- Mobile check size of 390×844;
- Save screenshots of the first version for each group;
- Allow only one round of repair;
- Retain the final code, screenshots, execution logs, review records, and metric data.
The “one round of repair” constraint is especially important. It simulates a constrained iteration scenario in real development: the model cannot modify pages indefinitely, nor can it rely on many repeated attempts to hide insufficient early planning. The model must establish the correct structure as much as possible during the initial implementation and use the limited repair opportunity to solve core issues affecting acceptance.
4. Execution Results of the Four Models
In the experiment, all four models eventually completed the task, but there were clear differences in execution efficiency and failure recovery methods.
Model A: Completed in About 17 Minutes
Model A completed the entire system in about 17 minutes, and the overall process was relatively direct. It adopted a lightweight information architecture, presenting project status, assets, and member relationships together. Users could quickly understand what phase the current project was in, who the participants were, and what operations could be performed next.
In terms of execution efficiency, the model’s advantages were mainly reflected in three aspects:
First, task decomposition was relatively concise, and it did not consume a large number of rounds too early on local visual details.
Second, the tool-call path was relatively short, and it could quickly enter the browser validation stage after completing the basic pages.
Third, when problems occurred, it tended to make local fixes rather than large-scale refactoring, reducing context and time costs.
Its interface style leaned toward a light-colored editorial workspace, with clear structure and fewer hierarchy levels. Although the visual expression was not necessarily the most complex, the presentation of project overview, asset management, and member relationships was direct, reflecting the tradeoff of “ensuring understandability first, then adding decoration.”
Model B: First Failure, Completed in About 72 Minutes After Rerun
Model B did not pass full acceptance on its first execution and then completed the task by rerunning. The total time was about 72 minutes.
This result shows that whether a model can generate pages is not equivalent to whether it can stably complete an agent task. The first failure may come from a combination of multiple stages, such as:
- The initial page structure did not fully match the acceptance rules;
- Key interactions were not truly closed-loop in the browser;
- Mobile layout overflowed;
- State recovery after refresh was incomplete;
- Role permissions remained only at the visual level and were not implemented in operation logic;
- The first implementation consumed too much time, leaving insufficient room for later repair.
The fact that rerunning could complete the task shows that the model itself has a certain implementation capability, but stability and first-round planning ability remain important factors affecting efficiency.
In terms of design performance, Model B used a visual scheme combining dark colors and warm colors, and strengthened brand expression through serif headings, approval statuses, and a permission matrix. Among the four results, it stood out in mobile consistency and overall visual recognition. This shows that visual completeness and execution efficiency are not always synchronized: a model may need more time to produce a more stylized result, but this does not mean its action path is more efficient.
Model C: Multiple Internal Retries, Completed in About 88 Minutes
Model C did not directly rerun the entire round. Instead, it performed three automatic retries during execution and finally took about 88 minutes.
This execution pattern differs from Model B. Model B’s problem appeared as restarting after a whole-round failure, while Model C continuously patched local issues within one task lifecycle. Both increase total time, but they place different demands on context continuity, code state, and error recovery strategy.
Model C eventually built a dark professional workbench. Navigation, project progress, role management, delivery, approval, and asset canvas were organized into a unified system. Its advantages were high information density, relatively complete functional coverage, and a more systematic engineering structure.
However, high information density also brought two risks:
First, as page complexity increased, the model needed to handle state relationships among more components.
Second, the tool-call chain became longer, and any local state error could trigger subsequent cascading problems.
Therefore, although automatic retries can improve the probability of task completion, they do not necessarily improve actual efficiency. If retries are not accompanied by clear error classification and context compression, the model may simply repeat similar operations, rapidly consuming time and budget.
Model D: Completed in About 64 Minutes After Backoff Adjustment
Model D initially used the default retry strategy and failed twice in a row. It then expanded the backoff interval and adjusted the execution method, achieving its first successful completion after about 64 minutes in total.
This result reflects the direct impact of service stability and retry strategy on agent efficiency. When there are temporary errors, response delays, or tool execution failures in the call chain, immediately repeating the request may not be effective. A reasonable backoff strategy should distinguish:
- Parameter or code errors that can be fixed immediately;
- Temporary errors that require waiting for environment recovery;
- Structural errors that require changing the execution plan.
Model D ultimately presented an art museum archive system style, using dark serif headings, an ivory background, thin-line tables, and artwork thumbnails. The overall tone was restrained and unified. It emphasized content arrangement and an archival feel rather than a high-density control panel.
5. Total Time Does Not Equal Model Capability
The completion results of the four experiments can be summarized as follows:
| Model Category | Execution Characteristics | Final Time Performance |
|---|---|---|
| Model A | Direct path, fewer tool calls | About 17 minutes |
| Model B | Whole-round rerun after first failure | About 72 minutes |
| Model C | Multiple automatic retries within the task | About 88 minutes |
| Model D | Default retry failed, succeeded after backoff adjustment | About 64 minutes |
These data cannot simply be interpreted as “the fastest model is comprehensively the best.” The reason is that task completion includes at least four dimensions:
- Functional correctness: Can the page complete the specified operations?
- Interaction completeness: Are drag-and-drop, comments, version switching, and approval truly usable?
- Visual and responsive quality: Are desktop and mobile consistent?
- Process efficiency: How many rounds of action, calls, and recovery are needed to complete the task?
If only final screenshots are considered, one may overestimate a model with outstanding visual performance but a long execution process, and may underestimate a model with a plain interface but stable delivery.
A more reasonable evaluation method is to break total time into:
Total time =Planning time+ Code generation time+ Browser operation time+ Error localization time+ Retry wait time+ Final repair time
Among these, retry wait time and error localization time are often not emphasized by traditional model benchmarks, but they are critical parts of real agent systems.
6. Five Core Factors Affecting Agent Efficiency
1. Number of Action Rounds
Every time a model performs a browser operation, file modification, or state read, additional cost is incurred. The more action rounds, the more likely context expansion, state drift, and local error accumulation become.
2. Tool-Call Quality
More tool calls are not necessarily better. High-quality calls should have clear purposes, such as first confirming page structure, then checking key paths, and finally repairing acceptance items. Aimless repeated screenshots, repeated reads, and indiscriminate retries significantly reduce efficiency.
3. Error Recovery Capability
A stable agent needs to first determine the error type and then decide whether to modify code, reload the page, wait for service recovery, or adjust the execution plan. Treating all errors as “try again” often wastes time.
4. Context Management
Complex tasks simultaneously contain requirements, code, logs, page state, and acceptance criteria. If the model cannot effectively compress historical information, it may forget early constraints or break completed functionality during later repairs.
5. Service and Environment Stability
Model performance is not entirely determined by reasoning quality. Call latency, browser response, execution environment status, and retry backoff strategy all affect final delivery time. Therefore, when evaluating agents, both model metrics and runtime environment metrics must be recorded.
7. Practical Recommendations for Developers
Establish a Layered Acceptance Checklist
It is recommended to divide acceptance rules into three layers:
- Basic layer: Can pages load, and are core routes accessible?
- Interaction layer: Are drag-and-drop, comments, approval, and version switching closed-loop?
- Quality layer: Do mobile adaptation, refresh persistence, and visual consistency meet standards?
This prevents the model from focusing only on “the page looks complete” while missing key business states.
Prioritize a Clear State Model
In complex collaboration systems, projects, members, assets, versions, comments, and approval states are related. The data structure should be determined before generating page components. A simple, stable state model is usually more important than stacking more visual components.
Make Retry Strategies Explicit
Retries should not be unconditional repetition. The following approach can be used:
If it is a code error:Read logs and make a local repairIf it is a page state error:Reload and confirm the current stateIf it is a temporary service error:Wait with increasing intervals and retryIf consecutive failures exceed the threshold:Replan the execution path
Save Process Metrics as Well
In addition to final screenshots, record:
- Time to first runnable version;
- Time to first pass through the core flow;
- Number of tool calls;
- Number of automatic retries;
- Distribution of failure types;
- Acceptance differences before and after repair;
- Number of desktop and mobile issues.
These metrics are more helpful than a single total time for locating system bottlenecks.
8. Conclusion
The most important conclusion of this comparison is not that one model is superior in all scenarios, but that the efficiency of complex agent tasks depends on a complete coordination mechanism.
Under a unified framework, unified task, and unified acceptance conditions, all four models completed the creative collaboration system, but completion time ranged from about 17 minutes to about 88 minutes. The differences mainly came from execution paths, tool calls, failure recovery, retry strategies, and context management, not merely model parameter scale or final page style.
For enterprise applications and real R&D workflows, model selection should not focus only on generation quality or single benchmark scores. It should also evaluate stability, recoverability, and delivery efficiency in real workflows. A more mature agent system should not leave all problems to the model to solve on its own. Instead, it should use clear task decomposition, structured state, layered acceptance, logging, and refined retry mechanisms to help the model reliably complete complex tasks within a limited budget.
