Hands-On with a Domestic Open-Source LLM: Frontend Code Generation and Multi-Scenario Viability

1. Background and Testing Methodology

A new domestic open-source large language model has recently been released, with claimed performance metrics aimed directly at leading international closed-source products. Community discussion about whether domestic models are ready to serve as replacements has been heating up. To verify its real capabilities, I selected an unfinished real business project as the test vehicle. The project includes the classic frontend trio: a login page, a list page, and a complex form page. No specific tech stack was mandated; the model was allowed to choose its own implementation approach. The evaluation focused on its depth of understanding of natural language requirement documents, code generation quality, and the usability of final deliverables.

It should be noted that this test used the model’s free service tier, so results may be affected by resource scheduling policies and could differ from the paid commercial version. Even so, the testing process still reflects the model’s fundamental capabilities in comprehension, generation speed, and engineering output.

2. User Experience: From Queueing to Running

2.1 Access Barriers: Queueing and Rate Limiting

On the first attempt, the system repeatedly returned “too many users currently; please queue or try again later.” This was not an isolated case—similar feedback is common in the community. For an open-source model that champions openness and accessibility, elastic compute scaling is the first test. If queueing becomes the norm, even a highly capable model will lose credibility with developers. Future versions should provide more transparent queue status indicators and optimize idle resource scheduling.

2.2 Generation Speed and Delivery Format

Once actual generation began, the entire process took about 35 minutes—noticeably slower than the minute-level responses of comparable commercial models. That said, the model deserves credit for not merely outputting scattered code snippets. Instead, it generated a complete, independently accessible project including routing, components, styles, and basic interaction logic. This “full-package delivery” model has practical value for prototype validation and rapid iteration, saving developers the time of manually assembling a project skeleton.

3. Page-by-Page Deep Evaluation

3.1 Login Page: Completion Level Approaching Production-Ready

The login page was the best-performing part of this test. The requirements document explicitly called for a carousel module, and the model not only implemented static display but also correctly added auto-switching and indicator synchronization. Page layout, input field states, and button feedback showed no obvious omissions. The overall impression falls between “high-fidelity prototype” and “launchable first version”—the kind of output that could be delivered directly after tweaking copy and color schemes.

3.2 List Page: A Clear Drop in Comprehension

Upon reaching the list page, the model’s shortcomings began to surface. The requirements document detailed three types of filter conditions, two sorting dimensions, and placeholder messages for empty data states. However, the generated result showed:

• Filters entirely missing: The search box was present, but dropdown filters and date range pickers were not implemented;
• Pagination logic broken: Clicking page numbers did not trigger data requests—only static frontend switching;
• Incomplete field display: The list lacked the status tags and action button columns required by the document.

This is not an isolated defect but a systematic weakness shared across multiple list pages. It indicates the model has insufficient understanding of common business logic like “combined conditional queries.” It is largely mimicking visual patterns rather than truly understanding the data flow and state management behind the page.

3.3 Form Page: Validation and Conditional Logic Under-Implemented

The form page was another weak spot. Although the page rendered correctly, the following details diverged from the document:

• Required field validation missing: The “company tax ID” field marked as required in the document could still be submitted when empty;
• Conditional logic not implemented: After selecting “individual user” type, the “company address” input group should have been hidden, but it remained visible;
• Length restrictions not enforced: The phone number field did not limit input to 11 digits and had no format validation feedback.

These are not advanced technical challenges—they are the most basic rules in form development. The model’s tendency to “assume” during generation reflects that its ability to extract conditional constraints from long Chinese text still has significant room for improvement.

3.4 Overall Visual Style

From a visual perspective, the output falls somewhere between “raw component stacking” and “a designed product.” It is more polished than most developers’ hand-written native styles, but compared to directly using mature UI frameworks (such as the industry-standard Ant Design or Tailwind templates), it lacks a unified visual language and spacing system. The overall assessment is “engineering usability exceeds visual impressiveness.” For B2B products pursuing pixel-level fidelity, secondary design optimization is still required.

4. Side-by-Side Comparison: Gap Analysis vs. International Closed-Source Models

Coincidentally, I had previously implemented the same set of requirements documents using an international closed-source model. The comparison is shown in the table below:

Comparison Dimension	Domestic Open-Source Model (This Test)	International Closed-Source Model (Reference)
Login page completeness	Good; carousel fully implemented	Excellent; additional theme switching support
List page comprehension	Filter logic missing; pagination broken	All filters and sorting fully implemented
Form validation	Field relationships under-implemented	Validation rules and conditional logic accurate
Generation speed	~35 minutes (free tier)	~3 minutes
Deliverable format	Accessible complete project	Code repository + local run instructions

4.1 The Core Gap: Depth of Understanding, Not Generation Speed

The speed difference may be related to resource priority on the free tier, but the gap in understanding depth is structural. The international closed-source model could accurately extract implicit logic from the document, such as “which fields belong to the advanced filter group” and “which actions require confirmation dialogs.” The domestic open-source model, by contrast, could only “identify titles and input boxes.” Its reasoning ability for state coupling and business constraints is clearly weaker.

4.2 Possible Reasons Behind the Gap

This is closely related to training data diversity and instruction fine-tuning quality. In the code generation domain, closed-source models can leverage massive private repositories and human feedback for reinforcement learning, while open-source models are inherently constrained by data compliance and compute resources. Additionally, “conditional adverbial clauses” in Chinese business documents are often comprehension pain points. The model needs strong semantic parsing ability to correctly map them to code logic—precisely a weak link in the current open-source community.

5. Rethinking the Significance of Open Source for AI Development

5.1 Open Source Does Not Mean Low Quality, But It Must Find Its Ecological Niche

Despite the shortcomings exposed in this test, the strategic value of open source should not be underestimated. Its significance lies in:

• Lowering the trial-and-error barrier: Developers can deploy locally and run privately, avoiding sensitive data leaks;
• Encouraging community contributions: Any user can contribute patches for specific defects, creating a positive cycle of “the more it’s used, the stronger it gets”;
• Breaking capability monopolies: Even if single-point capabilities still lag, the existence of open-source models provides a “second option” in the market, forcing closed-source products to improve their cost-performance ratio.

5.2 Practical Selection Advice for Developers

Based on this test, here are my pragmatic recommendations:

• Prototype validation stage: This open-source model is recommended; its full-package delivery capability helps quickly map out page structures;
• Complex business logic scenarios: International closed-source models should still be relied upon for now, or a hybrid approach of “manually written rules + model-generated skeletons” should be adopted;
• Watch version iterations: The open-source community iterates quickly. It is advisable to continuously track the model’s changelog in two dimensions: form validation and state management.

6. Summary and Outlook

Returning to the original question: what does it really mean for a domestic open-source model to benchmark against international closed-source products? My assessment is: this is a critical step from “unusable” to “basically usable,” but there is still a gap of at least two major versions before “full replacement” is achievable. At the current stage, it is more like an “efficient junior engineer”—capable of rapidly producing structurally correct code, but lacking sensitivity to business complexity.

The good news is that the open-source nature means its progress is linear and predictable. As more developers feed real project feedback back into the community, I believe domestic models will see significant leaps in core scenarios like filter logic comprehension and complex form implementation within the next six months. For technical decision-makers, now is the best time for “small-scale pilots and parallel validation”—neither missing out on open-source dividends nor betting core business on capabilities that are not yet mature.