How Chinese LLMs Are Reshaping Intelligent Programming: Experience and Value

    Evaluating Large Language Models (LLMs) requires looking beyond benchmark scores to their practical capabilities in understanding requirements, designing solutions, and executing tasks. This article explores the boundaries of natural language-driven "wish-based programming," the division of labor between large and small models, and the importance of adhering to engineering "contracts" in real-world projects. By proposing a collaborative workflow and evaluating the total cost of ownership, it highlights how Chinese LLMs can establish competitive advantages by transitioning from simply generating code to comprehensively understanding software engineering.

    I. Evaluating LLMs: Beyond Benchmark Scores

    When discussing whether domestic large models can rebuild their competitive edge, it cannot simply be boiled down to “who has the higher score” or “who generates code faster.” In actual development, a model’s value includes at least four dimensions: the ability to understand requirements, the ability to design solutions, the ability to execute specific tasks, and the ability to maintain stability under complex constraints.

    Many models perform similarly in standardized tests, but once they enter real projects, the experience can differ significantly. This is because standard tests usually describe problems completely, with clear inputs, outputs, and evaluation criteria. In real projects, however, requirements are often incomplete. Developers use extensive context, historical code, business habits, and implicit constraints to express their goals.

    Therefore, models must not only “write code according to instructions” but also understand the following:

    • What is the actual business problem the user wants to solve;
    • How the current requirement relates to the existing system;
    • Which requirements are hard constraints and which are just implementation preferences;
    • Which seemingly reasonable solutions might break the existing architecture;
    • Whether to proactively ask questions or make assumptions when requirements are ambiguous;
    • How to balance code readability, performance, compatibility, and delivery timelines.

    This is the core value of large-parameter models in complex development scenarios. They may not be faster than small models in every local task, but they are often better at extracting goals from context, reducing the cost of developers constantly correcting the direction.

    II. The Efficiency Boundaries of “Wish-Based Programming”

    A significant change in current intelligent programming is that developers can describe goals in natural language and let the model automatically handle code generation, file modifications, test additions, and problem troubleshooting. This approach can be called “wish-based programming”: the developer describes the desired outcome, and the model is responsible for transforming the goal into an executable implementation.

    This mode is highly efficient for the following tasks:

    1. Creating basic project structures;
    2. Generating conventional interfaces and data models;
    3. Writing highly repetitive CRUD logic;
    4. Adding parameter validation, logging, and exception handling;
    5. Generating test cases based on existing code;
    6. Refactoring or unifying formats for local modules;
    7. Quickly verifying the feasibility of a technical idea.

    However, it also has clear boundaries. Wish-based programming truly relies not on whether the model “can write code,” but on whether it can accurately understand the wish.

    If the goal description is too vague, small models will usually generate implementations directly based on the most common engineering patterns. This implementation might compile and pass surface-level tests, but it may not align with the project’s true design intent. Developers then need to constantly add conditions, correct assumptions, and undo incorrect modifications, ultimately saving little time.

    Therefore, wish-based programming can be divided into two phases:

    • Goal Inference Phase: Clarifying the problem background, business objectives, constraints, and design direction;
    • Execution Phase: Writing code, adjusting files, running tests, and fixing errors.

    The former relies more on the context understanding and reasoning capabilities of large models, while the latter can be handed over to smaller models with lower costs and faster speeds. Conflating the two easily leads to misjudgments of a model’s capabilities.

    III. Small Models for Execution, Large Models for Understanding

    Small-parameter models are not without value. On the contrary, in scenarios with clear task boundaries and stable inputs/outputs, small models often have advantages such as fast response times, low costs, and suitability for batch invocations.

    For example, the following tasks usually do not require complex reasoning:

    • Converting a set of fields to another format;
    • Generating API documentation from a fixed template;
    • Adding comments to existing functions;
    • Modifying variable names according to explicit rules;
    • Batch generating simple test data;
    • Checking code formatting and common syntax issues;
    • Extracting exception types from error logs.

    The common characteristic of these tasks is: clear goals, single paths, and easily expressible constraints. As long as the prompt is clear enough, small models can complete them stably.

    However, when tasks contain implicit business rules, small models are prone to the problem of being “locally correct but globally deviant.” They might complete a specific action without understanding why it exists, or choose a common solution while ignoring the unique constraints of the current system.

    The advantage of large models is reflected in higher-level judgment. They can combine historical dialogue, code structure, data relationships, and business descriptions to make reasonable inferences about user intents that are not explicitly stated. Of course, this inference is not absolutely reliable, so in critical scenarios, developers still need to explicitly confirm it and cannot treat the model’s “guesses” as formal design conclusions.

    IV. A Typical Case: Data and Document Management

    Take data and document management in an enterprise system as an example. The system contains a business data record that is simultaneously associated with several document files. The documents are not completely independent entities but exist attached to the business data.

    To avoid data inconsistency caused by concurrent operations, the system needs to add exclusive control when performing modifications, deletions, or document changes. At this point, “what object should be locked” is an easily overlooked but crucial design issue.

    If the prompt only says “add an exclusive lock to document operations,” the model might choose to lock the document directory ID. This solution is not entirely wrong, as it does prevent some concurrent operations and might pass existing tests. However, from the perspective of business relationships and overall design, a more reasonable approach might be to directly lock the primary key ID of the business data.

    The reasons include:

    1. Lock Object Aligns with Business Boundary
      Documents depend on business data, and business data is the core subject of concurrent operations. Locking directly around the business data makes it easier to express that “a single business record can only be modified by one process at a time.”

    2. Reduces Concurrent Deletion Risk
      If a background task is processing documents while another user deletes the corresponding business data, locking only the document directory may not prevent this. After locking the business data, deletions, modifications, and document operations can share the same concurrency control boundary.

    3. Simplifies Transaction Logic
      When all related operations are coordinated around the business data primary key, the transaction flow is easier to understand and facilitates troubleshooting lock waits, deadlocks, and exception rollbacks.

    4. Improves Design Maintainability
      When subsequently adding features like attachment replacement, archiving, restoration, and batch import, the same set of business-data-level concurrency control strategies can be reused.

    This case illustrates that model-generated code that “runs” does not mean the solution “fits the design.” If the model only focuses on local actions, it may choose a technically feasible implementation; if the model understands the relationship between business objects, it is more likely to complete the design along the developer’s overall line of thought.

    V. In Real Projects, “Contracts” Matter More Than Optimal Algorithms

    Engineering development does not always pursue theoretically optimal solutions. Often, projects have numerous contracts that must be followed, such as:

    • Database table structures cannot be modified arbitrarily;
    • API field names must remain compatible;
    • Certain exceptions must use unified error codes;
    • Log formats must meet audit requirements;
    • Business processes must remain consistent with legacy systems;
    • Code structures need to be easy for team members to read;
    • Certain performance optimizations cannot sacrifice observability;
    • Even if more complex high-performance solutions exist, maintenance costs should be prioritized.

    A model might propose a higher-performance, shorter implementation, but that does not mean it suits the current project. For enterprise systems, readability, auditability, portability, and long-term maintenance costs are often more important than local performance.

    An excellent model should be able to distinguish between “suggested solutions” and “mandatory constraints.” If a developer explicitly states “no additional abstraction to maintain structural uniformity,” the model should not repeatedly pitch a more complex design; if a project requires compatibility with legacy APIs, the model cannot arbitrarily change field meanings for the sake of clean code.

    These capabilities are difficult to achieve with a single prompt and usually require providing sufficient context, including project specifications, directory structures, key data relationships, historical decisions, and acceptance criteria.

    VI. Designing a Collaborative Workflow for Large and Small Models

    A more practical approach is not trying to let one model handle all tasks, but establishing a layered collaborative workflow.

    1. Large Models Clarify Requirements First

    Input the project background, existing code, business objectives, and constraints. Let the large model output:

    • Its understanding of the requirements;
    • Known constraints;
    • Potential ambiguities;
    • Recommended implementation paths;
    • Questions that need developer confirmation;
    • Potentially affected modules and test scopes.

    The goal of this step is not to write code immediately, but to avoid directional errors.

    2. Large Models Define Design Boundaries

    The design phase should focus on confirming data relationships, transaction boundaries, concurrency strategies, exception handling, and compatibility requirements. For key solutions, the model can be asked to provide both “reasons for adopting this solution” and “reasons for abandoning other solutions.”

    3. Small Models Execute Defined Tasks

    Once the design is finalized, work can be broken down into clear tasks, such as:

    • Creating specified files;
    • Adding implementations based on defined interfaces;
    • Adding unit tests for a specific function;
    • Adding logs according to unified standards;
    • Checking null handling in specified modules;
    • Completing refactoring based on existing code styles.

    At this point, the efficiency advantages of small models can be fully utilized.

    4. Large Models Conduct Final Reviews

    After the code is completed, have the large model check from an overall perspective:

    • Whether it deviates from the original design;
    • Whether boundary conditions are missing;
    • Whether concurrency safety issues arise;
    • Whether existing interfaces are broken;
    • Whether unnecessary complexity is introduced;
    • Whether tests cover critical paths;
    • Whether exceptions and rollbacks are complete.

    This “design—execute—review” workflow is more reliable than simply pursuing a one-shot generation success rate.

    VII. Evaluating the Competitiveness of Chinese LLMs

    Whether Chinese large models can rebuild competitive advantages ultimately depends not just on parameter size or marketing metrics, but on whether they can reduce total costs in actual workflows.

    Total costs here include:

    • Model invocation fees;
    • Time spent by developers writing prompts;
    • Time spent repeatedly correcting erroneous results;
    • Code review and testing costs;
    • Rework costs due to misunderstandings;
    • Enterprise compliance and data governance costs;
    • Long-term maintenance and personnel training costs.

    For individual developers, response speed and price may be more important. For enterprise projects, stability, data security, deployment methods, permission management, audit capabilities, and complex requirement understanding are equally critical.

    If a model is cheap per invocation but frequently misinterprets goals, requiring developers to correct it over multiple rounds, its actual cost may not be lower. Conversely, a large model that can accurately understand context and reduce rework may show better cost-effectiveness over the complete project lifecycle, even if its per-invocation cost is higher.

    VIII. Conclusion: Shifting from “Code Generation” to “Engineering Understanding”

    The next phase of intelligent programming competition will not stop at “whether it can generate code,” but will shift to “whether it can understand engineering.”

    A truly valuable model should be able to:

    • Extract real goals from vague descriptions;
    • Identify relationships between business objects;
    • Comply with existing project contracts;
    • Make reasonable trade-offs between multiple solutions;
    • Proactively identify concurrency, security, and compatibility risks;
    • Choose appropriate execution methods based on task difficulty;
    • Ask questions when uncertain, rather than making arbitrary assumptions.

    To form stronger competitiveness, Chinese large models also need to continuously improve these real-world engineering capabilities. They do not need to pursue absolute leadership in all scenarios, but must establish stable advantages in enterprise usability, complex requirement understanding, compliant deployment, and development workflow synergy.

    From a developer’s perspective, the most reasonable conclusion is not that “large models will definitely replace small models,” but that the two should assume different responsibilities: large models are responsible for understanding goals, designing solutions, and steering direction, while small models handle high-frequency execution, batch processing, and rapid verification. Only by establishing this division of labor can intelligent programming truly evolve from one-off code generation into a sustainable, auditable, and maintainable engineering productivity.