Chinese LLMs Advance Towards Advanced Programming and Real-World Agents

    As Large Language Model (LLM) technology evolves, industry evaluation standards are shifting from basic parameters and single-turn Q&A to complex coding, long-task execution, and reliable tool integration. This article explores how emerging Chinese LLMs are transitioning from content generation tools to real-world software agents. It covers the shift toward open training processes, the importance of end-to-end coding capabilities, GUI-based agent operations, the impact of post-training on unit costs, and the necessary strategies for enterprise system evaluation and developer adaptation in this changing landscape.

    With the continuous evolution of Large Language Model (LLM) technology, industry evaluation standards are undergoing a significant shift. Early focus centered on parameter size, basic Q&A, and single-turn generation quality. Today, a model’s true application value is increasingly determined by its complex programming capabilities, long-task execution, reliability of tool invocation, and its ability to integrate stably into enterprise workflows at a reasonable cost.

    Recently, several Chinese model teams have been advancing open-source training processes, improving code generation quality, operating real devices, and optimizing inference costs. While these efforts span different technical directions, they reflect a unified trend: LLMs are evolving from mere “content generation tools” into “software agents” capable of understanding environments, planning steps, and executing tasks.

    1. Model Openness is Moving from “Open Weights” to “Open Training Process”

    In the past, open-source models typically only provided the final trained weights. Users could download, deploy, and fine-tune them, but it was difficult to understand what the model experienced during pre-training, mid-training, capability merging, and post-training phases. While this approach lowered the barrier to usage for researchers, it limited in-depth verification of the training mechanisms.

    A more valuable new trend is the gradual release of checkpoints from different training stages. For example, in addition to the final base weights, models may provide multiple versions covering the pre-training phase, mid-training capability enhancement phase, and phases where different training strategies are merged. This allows the community to study not just “how the final model performs,” but also to analyze:

    1. Whether a specific capability was formed during pre-training or acquired in subsequent training stages;
    2. The impact of different data ratios on coding, math, reasoning, and language capabilities;
    3. Whether mid-training leads to a degradation of general capabilities;
    4. Whether capabilities complement each other after merging multiple training objectives;
    5. Whether training conclusions from small-scale models can be transferred to larger-scale models.

    This level of openness is significant for both academic research and engineering practice. Researchers can conduct controlled experiments using different checkpoints rather than training a full model from scratch. Enterprises can also continue domain adaptation based on existing training stages using their own business data, thereby reducing compute, time, and data preparation costs.

    However, open checkpoints do not mean the training process is entirely transparent. A fully reproducible system must also include data processing principles, training objectives, evaluation methods, version differences, and safety constraints. Without this context, differences between checkpoints remain difficult to accurately explain. Therefore, the future competitiveness of open-source models will depend not only on weight size but also on whether the training process is analyzable, reusable, and verifiable.

    2. Programming Capability has Become a Key Metric for Practical Value

    The improvement of LLM coding capabilities is one of the most directly perceivable changes for developers today. Simple code completion is no longer the main challenge; the capabilities of true engineering value include:

    • Reading and understanding large code repositories;
    • Modifying multiple related files based on requirements;
    • Locating anomalous logs and proposing fixes;
    • Writing test cases and iterating based on results;
    • Handling dependencies, configuration files, and build processes;
    • Refactoring code while maintaining original functional stability;
    • Proactively clarifying boundaries and risks when requirements are uncertain.

    This means that model evaluation cannot solely focus on whether a single function is generated correctly; it must also observe whether the model can complete a full software development loop.

    A reasonable evaluation process usually includes the following steps:

    1. Requirement Understanding

    The model needs to identify task goals, inputs/outputs, boundary conditions, and potential constraints. For example, “adding an API endpoint” does not just mean generating routing code; it also involves parameter validation, access control, error handling, logging, and test coverage.

    2. Code Localization

    In real-world projects, related logic may be scattered across multiple directories and modules. The model must determine which files need modification, which code can be reused, and which components have implicit coupling.

    3. Solution Design

    The model should provide a modification plan first, rather than immediately generating large amounts of code. A high-quality plan typically explains the scope of impact, compatibility risks, data structure changes, and testing strategy.

    4. Code Implementation

    The implementation phase should not only aim for functional viability but also consider readability, exception handling, performance, and security. For production systems, generating “plausible-looking” code is far from sufficient.

    5. Automated Verification

    After the model generates code, it needs to be verified through compilation, static analysis, unit testing, and integration testing. Only through multiple rounds of tool-feedback corrections can a code agent reach an engineering-ready level.

    Therefore, the core capability of a coding model is not simply “writing code faster,” but the ability to continuously understand, modify, verify, and iterate within a real development environment. When integrating a model, development teams should prioritize observing task completion rates, rework counts, test pass rates, and manual review costs, rather than just the quality of code snippets in demo environments.

    3. GUI Operations Mark Agents Entering Real Work Environments

    Traditional language models interact primarily through text, whereas a vast number of real-world tasks occur on web pages, desktop software, and mobile devices. For example, users may need to log into a system, fill out forms, query information, copy data, upload files, and submit approvals. These tasks often span multiple pages and applications and cannot be accomplished merely by generating text.

    Agents oriented toward Graphical User Interfaces (GUIs) need to possess four types of capabilities:

    1. Visual Understanding: Recognizing buttons, text, lists, pop-ups, icons, and page hierarchies;
    2. State Judgment: Understanding what stage the current page is at and judging whether the previous operation was successful;
    3. Action Planning: Breaking down goals into steps such as clicking, inputting, scrolling, selecting, and confirming;
    4. Exception Recovery: Handling page loading failures, element displacement, permission pop-ups, and network fluctuations.

    Compared to single-turn Q&A, the difficulty of GUI tasks lies in the accumulation of errors. A wrong click in a previous step can lead to a completely different page state subsequently. Therefore, agents cannot rely solely on one-time planning; they must continuously observe the environment during execution and adjust actions based on feedback.

    Real device data is crucial for training these capabilities. Simulated environments can quickly generate a large number of trajectories, but they often fail to fully cover the latency, pop-ups, layout differences, and exceptions present in real applications. Using real mobile phones, computers, and web environments to collect operation trajectories helps improve the model’s adaptability to complex interfaces and uncertain states.

    However, several key issues still need to be resolved before GUI agents can be applied at scale:

    • How to prevent misoperations and high-risk actions;
    • How to require human confirmation for actions involving payments, permissions, and privacy;
    • How to save auditable operation logs;
    • How to determine if a task is truly completed, rather than just stopping at a superficial state;
    • How to maintain a certain level of adaptability after UI redesigns.

    Therefore, when deploying such systems, enterprises should adopt a “low-risk tasks first” strategy. Information retrieval, data summarization, and spreadsheet organization can be prioritized, while operations involving funds, account permissions, contract submissions, and production environment changes should have human approval, permission isolation, and rollback mechanisms in place.

    4. Model Upgrades are Shifting Focus to Post-Training and Unit Cost

    Improvements in LLM capabilities do not necessarily mean a continuous expansion of parameter size. As base models mature, post-training, task alignment, inference strategy optimization, and tool invocation optimization are becoming critical means to enhance practical performance.

    Post-training can reinforce model behavior for specific scenarios, such as:

    • Improving planning capabilities for complex coding tasks;
    • Reducing the probability of generating unsafe code;
    • Enhancing information retention in long texts;
    • Improving context consistency in multi-turn dialogues;
    • Enhancing format stability when the model calls external tools;
    • Prompting the model to actively request more information when uncertain.

    If a model maintains a similar call cost after an upgrade, developers can more easily migrate it from experimental projects to daily workflows. For enterprises, what truly needs to be calculated is not just the single-call price, but the task success rate, average retry count, manual intervention ratio, and overall processing time.

    A simple metric can be used to measure actual costs:

    Unit Effective Task Cost = Total Call Cost ÷ Number of Successfully Completed Tasks

    If a model is cheap per call but frequently produces errors, requires multiple retries, or must be manually corrected, its unit effective task cost may not be low. Conversely, a model with a slightly higher call price that can stably complete complex tasks might be more suitable for production environments.

    5. Enterprise Deployment Needs to Shift from Model Evaluation to System Evaluation

    Evaluating model capabilities in isolation cannot fully reflect the true effectiveness of an agent system. Enterprises should test the model, tools, data, permissions, and monitoring mechanisms as an integrated whole.

    It is recommended to cover at least the following dimensions:

    Task Success Rate

    Focus on whether the model truly achieves the business objective, rather than just whether it generated text in the correct format.

    Stability

    Observe result fluctuations under different inputs, page states, and network conditions.

    Security

    Test whether the model will access data beyond its permissions, leak sensitive information, execute high-risk commands, or generate unsafe code.

    Explainability and Auditability

    Record the model’s inputs, outputs, tool calls, key decisions, and final results to facilitate issue tracking.

    Cost and Latency

    Evaluate average response time, peak concurrency, retry overhead, and long-term operating costs.

    At the infrastructure level, this can be combined with object storage to save task data, use message queues to handle asynchronous tasks, deploy model services via container platforms, and utilize logging services and monitoring alert systems to track operational status. These are general engineering capabilities and do not depend on any specific model or platform.

    6. How Developers Should Adapt to These Changes

    Faced with continuously enhancing model capabilities, the developer’s role will not simply disappear; it will shift. Repetitive coding tasks may be automated, but system design, requirement breakdown, risk assessment, quality assurance, and architectural decisions still require professional experience.

    Developers can focus on improving the following capabilities:

    1. Task Breakdown: Transforming vague requirements into clear, verifiable steps;
    2. Code Review: Identifying hidden defects in model-generated code;
    3. Test Design: Constraining model behavior through tests rather than relying on subjective judgment;
    4. Tool Orchestration: Allowing models to securely call databases, retrieval services, and automated workflows;
    5. System Security Awareness: Controlling permissions, data boundaries, and high-risk actions;
    6. Result Evaluation: Establishing quantifiable metrics for quality and cost.

    The more efficient development model of the future will likely not be a choice between “developer or model.” Instead, developers will be responsible for goals, constraints, and acceptance criteria. The model will handle information gathering, code drafting, localized implementation, and repetitive operations, while the system ensures result quality through automated testing and monitoring.

    Conclusion

    Overall, the development of Chinese LLMs is entering a more pragmatic phase. On one hand, training processes are gradually opening up to the community, lowering the barriers to research and adaptation. On the other hand, models are beginning to delve deep into code repositories, real devices, and enterprise workflows, with the focus of competition shifting from parameter size to task completion capability. For developers and enterprises, the most important thing is not to chase any single model, but to establish an intelligent application engineering system that is evaluable, controllable, auditable, and continuously optimizable. Only when model capabilities are truly combined with data, tools, permissions, and business processes will LLMs transition from demo technologies into stable productivity tools.