Long-Horizon Coding with LLMs: From Working Code to Deliverable Software

    A hands-on evaluation of a Chinese-developed large model on two long-horizon coding tasks: a legacy-project enhancement with security constraints and a zero-to-product Agent learning game. The results show clear progress in complex context understanding, cross-module execution, and sustained autonomous work, but a P1 permission-isolation flaw found during manual acceptance highlights that human review remains essential for permissions, security, and final delivery. The article also offers practical guidance for using such models on real engineering work.

    1. Bottom Line Up Front

    The core takeaway from this hands-on test is that the upper bound of coding capability for Chinese-developed models has moved up again. More important than leaderboard scores, however, is performance on long-horizon tasks: whether the model can understand complex context, sustain cross-module changes, and bring a task to a runnable and testable state. These three dimensions are exactly what production-grade software engineering demands from AI coding tools.

    But “working” does not equal “deliverable.” In the test, the model completed two very different projects. The legacy-project modification, which involved historical code and security constraints, exposed a P1-level permission-isolation issue during manual acceptance. This shows that humans cannot yet step away from permissions, security, and final acceptance.

    2. What Changed: Same Base Model + Post-Training Scaling

    This upgrade did not replace the base model. It still uses the same large-parameter base at the same scale as the previous generation. The capability gains come mainly from post-training scaling:

    • Long-horizon task environments expanded dozens of times larger: the number and types of task environments exposed to the model during training increased substantially, giving the model a richer set of engineering contexts;
    • Longer training: more reinforcement-learning rounds on complex tasks improved stability across long task chains.

    The vendor groups the improvements into three areas:

    1. Coding: positioned as an open-source model with leading coding capability, maintaining an open-source lead on terminal tasks and software-engineering benchmarks, with greater emphasis on long-horizon tasks and production-grade engineering;
    2. Cybersecurity: capability extends from “generating code” to “white-box code review and vulnerability reasoning.” On basic security tasks it approaches frontier closed-source models, but gaps remain in full exploit scenarios;
    3. Open-source strategy: full weights will be released after safety evaluation and model hardening, so developers can deploy, verify, and improve the model themselves.

    This upgrade path is worth noting: the base model is unchanged while post-training does the heavy lifting. This suggests that the current bottleneck may not lie entirely in parameter count, but also in the breadth of task-environment coverage and the depth of training.

    3. Test Design: Two Complementary Tasks

    To make the results meaningful, I deliberately chose two tasks with very different capability demands.

    Task 1: Legacy-project evolution. In a project with a large amount of historical code, add “secure user image asset import and HTML animation scene referencing” capability. This task tests:

    • Understanding the old system’s architectural conventions and code style;
    • Handling security constraints such as upload validation, permission control, and resource-reference isolation;
    • Making incremental changes without breaking existing functionality.

    Task 2: Zero-to-product. Provide only a set of Agent learning materials and a product requirements document, and ask the model to build from scratch a playable Agent knowledge-learning game that can actually be completed. This task tests:

    • The ability to extract knowledge points from unstructured materials;
    • Completeness in requirement decomposition, technology selection, and module design;
    • The ability to push a task forward autonomously for a long time without drifting.

    One task is “adding within constraints,” and the other is “subtracting from scratch.” Together they cover the two most common AI collaboration scenarios in real development.

    4. Task 1: Legacy Project—Worked, but Failed Acceptance

    The model’s performance on this task fell into two phases.

    Phase 1: Strong implementation. It proactively browsed historical code and understood the existing module layering and interface conventions. The new feature spanned the data layer, service layer, API layer, and frontend presentation layer. The final code ran, and the basic flow worked. From the perspective of “getting AI to build the feature,” it was qualified.

    Phase 2: Manual acceptance exposed a P1 issue. When I checked the security design item by item, I found a permission-isolation flaw in the asset-import chain: in some cases, user-submitted assets were not strictly validated for ownership at the reference stage, creating a theoretical risk of unauthorized referencing of other users’ resources. The model would not proactively find this kind of issue because it had “made the flow work,” and security testing was not in its default execution path.

    This case offers an important lesson: AI models are far more sensitive to functional correctness than to security completeness. They optimize for goals with clear validation signals—compilation passes, tests pass, pages open—while constraints such as permission boundaries and data ownership require adversarial thinking and still depend on human review. In security-sensitive systems, AI-generated code must undergo security review at least as rigorous as—if not more rigorous than—that applied to human-written code.

    5. Task 2: Zero-to-Product—Completeness Exceeded Expectations

    The difficulty of the second task lay in its extremely long chain: read materials → extract the knowledge system → design game mechanics → implement frontend interaction and backend logic → complete the play-through flow.

    The model showed several strengths:

    1. Strong material understanding: it did not simply list the learning materials. It extracted a hierarchical structure of knowledge points and mapped them into the game’s level design—learning concepts first, then answering questions to reinforce them, and gradually unlocking advanced content;
    2. Complete product awareness: interaction details mentioned in the requirements document were largely implemented, including answer feedback, progress saving, and completion/win-condition checks;
    3. Good long-task stability: the development process lasted several hours, with multiple rounds of self-verification and repair, without obvious goal drift or duplicated work.

    The final game could actually be played through, and the completeness of the transformation from materials to finished product exceeded my expectations. This shows that in scenarios with “given materials + clear requirements,” the current model already has end-to-end capability from analysis to implementation.

    6. How to Use These Models Rationally

    Based on this test, I summarized four practical recommendations:

    1. Use them for long-horizon tasks, but define acceptance checkpoints. Break large tasks into multiple checkable milestones, and require human confirmation before continuing, so errors do not compound across a long chain;
    2. Security-related code must be reviewed line by line by humans. For logic such as permissions, authentication, data isolation, and input validation, do not trust a “tests passed” signal alone. Proactively design adversarial test cases;
    3. Provide structured context. For legacy-project changes, providing architecture notes, coding standards, and a checklist of security constraints can significantly reduce the risk of free-form improvisation by the model;
    4. Treat AI as a junior engineer, not a senior engineer. Its execution and speed are strong, but judgment still needs human backup—especially the judgment of “what should not be done.”

    7. Conclusion

    This test gave me a clearer judgment of the coding capability of Chinese-developed models: they are now worth testing on real projects, especially long-horizon, cross-module engineering tasks. But in permissions, security, and final acceptance, the human role remains irreplaceable.

    The maturity of AI coding tools is not about “replacing developers,” but about redrawing the boundary of human-machine division of labor: models handle high-intensity, high-speed implementation, while humans define constraints, review risks, and make final decisions. Those who map this boundary faster can capture real efficiency gains while maintaining quality.