From Base to Post-Training: Evaluating the Boundaries of LLM Visual Programming Capabilities

1. The Starting Point: Base Model Scale Is No Longer the Only Variable

Over the past year, the primary competitive landscape for open-source large language models (LLMs) has shifted significantly. Early competitions focused on parameter scale and pre-training compute power—whoever could build a higher benchmark score using larger corpora and longer training cycles dominated the conversation. However, as base model sizes approach the trillion-parameter level, marginal returns have begun to diminish. A larger foundation model undoubtedly delivers stronger general comprehension, but this does not necessarily translate into usability in real-world development scenarios.

A typical gap appears in terminal-based tasks. An open-source flagship base model might perform exceptionally well on general knowledge Q&A and code completion benchmarks, yet achieve only a low double-digit pass rate on terminal operation benchmarks that require multi-step command orchestration, environment state evaluation, and error backtracking. This indicates that while base models possess “knowledge,” real-world productivity requires the ability to “get things done within constrained environments.” A gap exists between the two that must be bridged through post-training.

This leads to the core question of this article: When a sufficiently large base model undergoes prolonged, multi-domain joint reinforcement learning (RL), to what extent can it improve in key dimensions such as visual aesthetics, front-end code generation, and terminal task execution? What is the cost of this improvement, and where do the boundaries lie?

2. Technical Approach: Why Post-Training Over Scaling Parameters

From an engineering perspective, continuing to scale up a base model is a “high-certainty, but extremely high-cost” path. It requires preparing even larger corpora, redesigning parallel strategies, and accepting longer training cycles with higher failure risks. In contrast, applying RL during post-training on an existing base model offers three practical advantages.

First, directional capability injection. RL excels at optimizing “verifiable objectives.” Whether code passes tests, front-end pages render as expected, or terminal tasks exit successfully—these are automatically verifiable signals that naturally serve as reward sources. Compared to unsupervised next-token prediction during pre-training, post-training allows compute to be concentrated on the aspects that directly impact usability.

Second, synchronized toolchain evolution. Improving model weights alone cannot release capabilities if the external execution environment fails to keep pace. Therefore, supporting sandboxes, terminals, and browser toolchains must iterate simultaneously with the model. Sandboxes isolate execution risks, terminals provide command-level interaction, and browser tools support page rendering and visual validation. The model and the toolchain share a collaborative, not alternative, relationship.

Third, controllable cost structure. Post-training does not require rebuilding the entire training infrastructure. It can iterate on existing clusters while driving down per-call costs through inference-side optimizations. This is crucial for practical deployment—no matter how capable a model is, if the invocation cost is too high to scale, it remains stuck at the demo stage.

To summarize this approach: Maintain the base model’s architecture, use multi-domain joint RL to refine behavioral policies, and drive service costs down to a scalable level.

3. Capability Evaluation: Real-World Performance Across Three Dimensions

3.1 Visual Aesthetics and Front-End Generation

Front-end code generation serves as a touchstone for evaluating multimodal comprehension and aesthetic capabilities. It requires the model to not only write syntactically correct structures but also make reasonable judgments regarding layout proportions, spacing rhythm, color matching, and hierarchical contrast. In tests, models undergoing this specific post-training demonstrated outstanding performance in the following areas:

• Layout structure rationality: Generated page skeletons typically exhibit a clear sense of grid layout, with block divisions conforming to common design specifications, minimizing element overlap or overflow.
• Visual hierarchy expression: The model proactively uses font size gradients, font weight contrasts, whitespace, and shadows to construct information hierarchies, rather than laying out all content with the same visual weight.
• Style consistency: It maintains uniform color schemes, border radii, borders, and shadow styles within the same page, reducing the “patchwork feel.”

It must be objectively noted that visual aesthetics are subjective. Preference voting in benchmark tests only reflects statistical tendencies and cannot replace real user acceptance. In engineering practice, it is still recommended to treat model outputs as a “high-quality first draft,” with final reviews conducted by design or front-end personnel.

3.2 Programming Benchmarks and Complex Tasks

On comprehensive programming benchmarks, the improvements brought by post-training are primarily reflected in the completion rate of long-chain tasks. A common failure mode for base models is “losing the objective midway” or “repeatedly retrying in the wrong direction.” Models that have undergone RL show significant improvements in the following areas:

Task decomposition   -> Breaking requirements into verifiable sub-steps
State tracking       -> Remembering completed modifications and pending items
Error backtracking   -> Locating root causes based on errors rather than blind retries
Convergence judgment -> Recognizing task completion conditions to avoid over-editing

These four points correspond to the core capabilities of agentic programming. In practical use, the model can be integrated into a loop with test feedback: the model generates modifications, tests provide pass or fail signals, and failures trigger regeneration with the error information. This “generate-verify-correct” closed loop is much closer to real-world engineering workflows than one-shot generation.

3.3 Terminal Task Execution

Terminal tasks represent the weakest area for base models, yet they show the most significant gains from post-training. These tasks require the model to understand command semantics, evaluate the current environment state, execute sequentially, and handle exceptions. Early base models had very low pass rates on these benchmarks, but after targeted RL, performance improved by an order of magnitude.

The technical key lies in the fact that every step of a terminal task has a clear, observable outcome (exit codes, output text, file states). This allows RL to receive dense reward signals, unlike open-ended generation which relies solely on sparse human preference annotations.

4. Cost Perspective: Strategy Shifts Driven by Price Reductions

When capability improvements are accompanied by cost reductions, usage strategies shift directly. As per-call costs drop to roughly one-fifth of their original price, several previously uneconomical use cases become viable:

1. High-frequency, small tasks: Short requests like code completion, unit test generation, and comment completion can be confidently delegated to the model without worrying about runaway costs.
2. Multi-round self-correction: Allowing the model to self-check after generation and retry upon failure uses more rounds to achieve a higher first-pass success rate, while keeping the total cost acceptable.
3. Batch refactoring: Large-scale style unification or interface adaptation for legacy code is well-suited for running batch pipelines with low-cost models.

However, a reminder is necessary: low cost does not mean zero cost. Multi-round retries amplify request volumes, and batch tasks amplify token consumption. Actual budgets must still be estimated using “task volume × average rounds × unit price,” rather than just looking at the per-call price.

5. Productivity Boundaries: What Is Usable and What Requires Caution

Based on comprehensive testing, applicability can be categorized into three tiers:

Ready for direct deployment

• Front-end page draft generation, paired with manual visual review
• Unit test and boilerplate code generation
• Format conversion, documentation generation, comment completion
• Script writing and debugging within a controlled sandbox

Requires strong manual oversight

• Automated operations involving production environment changes
• Refactoring and migration of complex business logic
• Security-sensitive code (authentication, encryption, input validation)
• Long-period tasks spanning multiple systems

Not recommended to rely on currently

• Correctness judgments lacking test coverage and automated verification
• Decisions requiring the tacit knowledge of domain experts
• Autonomous execution of high-risk, irreversible operations

The core logic of this categorization is: Tasks that can be automatically verified are suitable for closed-loop model execution, while non-verifiable tasks must retain a human confirmation step. RL can only optimize the former.

6. Engineering Practice Recommendations

If you plan to introduce such models into your projects, it is recommended to proceed in the following order:

1. Build an evaluation set first: Extract representative tasks from real business scenarios, label expected results, and form a reproducible evaluation set. Without an evaluation set, any claim of “improved performance” lacks foundation.
2. Integrate the verification loop: Configure test cases for code tasks, render screenshot comparisons for front-end tasks, and exit code/output assertions for terminal tasks. The clearer the verification signal, the more effective the model’s self-correction.
3. Control execution boundaries: All automated executions should occur within an isolated sandbox, restricting network and file system permissions, and retaining comprehensive operation logs for backtracking.
4. Gradual rollout: Run low-risk tasks successfully first, then gradually expand to core pipelines. Re-evaluate the pass rate and manual intervention ratio with each tier of expansion.

7. Conclusion

The base model scale determines the ceiling of a model’s capability, but post-training determines how much of that ceiling can be realized. From our testing, prolonged, multi-domain joint RL genuinely delivers generational improvements in visual aesthetics, front-end generation, and terminal execution. Coupled with cost reductions, capabilities once limited to demos are entering the usable range. However, the boundaries remain clear: verifiable tasks are suitable for closed-loop model execution, while non-verifiable tasks still require human fallback. Understanding this dividing line holds far more engineering value than chasing any single benchmark score.