LLM Vertical Capabilities Explained: Code Agents, Script-Driven 3D, and Financial Document Analysis

1. From Chatting to Getting Work Done: The Common Thread in Recent Model Updates

The pace of large language model releases has picked up noticeably across the industry. Put the evaluation dimensions of these versions side by side, and a clear pattern emerges: the competitive focus has shifted from conversational experience to professional tasks that are executable and verifiable, concentrated in four areas:

1. Code agents: benchmarks no longer just ask the model to “write a function”; they test the full ability to execute multi-step work, debug, and deliver inside a real terminal environment;
2. Script-driven 3D creation: the model writes scripts for professional modeling tools directly, building and refining geometry, materials, and layouts;
3. Financial and other professional document analysis: cross-document retrieval, definition alignment, and multi-step numerical reasoning;
4. Reliability: factuality — represented by hallucination rate — is now treated as a hard metric.

The same technical logic underlies all four tracks. Understanding that logic is worth more than memorizing any single benchmark score.

2. Post-Training: The Technical Foundation of the Capability Leap

Base models set the ceiling; post-training determines how much of that ceiling gets realized. The current mainstream post-training pipeline roughly includes:

• Supervised fine-tuning (SFT): high-quality task trajectory data teaches the model “the correct way to do the work” — for example, a complete terminal troubleshooting session, or a modeling job taken from requirements to script;
• Reinforcement learning with verifiable rewards: whether code runs, whether geometry satisfies constraints, whether a numerical result is correct — these naturally verifiable signals free rewards from subjective human preference scoring and produce a cleaner training signal;
• Tool-use training: calling external tools becomes a training objective, so the model learns when to look something up, when to run code, and when to ask the user for clarification;
• Long-horizon task learning: real tasks often span dozens of steps, requiring the model to keep goals consistent and maintain state memory over long contexts.

Evaluation has evolved in step: terminal benchmarks measure end-to-end delivery rates; computer-aided design (CAD) benchmarks probe geometric and spatial reasoning; financial document benchmarks test retrieval and cross-document verification; hallucination stress sets measure factuality. Scores are just outcomes — what matters more is that these benchmarks map the boundaries of model capability.

3. Code: From Autocomplete to Terminal-Level Agents

Terminal tasks are currently the most representative touchstone. The model must complete a closed loop in a sandboxed environment: understand the task → plan the steps → run commands → observe output → fix errors → verify delivery. The core loop can be sketched in minimal pseudocode:

# Minimal closed loop for a terminal agent (illustrative)
while not task.done:
    action = model.decide(task.goal, history, observation)
    observation = sandbox.execute(action)   # Commands run in an isolated environment
    history.append((action, observation))
task.verify()  # Acceptance against verifiable criteria

The key to this round of improvement is that training now incorporates environments with executable feedback: when the model makes a mistake, the real environment “bounces it back,” so the model learns to debug instead of hard-coding its way through. Another often-overlooked dimension is cost: at the same task completion rate, differences in token consumption, number of calls, and context length translate directly into differences in your bill. Common optimizations in practice include result caching, small-model routing (delegating simple steps to lightweight models), and trimming system prompts. When evaluating models, developers should look at the “performance–cost curve,” not a single-point score.

4. Script-Driven 3D: A New Paradigm for Spatial Reasoning

Traditional 3D creation is hard to break into and slow to work through: you learn modeling software, fix meshes by hand, and tune materials over and over. The script-driven approach instead has the model generate scripts for the modeling tool directly, describing geometry, materials, lighting, and layout parametrically, which the tool then renders and executes. A sketch:

# Parametric modeling script (illustrative)
scene = Scene()
body = scene.box(size=(1.2, 0.6, 0.75), material="oak")       # Tabletop
for pos in CORNERS:
    scene.cylinder(r=0.03, h=0.72, at=pos, material="steel")  # Table legs
scene.export("desk.stl")   # Ready for 3D printing

The standout advantages of this approach:

• Reproducible and version-controllable: the model’s output is code, so it can go into version control, support diffs, and be rolled back;
• Parametric edits: change one dimension parameter and regenerate — no need to start from scratch;
• A natural fit for generative AI: LLMs are good at writing code, and a modeling tool’s scripting interface is exactly code.

The technical difficulty centers on spatial reasoning: occlusion, proportion, and assembly constraints between objects, plus the accuracy of the model’s knowledge of professional tool interfaces. On the product side, a closed loop of “online preview → online editing → export printable files → generate shareable videos” has already formed, significantly lowering the barrier to 3D content creation. For content platforms, this means the supply of user-generated 3D assets may grow — bringing new engineering challenges such as format standards, copyright review, and printability checks.

5. Financial Document Analysis: Retrieval, Verification, and Multi-Step Reasoning

Finance is a touchstone for model rigor. Its tasks are hard for three reasons:

1. Complex document formats: annual reports, filings, and audit reports are full of cross-page tables, scanned pages, and footnotes, and parsing itself loses information;
2. Cross-document verification: the same metric may be defined differently in different documents; definitions must be aligned before numbers can be compared;
3. Multi-step numerical reasoning: getting from raw data to a conclusion usually takes several rounds of computation, and an error at any step propagates to the final result.

The mature engineering pattern is a three-stage pipeline: retrieval augmentation + tool-based computation + result verification. First, structure the documents into a searchable store; then the model retrieves and locates evidence; numerical computation is handed to a code interpreter; finally, the model is forced to cite the source text to verify its answers. On the evaluation side, financial document QA benchmarks test the first two stages, while investment-workflow benchmarks test end-to-end support from screening targets to profitability analysis. The real value of these benchmarks is that they expose hallucination-prone zones: wherever the model skips tools and does mental math, error rates climb noticeably.

6. Hallucination Mitigation: From Stress Testing to Engineering Fixes

Hallucinations stem mainly from three sources: parametric knowledge is inherently bounded, training data contains noise, and decoding involves randomness. The standard industry practice is to build high-difficulty stress test sets, measure factual error rates, and treat error reduction as a core metric for version iteration. Public information suggests leading models have pushed factual error rates from double digits down to single-digit percentages — an improvement driven mainly by refusal-to-answer training and evidence-checking training in the post-training stage.

Engineering mitigations matter just as much:

• Mandatory citations: key conclusions must come with source references; without one, downgrade the answer to “uncertain”;
• Tools first: numbers, dates, and statistics always go through computation or retrieval — never let the model “recall” answers;
• Self-verification: after generation, have the model review its own output critically, or cross-check with a second model;
• Expressing uncertainty: at both the training and prompt levels, encourage “say you don’t know when you don’t know.”

7. Practical Advice for Developers

Based on these trends, four actionable recommendations:

1. Align evaluation with your business: don’t put blind faith in general leaderboards. Build a small evaluation set from your own real business tasks, and focus on completion rate, error rate, and unit cost;
2. Design a proper “tool layer”: whether it’s code execution, 3D rendering, or financial computation, connect it outside the model through standardized tool-invocation protocols — the model decides, the tools execute and verify;
3. Lean on cloud infrastructure: mainstream cloud platforms, Baidu AI Cloud among them, provide commodity capabilities such as object storage, container platforms, function compute, logging, and monitoring and alerting — enough to support the full pipeline of “document ingestion → sandboxed execution → result verification → pipeline monitoring” without building the underlying infrastructure yourself;
4. Take security and compliance seriously: in financial scenarios, implement data masking and permission isolation; for generated 3D content, watch for copyright review and export-format compliance.

8. Closing Thoughts

The signal from this round of model iteration is clear: capabilities are moving from conversation to execution, from general to vertical, and reliability is becoming a competitive dimension on par with raw performance. For developers, rather than chasing single-point scores, it is more valuable to internalize the methodology of “post-training + verifiable feedback + tool orchestration” and to build solid evaluation and tool layers within your own business. That is what determines whether you can turn model capability gains into real product value.