How to Evaluate LLM Coding and Multimodal Capabilities: A Practical Guide to Legacy Code Refactoring and Structured 3D Tasks

As large language models evolve at breakneck speed, vendors keep announcing record-breaking scores on public leaderboards. For frontline developers and technical decision-makers, however, the reality is often awkward: public evaluation sets suffer from serious “benchmark gaming,” with models potentially overfit to specific datasets, while vendor demos of proprietary tasks look visually stunning but lack reproducibility, making them hard to verify. To cut through the marketing noise and genuinely assess an LLM’s hard capabilities in complex coding and multimodal generation, we need to build our own objective, rigorous evaluation system. This article explains how to run a deep health check on LLMs through two challenge tasks: legacy code refactoring and cross-disciplinary 3D structured generation.

Why Existing LLM Evaluations Fall Short—and How to Move Past Them

Current mainstream LLM evaluation relies on two paths, both with clear pain points.

The first is public benchmarks. These are highly standardized, but because the questions are public, some inevitably end up in the models’ training data. As a result, models score extremely high on specific leaderboards yet underperform on real business code—the classic benchmark-gaming effect.

The second is private demo tasks. Vendors typically showcase cases with strong visual impact and high polish, but these tasks are closed-source: outsiders cannot see the exact prompt details, the system prompts, or whether multiple retries were involved. Such black-box demos cannot serve as a reliable basis for technology selection.

The way out: stop blindly trusting leaderboards and shift to “custom complex task evaluation.” Design highly customized tasks containing multiple hidden checkpoints that force the model to reveal its true logical reasoning, code debugging, and multimodal generation abilities. At the same time, introduce an “AI as reviewer” mechanism to solve the problem of prohibitively expensive manual verification.

Frontend Evaluation: Cross-Disciplinary 3D Structured Generation

Frontend development has long evolved beyond slicing static pages into complex interaction and rendering. To fully stress-test an LLM’s frontend capabilities, don’t merely ask it to generate a beautiful page—examine its 3D modeling and rendering skills under domain-specific knowledge constraints.

1. Task Design Principles: Balance Visuals and Structural Correctness

Today’s AI coding tools are all competing on 3D modeling and animation generation, but most remain at the “it runs” stage. We can design a series of 3D generation tasks that fuse cross-disciplinary knowledge, requiring the model not only to render an image but also to keep physical or biological structures accurate.

• Celestial motion simulation: Ask the model to build a solar system simulation using WebGL or a similar frontend 3D technology. The test points go beyond planet textures and orbital animation: the accuracy of orbital inclination, the proportional relationships between revolution and rotation periods, and the physical computation of lighting and shadows.
• Human anatomy 3D model: Ask the model to build an interactive skeletal and muscular system using the frontend stack. Test points include handling hierarchical occlusion relationships from different viewpoints, binding events so that clicking a specific bone triggers a detail popup, and the accuracy of anatomical terminology.

2. Defining Scoring Criteria

For these tasks, we need a fine-grained checklist:

• Functionality: Is rendering smooth? Are there memory leaks? Do interactions (drag, zoom, click) respond correctly?
• Structural correctness: Does the 3D model’s topology match domain common sense? For example, in the celestial simulation, does any satellite trajectory pass through a planet’s body?
• Code quality: Is the scene graph used properly? How reusable are geometries and materials? Is the shader code efficient?

Backend Evaluation: Refactoring a Deliberately Broken Legacy Codebase

Backend evaluation should not stop at “write a CRUD API from scratch.” The real test is the ability to read, locate, and fix problems in a legacy system. To that end, we need to artificially build a sufficiently complex “spaghetti” codebase.

1. How to Build the Messy Codebase

A spaghetti codebase cannot just be random gibberish—it must carry the complexity and misleading structure of real business code.

• Find a genuinely complex project: On open-source code hosting platforms, select a backend project with reasonable complexity and multiple modules. Choose a branch with many defects (you can locate defect-dense versions through the issue list).
• Extract the baseline code: Download the branch’s full source code as the initial “relatively normal” code baseline.
• Introduce a weak model and misleading prompts: Hand this baseline to a weaker, earlier-generation LLM along with a series of misleading refactoring prompts. For example: “To improve performance, inline all Service layer logic into the Controllers, remove all interface abstractions, replace dependency injection with global variables, and convert all asynchronous operations to synchronous blocking.”
• Generate the ultimate spaghetti code: Guided by incorrect instructions, the weak model will produce a tangled mass of code—heavily coupled, logically contradictory, yet perhaps barely compilable. This is the touchstone for testing your target model.

2. Running the Refactoring Task and What to Test

Hand the generated codebase to the LLM under evaluation with concrete fix and refactoring instructions. Test points include:

• Context understanding: Can the model accurately track variable lifecycles and data flow through long, chaotic context?
• Bug detection: Can it identify logic flaws introduced by the weak model (deadlocks, null pointers, unreleased resources, etc.)?
• Safe, isolated refactoring: When fixing bugs, does it follow high-cohesion, low-coupling principles and redraw module boundaries, rather than patching symptoms one by one?

Automating Evaluation: Using a Strong AI Model as the Reviewer

For the complex 3D generation and refactoring tasks above, relying entirely on manual review is costly and subjective. The recommended approach is automated “LLM reviewing LLM” evaluation.

Choose a model widely recognized for its strong reasoning ability (the reviewer model) and give it a strict system prompt so it plays the role of a senior architect and technical reviewer.

1. How the Reviewer Works

For each evaluation task, prepare a detailed “checklist and test cases” in advance. The reviewer model’s workflow is as follows:

1. Receive input: Get the task description, the original spaghetti code (or 3D generation requirements), and the candidate model’s output code.
2. Static analysis: Check the code under review item by item against the checklist.
3. Logic deduction: For backend code, the reviewer model simulates execution paths through reasoning to determine whether edge cases are handled correctly; for frontend 3D code, it analyzes whether the geometry transform matrices and event logic are sound.
4. Output an evaluation report: The reviewer model produces a structured JSON report containing scores, defect lists, and improvement suggestions.

2. Sample Review Criteria

To keep the reviewer model from drifting away from the evaluation’s intent, define the standards in an engineering-oriented way, for example:

{
  "task_id": "backend_refactor_001",
  "task_description": "Fix and refactor the state pollution caused by global variables",
  "evaluation_criteria": {
    "bug_fix": {
      "weight": 0.5,
      "checks": [
        "Are unreasonable global variables removed?",
        "Is state safe under concurrent access?"
      ]
    },
    "code_quality": {
      "weight": 0.3,
      "checks": [
        "Is proper dependency injection introduced?",
        "Are class and method cyclomatic complexities within a reasonable range?"
      ]
    },
    "syntax_and_runnable": {
      "weight": 0.2,
      "checks": [
        "Is the output code compilable?",
        "Are there obvious syntax errors?"
      ]
    }
  }
}

Conclusion and Engineering Recommendations

In the face of LLM vendors’ marketing push, developers must stay clear-headed. By building 3D generation tasks with cross-disciplinary checkpoints and refactoring tasks derived from real projects, we can effectively push LLMs to the limits of their true capabilities. Meanwhile, using a strong model as an automated reviewer not only reduces evaluation costs but also improves the objectivity and consistency of the assessment.

For real-world enterprise adoption, consider integrating this evaluation system into your internal developer tooling platform. By combining a cloud-native container platform with CI/CD pipelines, you can automate task dispatch, sandboxed code execution, and log monitoring and alerting. With this continuous, objective, hardcore internal evaluation mechanism, enterprises can genuinely select AI coding assistants that match their own business complexity—instead of losing their way in the technology hype.