Evaluating LLM 3D Generation Capabilities and Engineering Practices

    This article explores how to systematically evaluate the 3D creation and spatial reasoning capabilities of Large Language Models (LLMs). Moving beyond simple visual impressions, it introduces a multi-dimensional evaluation framework, a structured "delivery contract" prompt design, and a checklist-based validation mechanism for 3D assets. It also discusses the boundaries of interactive web pages generated by models and provides engineering methods for conducting reproducible 3D generation experiments under API quota constraints.

    I. Problem Background: 3D Creation as a New Frontier in Model Evaluation

    In the past, LLM capability evaluations primarily focused on text understanding, code generation, and tool calling. With the growing demand for 3D content production, more technical teams are paying attention to model performance in 3D creation, spatial structure understanding, and interactive deliverable generation. Unlike pure text or code, 3D creation tasks simultaneously evaluate theme understanding, spatial layout, geometric modeling, material style, asset organization, and final delivery format, making the evaluation significantly more complex.

    A typical internal experiment involved providing a tested model with only a “bare prompt,” requiring it to generate a voxel miniature landscape themed around a “detective agency” and deliver an editable 3D engineering file and a rendering image. The experiment showed that the model could relatively completely reconstruct the main architectural structure, such as the second-floor agency, first-floor cafe, and external stairs, but omitted the very crucial sign in the theme scene. Meanwhile, under the instruction to simply generate an HTML page to open the file, the model proactively added features like scene annotations, camera focus, auto-rotation, drag instructions, model download, and image switching. This phenomenon indicates that evaluating 3D generation capabilities cannot solely rely on “whether the model built it,” but must also assess “whether key semantics are complete,” “whether the deliverable is usable,” and “whether the interaction layer is over-engineered or insufficient.”

    Therefore, rather than conducting a review around a specific product, this article abstracts the problem into a general technical topic: How can we systematically evaluate the 3D creation and spatial reasoning capabilities of LLMs? How should we design prompts and acceptance criteria? And how can we transform a single experiment into a reproducible, engineering-ready production workflow?

    II. Evaluation Dimensions: From “Looks Right” to “Usable”

    Evaluating 3D generation tasks should rely on a multi-dimensional scoring rubric to avoid subjective impressions. It is recommended to cover at least the following six dimensions:

    1. Theme Fidelity: Does it include the key buildings, signs, characters, or atmosphere required by the theme? Taking the detective agency theme as an example, the main building, floor functional zoning, external stairs, and sign are high-weight semantic elements. A missing sign should not be offset by the reconstruction of the main building, as it is the core anchor for theme recognition.

    2. Spatial Structure Rationality: Do floor relationships, entrance locations, stair connections, and indoor-outdoor transitions conform to common sense? The voxel style can be abstract, but there should be no structural breaks, floating elements, or unexplainable spatial relationships.

    3. Style Consistency: Is the voxel block language unified? Are materials, colors, and proportions coordinated? Style consistency is more important than single-point details because 3D miniature landscapes are typically used for display and visualization scenarios.

    4. Editability: Does the delivered 3D engineering file retain object hierarchy, naming, materials, and basic transform information? Editability determines whether secondary development is possible and is a key metric for engineering delivery.

    5. Interactive Usability: If an HTML page is delivered, evaluate whether features like dragging, auto-rotation, scene annotations, camera focus, and download entries are usable, user-friendly, and aligned with task objectives.

    6. Robustness and Reproducibility: Is the output stable under identical or similar prompts? Is the model prone to omitting constraints? Can it self-correct after supplementary prompts? When API quotas are limited, prioritize designing high-information-density prompts to reduce ineffective retries.

    Weighted scoring is recommended. For example, Theme Fidelity 30%, Spatial Structure 20%, Style Consistency 15%, Editability 15%, Interactive Usability 10%, and Robustness 10%. Weights can be adjusted based on business scenarios but should be fixed before the experiment to avoid post-hoc rationalization.

    III. Bare Prompt Design: Reducing Ambiguity with a “Delivery Contract”

    A bare prompt does not mean a random prompt. For 3D generation tasks, the prompt should ideally contain five types of information: theme, style, must-have elements, deliverables, and constraints. This can be expressed in a structured format:

    1. {
    2. "task": "Voxel miniature landscape",
    3. "theme": "Detective agency",
    4. "style": "Voxel block language, coordinated proportions, unified materials",
    5. "must_have": ["Second-floor agency", "First-floor cafe", "External stairs", "Thematic sign"],
    6. "deliverables": ["Editable 3D engineering file", "PNG rendering", "HTML interactive page"],
    7. "interaction": ["Drag to rotate", "Auto-rotation", "Scene annotation", "Camera focus"],
    8. "constraints": ["No irrelevant buildings", "Key signs must not be omitted", "Page must open offline"]
    9. }

    The value of this “delivery contract” lies in three aspects: First, it makes implicit needs explicit, reducing deviations caused by the model’s free play. Second, it provides a checklist for subsequent validation. Third, it facilitates multi-round experiment comparisons. In the original experiment, the model’s omission of the sign was likely related to the prompt not explicitly emphasizing the sign as a must-have element; the extra web page features indicated that the model proactively expanded on “opening the file in HTML.” Prompt design should strike a balance between “providing sufficient constraints” and “preserving creativity.”

    IV. Deliverable Validation: Establishing a Checklist Mechanism

    3D generation results cannot be judged solely by the final screenshot. It is recommended to perform a checklist-based validation for deliverables:

    • Engineering File Check: Are objects layered? Is naming readable? Are materials retained? Are there uneditable merged meshes? Are scale and origin reasonable?
    • Rendering Check: Is the theme recognizable? Are key elements complete? Is the perspective reasonable? Is there clipping or obvious noise?
    • HTML Page Check: Can the page open directly? Can the model load? Are drag and rotation smooth? Do annotations block the view? Is the download entry valid? Is it usable on mobile devices?
    • Semantic Check: Do key signs, character roles, and functional zones align with the theme? Character details can be abstract but must not affect recognition.
    • Security and Compliance Check: Does the page introduce unnecessary external dependencies? Are there script execution risks? Is the resource size controllable?

    In the experimental case, the main building was well reconstructed, but the crucial sign was missing, and the abstract character hairstyles made recognition difficult, indicating that “semantic completeness” should be an independent validation item. The interactive page added scene annotations and camera focus, which has reference value for scenarios like smart visualization, real estate sandboxes, and city modeling. However, if the task only requires static display, over-supplementing features may increase maintenance costs. Therefore, validation should distinguish between “essential features” and “bonus features.”

    V. Incremental Value and Boundaries of Interactive Web Pages

    Under the weak constraint of simply “opening the file in HTML,” the model generated a page featuring drag, auto-rotation, scene annotations, camera focus, model download, and image switching. This phenomenon demonstrates the LLM’s potential to encapsulate 3D assets into lightweight interactive deliverables. For presentation projects, interactive web pages can lower the viewing barrier, allowing non-professional users to browse models without installing modeling software.

    However, the interaction layer also has boundaries. First, the more features, the higher the page size and compatibility risks. Second, annotations and focus rely on object naming within the model; if engineering file naming is chaotic, interactive functions will fail. Third, experience features like auto-rotation and drag instructions should match business goals and should not sacrifice loading performance just to “look rich.” Fourth, download entries should clearly specify the asset list and version information to avoid delivery confusion.

    The engineering recommendation is to treat the interactive web page as a “display shell” and the 3D engineering files and renderings as “source assets,” managed through a unified resource directory and manifest file. The page should only handle loading, interaction, and display, without bearing non-traceable business logic.

    VI. Reproducible Experiment Methods Under Quota Constraints

    Actual evaluations are often constrained by API call quotas. To improve experiment efficiency, it is recommended to fix the experimental protocol: fix the prompt template, fix the validation scoring rubric, fix the deliverable naming rules, and record the differences in each output. If quotas are limited, test different themes and complexities in batches rather than repeatedly tweaking the same theme. Each experiment should at least record: prompt version, deliverable list, missing key elements, interactive features, manual score, and notes.

    For team collaboration, the scoring rubric can be implemented in a spreadsheet or a simple script to form a trackable record. This ensures consistent conclusions even across multi-day experiments, avoiding “impression-based evaluation.”

    VII. Conclusion

    Evaluating LLM 3D creation capabilities should shift from singular visual impressions to multi-dimensional engineering validation. Bare prompt experiments can expose the model’s true performance in key semantics, spatial structure, and delivery contracts. Structured prompts and checklist-based validation can significantly enhance reproducibility. Interactive web pages are a crucial increment but require clear boundaries and resource management. For developers, a more reliable approach is to integrate 3D generation into a workflow of “prompt contract — asset validation — interaction encapsulation — version logging,” using engineering methods rather than one-off experiences to judge capabilities.