Evaluating LLMs for 3D Content Creation and Interactive Delivery

    As multimodal and spatial reasoning capabilities advance, evaluating large language models (LLMs) has shifted towards 3D modeling, editable asset generation, and interactive delivery. This article explores best practices for evaluating LLM-driven 3D creation, covering task design, prompt engineering principles, acceptance criteria, technical observations, and engineering pipelines for production environments.

    1. Background: 3D Creation as a New Evaluation Dimension for LLMs

    In the past, LLM capability evaluation primarily focused on code generation, text comprehension, and knowledge-based Q&A. With the development of multimodal and spatial reasoning abilities, the evaluation focus has begun to extend to 3D modeling, scene understanding, editable asset generation, and interactive delivery. Unlike simply generating an image, 3D creation tasks require the model to simultaneously understand semantics, spatial structure, material lighting, engineering organization, and delivery formats. This makes it easier to expose the true boundaries of a model in complex tasks.

    From a general technical perspective, a complete 3D creation delivery includes at least three types of outputs: editable 3D project files, render or preview images, and an optional interactive web page. Evaluation should not only look at the “first-glance effect” but also simultaneously check semantic restoration, spatial rationality, engineering editability, interaction integrity, and performance accessibility. Otherwise, the model might generate a “black-box asset” that looks good but cannot be edited secondarily, or it might omit key subject identification symbols, leading to an unqualified delivery.

    2. Task Design and Prompting Principles

    2.1 Task Abstraction

    The task can be abstracted as follows: generate a voxel-style miniature landscape of a commercial block, featuring a two-story detective agency and a one-story coffee shop, including a side staircase and voxel character models at the entrance. The deliverables include an editable 3D project, render images, and an interactive web page. This task covers architectural structure, functional zoning, character abstraction, voxel language, and delivery formats, making it suitable as a general evaluation sample.

    2.2 Bare Prompts vs. Supplementary Prompts

    A “bare prompt” refers to using only natural language to describe the task without attaching additional skills or injecting industry templates, allowing observation of the model’s default capabilities. Supplementary prompts only constrain the delivery format, such as “provide an editable project file” or “generate an interactive page viewable in a browser.” This approach helps distinguish the model’s inherent capabilities from enhancements provided by external toolchains.

    2.3 Prompt Template Recommendations

    A robust prompt should contain seven types of information: theme, style, spatial structure, key elements, deliverables, acceptance criteria, and prohibited items. For example:

    • Theme: Voxel-style miniature landscape of a commercial block.
    • Style: Low-poly voxel block language with a unified color palette.
    • Structure: Two-story agency, one-story coffee shop, side staircase, entrance character.
    • Key elements: The main signboard identification must appear; character hairstyles must be recognizable.
    • Deliverables: Editable 3D project, render images, interactive web page.
    • Acceptance criteria: Clear object naming, material separation, accurate annotation coordinates.
    • Prohibited items: Do not generate copyrighted brand logos or specific IP images.

    A common issue in practical tests is that the model can restore floors, stairs, shop zoning, and characters but omits high-recognition elements like the “main signboard.” This indicates that key elements must be explicitly listed and distinguished as “must include” or “omittable.” Semantic anchors such as signboards, entrances, and main facades should be prioritized in the prompt.

    3. Deliverables and Acceptance Criteria

    It is recommended to break down the acceptance process into seven dimensions:

    1. Semantic Restoration: Whether the main building, floor relationships, stairs, shops, characters, and signboard identifications are complete.
    2. Spatial Rationality: Whether proportions, occlusions, circulation, and floor connections align with common sense.
    3. Voxel Consistency: Whether grid alignment, palette limitations, and block granularity are uniform.
    4. Engineering Editability: Object naming, hierarchy organization, material separation, and whether secondary modification is possible.
    5. Render Quality: Composition, lighting, shadows, filters, and output resolution.
    6. Interaction Integrity: Drag-and-drop, auto-rotation, scene annotation, view focus, asset download, image switching, and first-time user guidance.
    7. Performance and Accessibility: First-screen loading, mobile adaptation, weak network degradation, and static fallback when 3D capabilities are unavailable.

    The acceptance checklist can be described using the following pseudo-code:

    1. {
    2. "task": "voxel_scene_delivery",
    3. "deliverables": ["editable_3d_project", "render_image", "interactive_web_page"],
    4. "checks": {
    5. "semantic": ["main_building", "floor_relation", "stair", "shop_zone", "signboard", "character"],
    6. "spatial": ["scale_ratio", "occlusion", "walk_path"],
    7. "voxel": ["grid_alignment", "palette_limit", "block_granularity"],
    8. "editable": ["object_naming", "hierarchy", "material_separation"],
    9. "interaction": ["drag_rotate", "auto_rotate", "annotation", "focus_view", "download_asset", "first_guide"],
    10. "performance": ["first_screen_budget", "mobile_layout", "fallback_static"]
    11. },
    12. "scoring": {
    13. "critical_missing": "Overall downgrade when the signboard is missing",
    14. "interaction_bonus": "Interaction enhancements can add points but must not mask semantic missing"
    15. }
    16. }

    When scoring, adhere to the “critical element one-vote downgrade” principle. No matter how many interactive features are present, they cannot replace the semantic completeness of the main subject. Conversely, interaction enhancements can serve as bonus items but should not become the primary selling point.

    4. Technical Interpretation of Empirical Observations

    4.1 Semantic Restoration and Fine-Grained Control

    From general empirical observations, models can usually restore the main building, floor zoning, side staircases, shop entrances, and voxel characters, indicating significant progress in scene-level semantic understanding and spatial layout. However, issues like missing main signboards and abstract character hairstyles reflect that the model’s control over “high-recognition symbols” and “fine-grained geometric semantic binding” remains unstable. A signboard is not merely decoration; it is the identity of the scene. A character’s hairstyle is not just geometric detail; it is a clue for character recognition. Such elements should be incorporated into key acceptance items during evaluation.

    4.2 “Over-Delivery” and Risks in Interactive Web Pages

    When the prompt only requires the output to be “openable in a web page,” the model might automatically supplement features like drag-to-rotate, auto-rotation, scene annotation, view focus, asset download, image switching, and first-time operation guidance. These interactions are indeed valuable for scenarios like real estate sandboxes, city modeling, and smart park visualization, as annotations and focus help users quickly locate objects. However, risks must be noted: auto-generated interactive pages may suffer from excessive dependencies, slow first-screen loading, misaligned mobile layouts, and drifting annotation coordinates. If the web page model and the project file are not from the same source, annotation coordinates may fail as versions change. It is recommended to use a unified scene description file to constrain assets, annotations, and perspectives.

    4.3 Delivery Consistency

    The project file, render images, and web page should originate from the same scene version. Object naming, materials, camera perspectives, and annotation coordinates require unified metadata. Otherwise, a tripartite inconsistency will occur where “the render looks good, the project is uneditable, and the web annotations are misaligned.” During evaluation, version numbers and asset hashes should be checked to ensure that deliverables share the same source.

    5. Engineering Implementation Suggestions

    5.1 Pipeline Design

    A “specification-first” pipeline is recommended: Prompt Parsing -> Scene Spec Generation -> Asset Generation -> Rendering -> Web Packaging -> Automated Acceptance -> Human Review -> Publishing. The pseudo-code is as follows:

    1. parse_prompt -> scene_spec
    2. scene_spec -> generate_3d_asset
    3. scene_spec -> render_preview
    4. scene_spec + asset -> build_interactive_page
    5. asset + page + preview -> run_checks
    6. run_checks -> human_review -> publish

    5.2 Asset Governance and Cloud Hosting

    Generated assets should enter object storage and be archived by task ID, version number, and scene specification. Rendering tasks can be executed asynchronously in a container platform. Log services should record the generation and acceptance processes, while monitoring and alerting track failure rates, time consumption, and quota usage. In a general cloud environment, general-purpose capabilities such as object storage, container platforms, log services, and monitoring alerts provided by Baidu AI Cloud can be used to host the above processes, but specific API assumptions should not be bound. When quotas are limited, task prioritization, cache reuse, failure retries, and budget alerts should be employed to avoid repeatedly generating high-cost scenes.

    5.3 Security and Compliance

    3D generation also requires copyright and content governance. Prompts should prohibit the generation of copyrighted brand logos, specific IP images, and recognizable characters. Characters should be abstracted. Output assets should undergo content moderation, and project files should be accompanied by license descriptions. For commercial projects, it is recommended to establish a “prohibited elements vocabulary” and a “manual review checklist.”

    6. Best Practices Checklist

    • Explicitly list “must-include” semantic anchors in the prompt, especially signboards, entrances, and main facades.
    • Define the scene specification JSON before generating assets and pages.
    • Use a unified coordinate system and version number for project files, render images, and web annotations.
    • Provide a static image fallback for interactive pages to ensure accessibility in weak networks and environments without 3D capabilities.
    • Look at key semantics first during acceptance, followed by interaction bonuses.
    • Set budget alerts and caching strategies for quotas and costs.
    • Establish a reproducible evaluation set with fixed prompts, random seeds, and scoring rubrics.

    7. Conclusion

    General LLMs in 3D creation have transitioned from “generating a single image” to combined deliveries of “editable assets + render images + interactive pages.” Significant progress has been made in spatial understanding, voxel expression, and web interaction packaging. However, missing subject identifiers, fine-grained character control, delivery consistency, and performance governance remain the primary bottlenecks. For developers and enterprises, what matters more than a single visual impression is establishing a reusable evaluation methodology, acceptance checklists, and an engineering pipeline. Only by incorporating semantic completeness, engineering editability, interaction usability, and cost controllability into the acceptance process can 3D generation capabilities truly enter production environments.