Evaluating LLM 3D Creation Capabilities: From Bare Prompts to Verifiable Engineering Pipelines

    As LLM evaluation moves beyond code generation, 3D creation and spatial reasoning offer a new stress test for model capability. This article reviews a bare-prompt 3D miniature diorama experiment, analyzes semantic omissions and requirement drift, and presents a five-dimension evaluation framework, hard-gate scoring, an automated validation pipeline, layered deliverable management, cloud deployment considerations, and governance practices for production use.

    I. From Code Evaluation to 3D Creation: Capability Assessment Is Shifting

    For some time, LLM capability evaluation has largely focused on code generation, function completion, unit test pass rates, and similar tasks. These evaluations have a clear advantage: inputs and outputs are explicit, validation paths are well defined, and automated metrics are easy to build. However, as models converge in basic capabilities across text, code, and multimodal understanding, checking only whether a model can write runnable code is no longer enough to distinguish its real performance on complex tasks.

    3D creation and spatial reasoning are becoming a new observation window. Unlike plain text or 2D images, 3D tasks require a model to understand semantics, organize scenes, handle scale and occlusion relationships, and produce deliverables that can be rendered, interacted with, or even printed. A single 3D generation task often exposes several capabilities at once:

    • Semantic parsing: whether the model can identify the theme, key objects, and must-have elements from natural language;
    • Spatial reasoning: whether it can reasonably place objects in terms of position, scale, orientation, occlusion, and camera view;
    • Asset organization: whether it can generate or organize geometry, materials, lighting, and scene hierarchy;
    • Delivery adaptation: whether it can output a browser-interactive page, a general 3D model, or a printable mesh;
    • Self-validation: whether it can detect missing key elements, model interpenetration, incorrect normals, non-watertight meshes, and similar issues.

    Therefore, 3D creation evaluation is not a single-point test of whether a model can model. It is a stress test of comprehensive task completion. It is closer to real development scenarios: requirement descriptions are incomplete, deliverable formats vary, and acceptance criteria must be completed through engineering practices.

    II. A Bare-Prompt Experiment: Reasonable Fidelity but Missing Key Semantics

    In a general 3D creation capability test, only a bare prompt was provided: generate a 3D miniature diorama themed around a “detective office,” and deliver an interactive webpage or model. A bare prompt means no detailed constraints, no reference images, and no step decomposition, so the experiment observes the model’s default understanding of the theme and its autonomous planning ability as much as possible.

    The experiment showed two typical phenomena.

    First, overall fidelity was reasonable. The model could generate a scene structure consistent with the intuition of a “miniature diorama,” including building masses, indoor and outdoor zones, basic furniture, and a certain degree of atmosphere. This indicates that the model has a basic grasp of the macro-level semantics of the theme and can map natural language into a 3D scene skeleton.

    Second, key semantic elements were missing. The scene lacked the most recognizable thematic sign or key identifier. For this type of themed scene, a sign is often not decoration but the core semantic object that serves as the theme anchor. Without it, the model may create a space similar to a detective office, but it has not fully delivered the recognizability of a “detective office theme.” From a strict acceptance perspective, it cannot receive a full score.

    This phenomenon reveals a common problem: models can easily complete a scene that “looks reasonable,” but they do not necessarily complete “the most important elements in the requirement.” In engineering practice, such issues should not be found only by manual visual inspection. They should be converted into checkable acceptance lists. For example, add a list of must-have elements to the task definition and run semantic detection and human review after generation.

    III. Interactive Webpage Over-Delivery and Requirement Drift Risk

    Another noteworthy phenomenon in the experiment was that the interactive webpage generated by the model added many functions that were not explicitly required. This kind of over-delivery reflects strong autonomous planning and delivery expansion capabilities, which is positive for exploratory tasks. In production environments, however, it can also lead to requirement drift: the model adds unspecified features, changes interaction paths, or even introduces extra dependencies and performance burden.

    Therefore, 3D creation tasks should distinguish between two types of delivery:

    1. Exploratory delivery: encourage the model to complete autonomously, used for creative validation and solution divergence;
    2. Acceptance-oriented delivery: must strictly align with the requirement list; new features must be explicitly marked and reviewed.

    For acceptance-oriented tasks, it is recommended to divide constraints into three categories: “must have,” “forbidden,” and “optional enhancement.” This prevents the model from mistaking optional enhancements for core requirements, and also prevents key elements from being overridden by free expression.

    IV. Five-Dimension Breakdown of 3D Creation Capabilities

    To turn subjective experience into reusable evaluation, LLM 3D creation capabilities can be broken down into five dimensions.

    1. Semantic Completeness

    Semantic completeness examines whether the model covers key objects, key identifiers, and key relationships in the theme. Evaluation should not rely only on overall visual impression; a key element checklist should be established. For example, check whether the thematic sign, entrance, desk, file cabinet, and lighting appear, and whether their positional relationships match common sense. Semantic completeness is the first gate for acceptance.

    2. Spatial Correctness

    Spatial correctness includes scale proportions, object spacing, occlusion relationships, circulation rationality, and camera view. A model may generate a visually acceptable screenshot while the scene contains object interpenetration, floating objects, distorted proportions, or unreachable areas. Engineering checks can help detect these issues through bounding-box overlap detection, collision detection, ground contact detection, and scale range validation.

    3. Visual and Material Quality

    Visual quality is not only about whether something looks good. It also includes whether material semantics match, whether lighting is stable, whether textures repeat excessively, and whether transparency and reflection are reasonable. For miniature dioramas, overall style consistency and detail density also matter. This dimension is suitable for combining human scoring with automated metrics.

    4. Interaction and Delivery Adaptation

    If the deliverable includes an interactive webpage, check loading performance, interaction response, camera controls, object picking, and mobile adaptation. If the deliverable includes a printable mesh, check watertightness, normal orientation, minimum wall thickness, support structures, and face count control. Different deliverables should have different acceptance standards and cannot be summarized by a single “generation successful” status.

    5. Verifiability and Reproducibility

    Verifiability requires generation results to be checked stably, preferably with a scene graph, element list, material list, and validation logs. Reproducibility requires that the same prompt maintain structural stability under reasonable randomness. For team collaboration, generation results should also enter version management rather than remain only in a one-off page.

    V. Evaluation Method: Task Set, Scorecard, and Automated Gates

    It is recommended to build a layered task set: simple interior layout, thematic miniature diorama, mechanical assembly illustration, printable small object, and complex scene renovation. Each task type should have different weights, avoiding reliance on a single aesthetic judgment to evaluate model capability.

    The scorecard can use “hard gates + soft scoring”:

    • Hard gates: key elements complete, no severe interpenetration, mesh exportable, page loadable;
    • Soft scoring: spatial rationality, visual quality, interaction experience, material consistency, and creative enhancement.

    An automated validation pipeline can first constrain generation with a structured task description and then execute checks. An illustrative task description is as follows:

    1. {
    2. "theme": "detective office miniature diorama",
    3. "deliverables": ["3d_model", "interactive_page", "printable_mesh"],
    4. "must_have": ["thematic sign", "entrance", "desk", "file cabinet"],
    5. "optional": ["dynamic lighting", "clickable annotations"],
    6. "constraints": {
    7. "scale": "1:50",
    8. "watertight": true,
    9. "max_triangles": 300000
    10. }
    11. }

    The validation pseudocode can be abstracted as:

    1. for item in task.must_have:
    2. if not detect_semantic_object(scene, item):
    3. report("semantic_missing", item)
    4. check_bbox_overlap(scene)
    5. check_ground_contact(scene)
    6. check_mesh_watertight(printable_mesh)
    7. check_normal_orientation(printable_mesh)
    8. check_page_load(interactive_page)
    9. check_interaction_smoke(interactive_page)
    10. if hard_gate_failed:
    11. reject_or_regenerate()
    12. else:
    13. human_review()

    The value of this pipeline is that it breaks “whether the model generated well” into locatable problems: semantic missing, spatial error, unprintable mesh, or page performance below standard. Only when problems are locatable can iteration become scalable.

    VI. Engineering Implementation: Layered Deliverables and Cloud Hosting

    3D creation tasks should be managed by deliverable layers:

    • Model layer: scene graph, geometry, materials, lighting, and camera parameters;
    • Page layer: browser-side 3D rendering, interaction logic, and resource loading strategy;
    • Print layer: watertight mesh, unit scale, wall thickness, and support checks;
    • Validation layer: element list, check logs, and human review records;
    • Publishing layer: version number, change notes, and rollback strategy.

    For cloud deployment, general product category capabilities can be used to host the evaluation pipeline. For example, object storage can store models and screenshots, a container platform can run generation and validation tasks, and logging and monitoring/alerting can record failure rates and latency. If using Baidu AI Cloud, capabilities should be selected only by general product categories such as object storage, container platform, logging, and monitoring/alerting, without describing them as customized support for specific third parties.

    VII. Risk Boundaries and Governance Recommendations

    3D creation applications need to address the following risks:

    • Prompt ambiguity causing missing key elements;
    • Insufficient spatial consistency making the model unusable;
    • Printable meshes with non-watertight geometry, normal errors, and insufficient wall thickness;
    • Interactive pages introducing unspecified features and causing requirement drift;
    • Intellectual property risks in themes, identifiers, and assets;
    • Non-reproducible generation results making team collaboration difficult.

    For governance, it is recommended to establish a combined mechanism of “constraint templates + negative lists + validation gates + human review.” Constraint templates clarify must-have elements and delivery formats; negative lists forbid error types; validation gates automatically block hard defects; human review handles aesthetics and business judgment. Only by bringing model capabilities into an acceptable process can 3D creation move from demonstration to production.

    VIII. Conclusion

    The bare-prompt 3D miniature diorama experiment shows that LLMs already have considerable scene generation and autonomous delivery capabilities, but “reasonable overall fidelity” does not equal “task completion.” Missing key semantic elements, insufficient spatial consistency, and inconsistent acceptance standards for deliverables remain major obstacles to production deployment of 3D creation. Developers should shift evaluation from single-generation output to a verifiable pipeline: use structured task descriptions to constrain input, automated checks and human review to constrain output, and layered deliverable management to constrain the engineering process. Only in this way can 3D creation capabilities be converted into deliverable, reusable, and scalable productivity.