How to Evaluate the 3D Creation Capabilities of Large Language Models

    As large language models move beyond code generation, 3D creation—turning a natural-language prompt into editable models, rendered images, and interactive web pages—has become a key evaluation frontier. Using a voxel-style miniature street scene as a sample task, this article lays out a practical evaluation methodology. It shows how to break 3D creation into measurable dimensions, design tasks and layered prompts, and verify three types of deliverables: editable 3D project files, rendered images, and interactive web pages. It also covers a weighted scoring system, automated checks with parsing scripts and browser automation, common failure patterns and how to fix them, and how cloud infrastructure can turn a one-off test into a repeatable, production-grade evaluation pipeline.

    1. Introduction: The Evaluation Focus Is Shifting from Writing Code to Building Scenes

    For a while now, code generation has been the main track in large-model evaluation. As models have become more fluent at producing structured, executable output, the focus is gradually moving toward more comprehensive tasks—spatial creation, represented by 3D modeling and 3D interaction.

    This shift is not accidental. There are three reasons:

    • The tasks are closer to real business needs: Scenarios such as smart-campus sand tables, property visualizations, industrial digital twins, and product 3D displays often start from a single natural-language request and require the model to progressively produce editable 3D models, readable rendered images, and interactive display pages.
    • They demand more of the model: 3D creation simultaneously tests spatial-structure understanding, semantic detail recovery, constraint following, and front-end code generation. Being “good at writing code” alone is rarely enough to score well.
    • The results are easier to verify: 3D outputs can be viewed, edited, and run directly, so evaluation conclusions are more objective than those for open-ended text generation.

    Many cloud platforms now expose these capabilities to developers as model services. For example, with model services from cloud providers such as Baidu AI Cloud, developers can invoke models through a unified interface and pay by usage, while combining them with foundational cloud products—object storage, logging, monitoring and alerting—to build evaluation and validation environments at low cost.

    Yet there is still a meaningful gap between “can generate 3D content” and “can reliably deliver usable 3D content.” Using a typical voxel miniature-street-scene task as a sample, this article walks through a method for evaluating 3D creation capabilities.

    2. Breaking Down 3D Creation Capabilities

    2.1 From a Single Output to “Model + Images + Webpage”

    3D creation rarely delivers just one file. A complete deliverable usually consists of three kinds of output: an editable 3D project file, a rendered image for presentation, and an interactive web page. Each output tests a different capability:

    • 3D project file: Tests spatial-structure understanding and structured output—whether the model can decompose the buildings, streets, and characters described in text into sensible objects, hierarchies, and materials.
    • Rendered image: Tests scene organization, material pairing, and visual expression.
    • Interactive webpage: Tests front-end code generation, use of 3D rendering techniques, and interaction design.

    2.2 Key Evaluation Dimensions

    3D creation can be broken down into six dimensions:

    1. Theme fidelity and element completeness: Can the model identify and implement the “must-have elements” in the requirements—without omitting iconic elements or carelessly adding content beyond the requirements?
    2. Spatial-structure plausibility: Are the relative positions, connection relationships, and proportions of elements self-consistent—for example, does a staircase actually connect two floors, and are the door and window orientations reasonable?
    3. Geometry and style quality: Is the voxel style consistent? Are there geometric problems such as broken faces, interpenetration, or floating geometry? Do materials and lighting work together?
    4. Project editability: Are objects grouped and named, are materials well-organized, and is the scene easy to modify later?
    5. Interaction completeness: Does the web page have basic capabilities such as drag-to-rotate, zoom, auto-rotation, annotations, and view focusing, and are there performance or compatibility issues?
    6. Constraint following and stability: Does it satisfy all constraints, and are results consistent and stable across multiple generations?

    3. Test Design: Task Setup and Prompt Constraints

    3.1 Choosing Themes with Clear Spatial Structure

    Evaluation themes should have clear, verifiable spatial structure. An ideal task includes at least the following elements:

    • Multi-story building structure: For example, a two-story building with a street-facing shop on the first floor and office space on the second.
    • Connection and entrance relationships: For example, an outdoor staircase connecting the first and second floors, with independent entrances for the shop and the office.
    • Iconic exterior elements: For example, a shop sign hanging on the facade, serving as the most recognizable feature.
    • Character figures: Used to check scale and character fidelity.

    A theme that meets these conditions lets evaluators quickly judge whether the model truly understands the spatial relationships, rather than merely “assembling a decent-looking house.” One caution: such themes are often tied to specific films or animated works. For commercial evaluations, use scene assets you own or are licensed to use, to avoid intellectual-property risk.

    3.2 Prompts: Layered Expression, Clear Constraints

    Below is an example of a well-structured prompt that can be used directly for evaluation:

    1. Using mainstream open-source 3D creation software, create a voxel-style
    2. miniature street scene on the theme of a "retro detective agency."
    3. [Scene setup]
    4. A two-story corner building with a floor height of about 3 meters,
    5. proportioned like a 1:50 miniature sand table.
    6. [Must-have elements]
    7. 1. Second-floor agency: street-facing windows, a glass door, and a long
    8. horizontal sign hanging on the wall beside the door reading
    9. "Detective Agency";
    10. 2. First-floor cafe: a storefront along the street, floor-to-ceiling windows,
    11. outdoor tables and chairs, and an awning;
    12. 3. Outdoor wrought-iron staircase: running from the first to the second floor
    13. along the side wall, with railings;
    14. 4. Character figures: two people standing at the agency door, each about
    15. 1/6 of the floor height.
    16. [Style requirements]
    17. Model everything with voxel cubes at a uniform voxel granularity; free-form
    18. curved surfaces are not allowed; keep material colors restrained to bring out
    19. the layers of the street scene.
    20. [Deliverables]
    21. 1. An editable 3D project file, with objects grouped and named as
    22. "building/stairs/sign/cafe/characters";
    23. 2. At least 2 rendered images, from a 45-degree overview and a front
    24. elevation;
    25. 3. An interactive web page that opens directly in a browser and supports
    26. drag-to-rotate, wheel zoom, auto-rotation, scene annotations, and view
    27. focusing.
    28. [Self-check requirement]
    29. Before delivery, check each "must-have element" one by one and state the
    30. object name corresponding to each element.

    This prompt expresses requirements in layers—“scene setup + must-have elements + style requirements + deliverables + self-check requirement”—balancing the clarity of constraints with the brevity of the wording. The “self-check requirement” is especially important: it prompts the model to compare its output against the requirements item by item after generation, which significantly reduces the chance of omitting key elements.

    3.3 Testing Native Spatial Reasoning with Zero-Shot Prompts

    When the prompt explicitly lists all elements, you are mostly testing “constraint following.” To assess a model’s native spatial reasoning, run a control experiment: give only the theme, style, and delivery requirements, without enumerating specific elements. For example:

    1. Using mainstream open-source 3D creation software, create a voxel-style
    2. miniature street scene on the theme of a "detective agency," and deliver an
    3. editable 3D project file and rendered images.

    Whether the model proactively thinks of typical elements—such as a sign, an outdoor staircase, and street-facing shops—and which elements it treats as optional says more about its spatial imagination and common-sense understanding.

    When evaluating, you can form a “task × prompt” matrix and combine it with multiple sampling runs to observe consistency and stability:

    Task ID Scene theme Zero-shot prompt Constraint prompt Sampling runs
    T1 Multi-story corner shop ✓ ✓ 3
    T2 Urban block sand table ✓ ✓ 3
    T3 Industrial equipment or mechanical structure ✓ ✓ 3

    4. Deliverable Acceptance: Three Types of Outputs, Three Sets of Criteria

    4.1 3D Project Files: Look at Structure, and More at Editability

    Acceptance of a project file should not stop at “does it look right”—it must also consider whether the file can be modified and used downstream. Focus on five things:

    • Grouping and naming: Are objects grouped by function and named semantically (e.g., “second-floor agency/windows,” “outdoor stairs/railing”) rather than with many default names? Semantic names are the foundation for scene annotation, interaction targeting, and team collaboration.
    • Topology and geometry: Are there broken faces, non-manifold edges, internal interpenetration, or obvious floating geometry? Is the voxel granularity uniform?
    • Scale and units: Are the proportions among characters, buildings, and streets coherent? Are the scene units clearly defined?
    • Materials and hierarchy: Are material names well-organized and easy to replace in bulk?
    • Exchange and print compatibility: Can it be exported reliably to a common 3D exchange format? If 3D printing is needed, is the model closed (watertight) and are the wall thicknesses sufficient?

    4.2 Rendered Images: Composition, Lighting, and Detail Consistency

    Rendered images should be judged on composition, lighting, materials, and details. One aspect that is often overlooked is character appearance: in low-granularity voxel models, features such as facial features and hairstyles are easily “abstracted” and become nearly unrecognizable from a top-down view. When evaluating character fidelity, judge it by combining multiple viewpoints rather than relying on a single 45-degree overview.

    4.3 Interactive Webpages: From “It Rotates” to “It Works Well”

    Acceptance of an interactive webpage can proceed on three levels: functionality, engineering quality, and accessibility.

    Functional level:

    • Basic interaction: Are drag-to-rotate and wheel or pinch zoom smooth? Is the auto-rotation speed appropriate?
    • Scene annotations: Are annotation points correctly bound to their corresponding objects, and do their positions stay stable as the view changes?
    • View focusing: After clicking a list item, does the camera smoothly move to the target object, and is the target selected correctly?
    • Deliverable integration: Switching between rendered images, model download, and first-open usage guidance.

    Engineering-quality level:

    • The size of model assets and the loading strategy; time to first-screen load.
    • Display quality across different resolutions and devices.
    • Fallback prompts when asset loading fails or the device does not support hardware acceleration.

    Accessibility level:

    • Does the canvas provide a text description that assistive technologies can read?
    • Can page elements be operated with the keyboard?
    • Does the contrast between text and background meet accessibility standards?

    A phenomenon worth noting: models sometimes “proactively add” features not mentioned in the requirements, such as scene annotations, view focusing, and deliverable download. If these features are implemented correctly and do not affect performance, they improve the usefulness of the deliverable; if annotation points don’t line up with objects or the focus target is off, they become misleading. During evaluation, distinguish “valuable extensions” from “unverified inventions” and score them separately.

    5. Building a Quantifiable Scoring System

    5.1 Dimensions and Weights

    Evaluation is best done with a 100-point weighted score, where the weights can be adjusted to suit business priorities:

    Dimension Weight Key scoring points Suggested verification
    Theme fidelity and element completeness 20% Whether must-have elements are omitted or over-added Check against the element list item by item
    Spatial structure and proportions 20% Whether positions and connections are reasonable and proportions coherent Manual inspection + bounding-box statistics
    Geometry and visual quality 15% Consistent voxel style, no interpenetration, broken faces, or floating geometry Multi-angle screenshots + rendered-image checks
    Project editability 10% Grouping and naming, material conventions, exportability Structured parsing script
    Interaction and performance 20% Basic interaction, annotations, focusing, download, loading performance Browser automation tests
    Constraint following and stability 15% Whether all constraints are met and results are consistent across runs Comparison across multiple samples

    5.2 Graded Scoring: Turning “Feel” into “Standards”

    Each dimension uses a 5-level scale. For example, “theme fidelity and element completeness” could be defined as:

    • 5: All must-have elements are present and well positioned, with no obvious extra elements.
    • 4: Elements are complete, with minor deviations in a few details.
    • 3: One non-core element is missing, or there are minor positional errors.
    • 2: An iconic element is missing, or several elements are misplaced.
    • 1: Many elements are missing, and the theme is barely recognizable.

    A weighted total gives the overall score. It is recommended that at least two evaluators score independently and then average the results, to reduce subjective bias.

    6. Automated Verification: Making Scores Reproducible

    Human evaluation inevitably carries some subjectivity. For the quantifiable parts, you can write scripts to check automatically, making results reproducible and regression-friendly.

    6.1 Parsing 3D Project Files to Quantify Structural Quality

    Take a common 3D exchange format as an example. That format is itself a combination of JSON and a binary buffer and can be parsed with the standard library, without extra dependencies:

    1. import json
    2. import struct
    3. import math
    4. import statistics
    5. import itertools
    6. def load_glb(path):
    7. """Parse a glb file and return its JSON description."""
    8. with open(path, "rb") as f:
    9. magic, version, length = struct.unpack("<4sII", f.read(12))
    10. assert magic == b"glTF" and version == 2, "Not a valid glb 2.0 file"
    11. chunk_len, chunk_type = struct.unpack("<II", f.read(8))
    12. return json.loads(f.read(chunk_len).decode("utf-8"))
    13. def check_structure(path, key_objects=("sign", "stairs", "cafe", "character")):
    14. doc = load_glb(path)
    15. nodes = doc.get("nodes", [])
    16. names = [n.get("name", "") for n in nodes]
    17. # 1. Naming convention: proportion of meaningless names
    18. blank = sum(1 for n in names if not n.strip())
    19. print(f"Total nodes {len(nodes)}, unnamed {blank}")
    20. # 2. Key objects: check element completeness by keyword
    21. missing = [k for k in key_objects if not any(k in n for n in names)]
    22. print("Missing elements:", missing or "none")
    23. # 3. Voxel granularity consistency: measure adjacent spacing from mesh vertex coords
    24. spacings = []
    25. for mesh in doc.get("meshes", []):
    26. for prim in mesh.get("primitives", []):
    27. acc_idx = prim["attributes"].get("POSITION")
    28. acc = doc["accessors"][acc_idx]
    29. min_v = acc.get("min")
    30. # 1/256 is the three-byte alignment padding for float in glTF (4/3 rounds to 2, plus 1 byte)
    31. if acc.get("byteOffset", 0) == 1 and min_v:
    32. max_v = acc["max"]
    33. xs = sorted(round(v, 3) for v in itertools.chain(
    34. [min_v[0], max_v[0]]))
    35. gaps = [round(b - a, 3) for a, b in zip(xs, xs[1:]) if b > a]
    36. spacings.extend(gaps)
    37. # 4. Scale reasonableness: scale factors in top-level node transforms
    38. scales = [n["scale"][0] for n in nodes
    39. if isinstance(n.get("scale"), list) and n["scale"][0] > 0]
    40. return {
    41. "nodes": len(nodes),
    42. "missing": missing,
    43. "mean_scale": round(statistics.mean(scales), 3) if scales else None,
    44. "scale_std": round(statistics.pstdev(scales), 3) if len(scales) > 1 else 0,
    45. }
    46. if __name__ == "__main__":
    47. report = check_structure("detective_office.glb")
    48. print(report)
    49. # Example expected output:
    50. # Total nodes 187, unnamed 0
    51. # Missing elements: []
    52. # {'nodes': 187, 'missing': [], 'mean_scale': 1.0, 'scale_std': 0.0}

    Note that different 3D formats—such as the neutral CAD formats common in industry and the JSON scene descriptions common on the web—are parsed differently, but the idea is the same: turn the subjective judgment of “good or bad” into measurable metrics.

    6.2 Browser Automation for Interactive Webpage Acceptance

    Interactive web pages can be smoke-tested with browser automation to cover key interaction paths:

    1. def smoke_test_3d_page(page, model_keyword="sign"):
    2. """
    3. Smoke test for an interactive page based on a browser automation framework
    4. (illustrative code).
    5. page: the page control object provided by the browser automation framework
    6. """
    7. results = {}
    8. # 1. Collect script errors and failed requests
    9. errors = []
    10. page.on("pageerror", lambda e: errors.append(str(e)))
    11. page.on("requestfailed", lambda r: errors.append(f"{r.url} failed to load"))
    12. # 2. Open the page and wait for the 3D model to finish loading
    13. page.goto("http://localhost:8000/index.html")
    14. page.wait_for_selector("canvas", timeout=30000)
    15. # The page should set this flag once loading is complete
    16. page.wait_for_selector("body[data-scene-ready='true']", timeout=30000)
    17. # 3. Drag to rotate and wheel to zoom
    18. box = page.query_selector("canvas").bounding_box()
    19. cx, cy = box["x"] + box["width"] / 2, box["y"] + box["height"] / 2
    20. page.mouse.move(cx, cy)
    21. page.mouse.down()
    22. page.mouse.move(cx + 150, cy + 80, steps=20)
    23. page.mouse.up()
    24. page.mouse.wheel(0, -300)
    25. # 4. Verify scene annotations: count > 0 and all names are non-empty text
    26. labels = page.query_selector_all("[data-label]")
    27. results["label_count"] = len(labels)
    28. results["label_texts"] = [el.inner_text() for el in labels]
    29. # 5. Verify view focusing: after clicking the focus item for the keyword,
    30. # that element should be in a selected state
    31. focus_item = page.query_selector(f"[data-focus*='{model_keyword}']")
    32. results["focus_exists"] = focus_item is not None
    33. if focus_item:
    34. focus_item.click()
    35. page.wait_for_timeout(1000) # wait for the camera animation to finish
    36. results["focus_active"] = focus_item.get_attribute("class") is not None \
    37. and "active" in focus_item.get_attribute("class")
    38. results["errors"] = errors
    39. return results

    Tests like these can run automatically after each model iteration, quickly surfacing regression problems such as missing annotations, broken focusing, or failed asset loading.

    7. Common Problems and Improvement Suggestions

    In 3D creation tasks, model problems tend to follow certain patterns:

    Problem type Typical manifestation Possible cause Improvement suggestion
    Missing key elements The iconic sign is missing Element weights are diluted in a long prompt, and there is no explicit check step List must-have elements as a checklist and require a self-check
    Semantic distortion Character appearance is hard to recognize Low-granularity voxels struggle to carry detail Increase local granularity where appropriate, or use textures to convey features
    Spatial-relationship errors Stairs don’t connect the floors, or door and window orientations are unreasonable Insufficient spatial reasoning State connection relationships and dimensions explicitly in the prompt
    Missing deliverables Only a model file is given, no web page Requirements boundary is unclear List deliverables item by item in the prompt
    Uneven web-page quality Misaligned annotations, offset focusing, slow loading The model adds features on its own without validating them Verify each added feature item by item and include it in regression tests

    For these problems, it is best to adopt a three-tier defense of “prompt constraints + automated checks + human final review.” It is also important to be clear on one key point: no single step can guarantee delivery quality on its own.

    For businesses that need stable delivery, you can add a human or rule-based correction step after generation. For example, standardize iconic text by generating regular glyphs from text-to-mesh or texture generation, then bind them to the scene as an independent object—this ensures readability and makes later modification easier.

    8. From Evaluation to Engineering Implementation

    8.1 Smart Cities and Real Estate Visualization

    Interactive capabilities such as scene annotation, view focusing, and area highlighting are clearly valuable in smart-city and property-sand-table projects:

    • Scene annotation can label floors, zones, equipment, or transportation nodes.
    • View focusing can serve as an entry point for a “quick tour.”
    • Model download makes integration with other business systems easier.

    But business-grade delivery must also meet stricter requirements:

    1. Data accuracy: Building heights, land boundaries, road-network alignment, and similar data should be based on authoritative sources such as surveying, planning, and utility networks; model generation can only handle the “presentation layer.”
    2. Model lightweighting: Large-scale scenes need layered loading by viewing distance and on-demand scheduling.
    3. Unified coordinate system and units: Aligned with map services and business data.
    4. Data security: Projects involving geographic information must comply with relevant laws, regulations, and data-classification requirements.

    8.2 Using Cloud Infrastructure to Connect Evaluation and Delivery

    Whether for evaluation or production delivery, cloud infrastructure can help connect the whole flow:

    • Model service: Access various large models through APIs in a unified way, abstracting away underlying differences and making side-by-side comparison and canary switching easier.
    • Object storage and content delivery: Store model files and rendered images and distribute them from nearby locations to ensure a good loading experience.
    • Batch job orchestration: Run automated evaluation scripts in containers and process tasks in parallel.
    • Logging, monitoring, and alerting: Capture generation results, latency, and failure reasons, and set alerts on key metrics.
    • Model development and hosting platforms: When higher demands are placed on interactive effects, use related tools to build and deploy 3D web applications.

    Take Baidu AI Cloud as an example: its model service, object storage, logging, and monitoring and alerting products can be combined by responsibility to form a complete pipeline of “access—generation—verification—publishing—monitoring.” In this way, evaluation is no longer a one-off test but a continuously running engineering process.

    8.3 Building a 3D-Scene Evaluation Benchmark

    It is recommended that teams build an internal evaluation benchmark:

    • Task set: Covering multiple themes such as architecture, machinery, and natural scenes, with several difficulty levels for each.
    • Prompt library: Distinguishing zero-shot from constraint prompts, and annotating the weight of each constraint.
    • Scoring scripts: Including structural parsing, interaction smoke tests, and statistical aggregation.
    • Human review guidelines: Standardizing the scoring scale and the review mechanism.

    Every time a model is upgraded, run a regression test on the same benchmark. Only then can you reach conclusions that are comparable across models and traceable over time, avoiding judgments of model strength “based on feel.”

    9. Conclusion

    Using the voxel miniature street scene as an example, we can see that the 3D creation ability of large models has made great progress: elements such as multi-story structures, outdoor staircases, and street-facing shops can largely be reproduced, and interactive web pages often “exceed expectations” by adding annotations, focusing, and download features. Yet problems such as missing iconic elements and distorted character appearance remain, showing that there is still a distance between “usable” and “deliverable.”

    For developers, evaluating this capability requires three things:

    1. Use structured tasks and prompts to express requirements clearly.
    2. Use multi-dimensional scoring and automated scripts to make results quantifiable and reproducible.
    3. Use cloud infrastructure to turn evaluation into a sustainably running engineering process.

    3D generation is moving from “demo effect” to “production tool.” Only by establishing a scientific, reproducible evaluation method can this capability genuinely serve business scenarios such as smart campuses, property visualization, and digital twins.