A Practical Guide to Automated 3D Scene Generation and Evaluation
This article documents a hands-on experiment that generates a 3D voxel scene from a single bare prompt, then turns the experience into a repeatable methodology for building, evaluating, and automating 3D content. It explains why LLM evaluation is shifting from code toward 3D deliverables, how to break a scene request into verifiable specifications, and three engineering approaches for turning text into editable scenes: script-driven parametric generation, declarative scene descriptions, and procedural meshes. It also covers building interactive browser viewers, converting subjective impressions into a reproducible scoring rubric, running automated validation and targeted defect repair, and scaling the workflow to batch generation. Readers will find concrete code and JSON examples, a weighted evaluation checklist, and practical lessons on where current models still fall short.
1. Background: Why Evaluation Is Shifting from Code to 3D Content
For a long time, code has been the core dimension of model evaluation, because it comes with objective standards such as compilation and unit tests. Recently, tasks like 3D modeling, voxel art, and browser-based interactive visualization have gained clear traction in developer communities. These tasks share a few characteristics:
- Visual deliverables: a rendered image or a page you can rotate — quality is obvious at a glance.
- Composite capability: the model must simultaneously handle geometry, materials, lighting, cameras, and frontend interaction.
- Complex acceptance criteria: structure, proportions, style, and interaction all need to be checked individually; a single metric cannot summarize them.
I recently ran a 3D scene generation experiment using a single bare prompt, on the theme “a voxel miniature of a two-story street-corner building.” The experience exposed clear weaknesses and also delivered unexpected gains. This article uses that experiment as its main thread and lays out a complete method for task decomposition, engineering implementation, interaction, and evaluation.
2. Task Decomposition: Turning “Build a Scene” into Verifiable Specs
2.1 The Value of a Bare Prompt
A “bare prompt” means giving the model a single natural-language requirement, without attaching any preset skills, templates, or workflows. It is closer to what ordinary users actually type, and it better exposes a model’s weaknesses in:
- inferring implicit requirements;
- applying common-sense understanding of a scene;
- grasping spatial relationships.
The prerequisite is that the requirement itself is specific enough. The prompt I used was:
Using a mainstream 3D modeling tool, create a voxel miniature on the theme of a “two-story street-corner building,” inspired by the corner detective agency from an animated series: the second floor is the agency’s office area, and the first floor is a street-corner cafe. Use a voxel-block visual language throughout, and deliver an editable project file and a rendered image.
It pins down the theme, reference style, voxel expression, floor functions, and delivery format, but deliberately leaves out:
- the agency’s storefront signboard;
- character figures;
- an external staircase;
- any web interaction.
Leaving these gaps was intentional — to see whether the model could fill in the key details needed to “recreate the animation” on its own.
2.2 The Three Layers of 3D Scene Generation
Regardless of the toolchain, 3D scene generation can be divided into three layers:
- Scene structure: which objects are included, how they are grouped and named, and how the parts combine;
- Geometry and materials: what primitives each object is built from, and how surfaces are shaded;
- Lighting and camera: light placement, shadow settings, and camera angle and composition.
These three layers map closely to the layout, modeling, and lighting stages in animation and film production. The reason a model’s output is inconsistent is often that it is strong at different layers: it handles structural questions like object count and floor relationships well, but slips on details like proportions and text recognition. When decomposing an evaluation task, you should likewise set check items for each of these three layers.
2.3 Delivery Format Defines the Task Boundary
The original prompt only asked for a project file and a rendered image. Because there was no online page, I had to add a line: “also generate an interactive web page that opens in a browser.” This small omission shows that the delivery format must be defined before the task starts; otherwise the model produces according to its default interpretation, and rework costs more.
There are three common delivery formats, each with its own use cases:
| Delivery format | Primary use | Key requirements |
|---|---|---|
| Editable project file | Further editing, team collaboration, asset reuse | Clear object naming, sensible grouping, standard primitives, adjustable material parameters |
| Rendered image | Reporting, option comparison, promotional material | Consistent composition, lighting, and color; resolution suited to the use |
| Interactive web page | Online demo, interactive presentation, sharing results | Loading performance, smooth interaction, cross-device compatibility, accessibility |
3. Engineering Implementation: Three Approaches from Text to an Editable 3D Scene
3.1 Script-Driven Parametric Generation
Most mainstream 3D modeling tools expose scripting interfaces, so a model can write a script and let the tool generate the scene. The advantages:
- Reproducible: the same script produces consistent results under the same version.
- Parametric: building dimensions, floor counts, and material palettes can all be extracted into variables.
- Verifiable: object names, positions, and dimensions are readable data in the script, which makes automated checking easy.
Illustrative code (interface names are for illustration; refer to the documentation of the tool you actually use):
import mathVOXEL = 0.25 # Voxel edge length, in metersdef add_voxel_block(name, center, size, material):"""Create blocks at voxel granularity: snap dimensions to the voxel grid, then create a cube and assign a material"""snapped = tuple(round(v / VOXEL) * VOXEL for v in size)# create_cube(name, location=center, dimensions=snapped, material=material)def build_street_corner():# Ground floor: street-facing cafeadd_voxel_block("Cafe_GF", (0, 0, 1.5), (8, 6, 3), "mat_brick")# Second floor: detective agency (shares the foundation with the ground floor, set back above)add_voxel_block("Office_2F", (-0.5, -0.5, 4.5), (7, 5, 3), "mat_plaster")# External staircase: number of steps derived from floor height and voxel sizesteps = int(3.0 / VOXEL)for i in range(steps):add_voxel_block(f"Stair_{i:02d}",(-4 - i * VOXEL, 2.5, (i + 0.5) * VOXEL),(VOXEL, 1.0, (i + 1) * VOXEL), "mat_concrete")# Ground-floor overhanging awning to enrich the facade layersadd_voxel_block("Awning", (0, 3.25, 2.75), (8, 0.5, 0.25), "mat_stripe")build_street_corner()
The risk of the script approach is its dependence on specific interface calls. If a function name or parameter is wrong, the script errors out immediately, so after generation you must actually run it and feed the error back to the model for correction.
3.2 Declarative Scene Descriptions
Another approach is to have the model first output a structured scene description (such as JSON), which a fixed generator then parses and builds. The model only decides “what should exist,” while geometry details are handled by a stable generator. This concentrates uncertainty in the data layer, making validation and version management easier. Example:
{"theme": "street_corner_office","style": "voxel","voxel_size": 0.25,"nodes": [{"id": "building", "type": "group", "children": ["cafe_gf", "office_2f", "stair", "signboard"]},{"id": "cafe_gf", "type": "box", "position": [0, 0, 1.5], "size": [8, 6, 3], "material": "brick"},{"id": "office_2f", "type": "box", "position": [-0.5, -0.5, 4.5], "size": [7, 5, 3], "material": "plaster"},{"id": "stair", "type": "stair", "from": [-4, 2.5, 0], "to": [-3.5, 2.5, 3], "width": 1},{"id": "signboard", "type": "text_panel", "parent": "office_2f", "text": "Detective Agency", "face": "front"},{"id": "characters", "type": "group", "children": ["char_01", "char_02"]}],"lighting": {"preset": "warm_afternoon", "sun_angle": 35}}
The limitation of the declarative approach is that expressiveness is capped by the generator. When it hits an element the generator does not support (such as “an abstract hairstyle”), it can only degrade to an approximation.
3.3 Procedural Mesh Generation
For highly repetitive or organic objects like terrain, mountains, and rock formations, procedural meshes are often more efficient than placing blocks one by one. Voxel-style expression typically first generates a high-density voxel field, then merges adjacent voxels to reduce face count, or uses an isosurface extraction algorithm to produce a continuous surface and then discretizes it back into blocks.
The most common performance problem in voxel-style scenes is that each block becomes a separate object, causing draw calls to spike and interactive web pages to lag. Practical optimizations include:
- merging adjacent voxels that share a material;
- using instanced rendering so many blocks share the same geometry data;
- organizing objects into batches by material to reduce switching overhead.
The tradeoff: fully merging reduces the editability of the project file. A sensible approach is to group by functional object, merging within a group while keeping groups separate — preserving editability while controlling face count.
3.4 Choosing a Route
| Approach | Controllability | Detail expressiveness | Engineering cost | Use cases |
|---|---|---|---|---|
| Script-driven | High | Medium | Medium | Buildings, machinery, regular structures |
| Declarative | Highest | Limited by generator | High upfront, low reuse cost | Batch generation, smart-city sandboxes, real-estate models |
| Procedural mesh | Medium | High | High | Terrain, natural landscapes, organic forms |
In real projects you can combine them: use scripts or declarative descriptions for the building parts to guarantee precision, and use procedural generation for vegetation and terrain to add detail.
4. Interactive Web Page: Making the Model “Think One Step Ahead”
I originally asked only for a project file and rendered images, yet the actual delivery included a fairly complete interactive web page — the most unexpected part of this experiment.
4.1 Basic Capabilities
- Orbit controls: rotate, zoom, and pan.
- Auto-rotation: the scene spins on its own during presentation, making it easy to grasp the whole quickly.
- First-entry guidance: prompts that tell users “drag to rotate, scroll to zoom.”
- Artifact integration: a model download entry and switching among multiple rendered images.
4.2 Advanced Capabilities
- Scene annotations: display explanatory text on key objects — useful for smart-park sandboxes, real-estate models, city modeling, and similar visualization projects. Users understand what each part of the model means without a presenter.
- View focusing: a sidebar lists object names in the model; clicking one smoothly moves the camera to that location.
One detail worth noting: one character name in the focus list was so abstract that I stared at the model for a while before figuring out which character it referred to. This is a normal result of the abstraction in voxel-style modeling, but it reveals two engineering problems:
- Labels must be locatable: the focus list should include thumbnails or highlight hints to avoid users “not being able to find it.”
- Names must be readable: web labels should correspond one-to-one with scene object names, generated from structured data rather than ad-hoc names the model makes up on the fly.
4.3 Data-Driven Annotations and View Focusing
The recommended approach is to keep annotation and focus configuration in a separate JSON file, with rendering code only responsible for reading and rendering. This eases content maintenance and makes batch generation and regression testing easier.
{"annotations": [{"target": "Office_2F", "label": "2F · Detective Agency", "anchor": [0, 5.2, 2.6]},{"target": "Cafe_GF", "label": "1F · Street-Corner Cafe", "anchor": [0, 2.0, 3.2]},{"target": "Stair_To_2F", "label": "External Staircase", "anchor": [-4.5, 1.5, 2.5]}],"focusViews": [{"id": "overview", "label": "Overview", "position": [12, 9, 14], "target": [0, 3, 0]},{"id": "office", "label": "Agency", "position": [5, 7, 8], "target": [0, 4.5, 0]},{"id": "cafe", "label": "Cafe", "position": [4, 3, 9], "target": [0, 1.5, 1]},{"id": "char", "label": "Front-Door Character", "position": [2, 2.5, 6], "target": [1.5, 1, 3.5],"highlight": "char_01"}]}
If the project file already organizes groups by object name, this configuration can be semi-generated automatically: traverse the scene objects, read their names and bounding boxes, and batch-produce annotation anchors and camera positions.
4.4 Engineering Quality: More Than “It Spins”
As a deliverable, an interactive web page needs to meet basic engineering standards:
- Loading performance: control face count and model file size; where necessary, apply geometry compression and level-of-detail loading, and show a loading progress bar on first paint.
- Interaction performance: render on demand when idle, to avoid wasting GPU resources.
- Compatibility: prepare a static rendered-image fallback for users without hardware acceleration.
- Accessibility: provide keyboard operation for interactions like camera focusing, offer a reduced-motion option, and set text colors with sufficient contrast against the overall palette.
Rendering parameters such as tone mapping, ambient light, and soft shadows greatly affect the look of a voxel scene. A scene with correct modeling but rough lighting looks far worse.
5. Experiment Review: A “Detective Agency” Miniature
5.1 Test Setup
- Input: a single bare prompt, with no skills or templates attached.
- Delivery: two model renderings and an editable project file, followed by a supplemental interactive web page.
- Reference: the corner agency from the animation, including the route up an external staircase to the second floor, the second-floor agency, and the first-floor cafe.
5.2 What Went Well
- Complete structure: the main elements — second-floor agency, first-floor cafe, and the external staircase to the second floor — were essentially all present.
- Sensible completion: it placed characters at the entrance, showing the model understood that “a streetscape with people feels more alive.”
- Rich interaction: drag-to-rotate, auto-rotation, scene annotations, view focusing, artifact download, and image switching were all there — many of them not requested in the prompt.
- Visual quality: the rendering had consistent color and natural filter effects.
5.3 Shortcomings
- Missing key element: the storefront signboard — the most recognizable element in the animation — did not appear. The signboard is an “identity marker” type of element that is easily overlooked during generation when expressed in a single sentence of natural language.
- Abstract detail modeling: the characters’ hairstyles were hard to make out, and proportions were not well coordinated with the voxel expression, showing the model’s grasp of concrete detail is still unstable.
5.4 Conclusion and Scoring
Overall, this generation performed well on structural and interaction completeness, with weaknesses concentrated in the missing signboard and character details. On a 100-point scale:
- Structure and layout, 25 points: main spatial relationships complete, scored 22.
- Key details, 20 points: missing signboard, distorted character hairstyle, scored 10.
- Rendering quality, 15 points: good composition and color, scored 13.
- Interaction, 25 points: exceeded expectations, scored 24.
- Engineering standards, 15 points: naming clarity and performance to be verified, provisionally 11.
Total: about 80 points. Though the signboard is only a small element, it is the visual core of the theme — the key reason this could not earn a higher score.
6. Evaluation Method: Turning Subjective Impressions into a Reproducible Scorecard
A single experience is easily swayed by personal preference. To make conclusions comparable and reproducible, you need fixed dimensions, weights, and criteria. I recommend evaluating across six dimensions:
- Structural completeness (25%): are all key elements from the requirement present? Check them off against an element list, and deduct directly for missing core elements.
- Semantic correctness (20%): are object functions and positions correct, are names readable and locatable, and does the style meet requirements (e.g., consistent voxel granularity)?
- Spatial plausibility (15%): are there clipping, floating objects, or proportion problems, and is the circulation route smooth?
- Rendering quality (15%): lighting, color, composition, and material rendering.
- Interaction (15%): is basic interaction smooth, and are there bonus features like annotations, focusing, and fallbacks?
- Engineering standardness (10%): are objects named and grouped, is face count within budget, and does the project open and allow re-editing normally?
Before evaluating, rewrite the natural-language requirement into a weighted requirement list, then check item by item. Example:
REQUIREMENTS = {"building_2f": {"desc": "Second-floor agency main body", "weight": 3},"cafe_gf": {"desc": "Ground-floor street-corner cafe", "weight": 3},"stair": {"desc": "External staircase to the second floor", "weight": 2},"signboard": {"desc": "Agency storefront signboard", "weight": 3},"characters": {"desc": "Characters at the entrance", "weight": 1},"style_voxel": {"desc": "Overall voxel block style", "weight": 2},}def score(items, found):"""items: requirement list; found: set of object identifiers detected in the scene"""total = sum(v["weight"] for v in items.values())hit = sum(v["weight"] for k, v in items.items() if k in found)return round(hit / total * 100, 1)print(score(REQUIREMENTS, {"building_2f", "cafe_gf", "stair", "characters"}))# If the overall voxel style counts as a hit: 9 / 14 ≈ 64.3# If the voxel style is not a hit: 9 / 16 ≈ 56.2
I also recommend recording the task ID, prompt version, parameter settings, and a hash of the deliverable for each test, to make it easy to compare different models side by side or different versions of the same model over time.
7. Engineering Practice: Automated Validation and Defect Repair
7.1 Checking the Scene Automatically with a Script
Manually inspecting item by item is inefficient and hard to reproduce. You can write a check script to structurally validate the scene. The following is illustrative using Python-style modeling interfaces; refer to the documentation of the tool you actually use.
def validate_scene(scene, required, unit_scale=1.0, max_overlap=0.2):"""Schematic scene structure validationscene: scene object (must support object traversal)required: list of object names that must existunit_scale: scene unit conversion ratio, used to check physical dimensionsmax_overlap: maximum allowed bounding-box overlap ratio"""objects = {obj.name: obj for obj in scene.objects}issues = []# 1. Whether all required elements are presentfor name in required:if name not in objects:issues.append(f"[missing] {name}")# 2. Whether units and scale are reasonable (e.g., overall scene height should fall within a sensible range)for obj in scene.objects:height = obj.dimensions.z * unit_scaleif height > 20:issues.append(f"[scale] {obj.name} height {height:.1f}m is abnormal")# 3. Whether there is obvious interpenetration between major building massesmain = [o for o in scene.objects if o.name.startswith(("Cafe_", "Office_"))]for i, a in enumerate(main):for b in main[i + 1:]:if overlap_ratio(a, b) > max_overlap:issues.append(f"[overlap] {a.name} overlaps too much with {b.name}")# 4. Whether there are meshes without an assigned materialfor obj in scene.objects:if obj.type == "MESH" and not obj.materials:issues.append(f"[material] {obj.name} is missing a material")return issues
For objects that are not detected, you can further determine whether the object genuinely does not exist or the name simply does not match. The former requires adding creation in the script; the latter only needs a name or mapping fix.
7.2 Local Repair: Generating the Storefront Signboard Procedurally
A missing signboard should not trigger a full scene regeneration — a targeted addition is more efficient. Voxel-style text can be implemented two ways:
- Built-in text object + voxelization: easy to edit afterward, but tiny fragments may appear at corners.
- Custom dot-matrix glyph + per-dot instancing: each stroke is a complete block for a consistent style, but modifications require code changes.
Illustrative code:
FONT = {"D": ["1111100", "0100010", "0100010", "0100010", "0100010", "0100010", "1111100"],}def build_sign(text, origin, pixel=VOXEL, thickness=VOXEL, board_size=(3.0, 0.8)):# 1. Backing boardadd_voxel_block("Signboard_Board", origin, (*board_size, thickness), "mat_board")# 2. Instantiate stroke blocks glyph by glyph, dot by dotx0, y0, z0 = origincursor = x0 - board_size[0] / 2 + pixelfor ch in text:glyph = FONT.get(ch)if glyph is None:continuefor row, line in enumerate(glyph):for col, bit in enumerate(line):if bit == "1":add_voxel_block(f"Signboard_Pixel_{ch}_{row}_{col}",(cursor + col * pixel, y0 + board_size[1] / 2 + thickness / 2,z0 + (len(glyph) - row) * pixel),(pixel, thickness, pixel), "mat_ink")cursor += (len(glyph[0]) + 1) * pixel
After adding it, sync the interactive web page’s annotation configuration and re-render the image to keep the project, web page, and image consistent.
7.3 Verifying the Repair
A repair should not stop at “the code looks fine” — it must run the full validation loop:
- the script runs without errors;
- the validation script confirms the signboard object exists, is correctly positioned, and has sensible dimensions;
- render the image and manually confirm the text is legible;
- the interactive web page loads normally and the new annotation displays correctly.
This “generate — validate — render — interaction-test” loop can plug directly into a CI pipeline. Run it automatically after each prompt or parameter adjustment to prevent quality regressions.
8. Scaling Up: Engineering for Batch Generation
A single-scene test is only the start; the real engineering value shows up in batch generation, such as real-estate sandboxes, park visualizations, and smart-city models. These scenes typically contain many buildings, roads, and greenery, making manual modeling costly and slow.
A workable pipeline:
- Structure the requirement: organize elements like land parcels, buildings, and roads into a standard list.
- Batch model generation: generate using a unified prompt template or structured description.
- Automated validation: use a unified check script to filter out results with missing elements, clipping, or scale anomalies.
- Cloud rendering: batch-produce preview images.
- Unified web packaging: reuse the same annotation and focus components, swapping only the data configuration.
In this flow, general-purpose cloud capabilities can take on key stages:
- object storage: storing model files, rendered images, and static web assets;
- container platforms: running modeling scripts and validation tasks with elastic scaling per task;
- logging and monitoring/alerting: tracking task failure rates and durations to catch anomalies early.
For example, cloud providers such as Baidu AI Cloud generally offer common capability categories like object storage, containers, and logging/monitoring, which can be used for artifact archiving, task execution, and operational observability respectively. Compared with generating one at a time on a local machine, this approach is better suited to batch, long-running tasks.
9. Limitations and Outlook
This experiment also revealed several typical limitations of current 3D content generation:
- Identity-marker elements get lost easily: small-but-critical elements like signboards, door plates, and identifying text are easily drowned out in natural-language expression.
- Unstable grasp of concrete detail: character shapes and hairstyles tend to suffer proportion imbalances or become unrecognizable.
- Interaction and generation must be coordinated: interaction features are great, but if label naming is messy, they add to the comprehension burden instead.
- Generation quality is constrained by usage quotas: model calls usually have daily quotas or rate limits. Before testing, plan your scene list and prompts, and spend the limited quota on high-value validation tasks.
Looking ahead, 3D content generation will keep evolving in these directions:
- models will infer “implicit requirements” more precisely;
- script execution and automated validation will form a more mature closed loop;
- declarative scene descriptions have a chance to become a standard cross-tool intermediate format.
For developers, rather than chasing the “wow” of a single generation, it is better to establish a reproducible requirement list, validation scripts, and scoring system early. That way you can keep making objective, comparable judgments as models iterate quickly.
