Evaluating and Practicing AI-Assisted 3D Voxel Scene Generation
This article explores the methodology for evaluating AI models in generating 3D voxel scenes. It covers task decomposition, evaluation dimensions, data representation in Python, interactive webpage implementation, common risks, and practical checklists to transform subjective assessments into reproducible engineering practices.
1. Background
Beyond code generation, large language models (LLMs) are increasingly taking on 3D content creation tasks, such as modeling, scene building, and interactive page generation. Compared to text and code, the quality of 3D content is much harder to judge. Whether a scene is acceptable requires considering spatial structure, proportional relationships, detailed elements, and interaction logic simultaneously. A simple “looks good” cannot provide a reproducible conclusion.
Therefore, when evaluating 3D generation capabilities, the first step is to clarify the problem definition: what input the model receives, what output it should deliver, and what the evaluation criteria are. Only by clarifying these three aspects can different models or different versions of the same model be comparable.
2. Task Decomposition: From Prompt to Deliverable
A typical 3D scene generation task can be broken down into four stages:
- Requirement Understanding: The model needs to identify the theme, style, scale, and essential key elements.
- Structure Planning: The model must determine the spatial layout of the scene, such as floor relationships and the positions of the main subject and background.
- Voxel Construction: The model translates the structure into a combination of voxel blocks, specifying a material or color for each.
- Delivery Packaging: The model outputs an editable project file, rendered images, or an interactive webpage that can be viewed directly in a browser.
The design of the prompt directly affects the evaluation results. A good prompt usually includes the following elements: theme and style description, a list of mandatory elements, deliverable format, and size or proportion constraints. When a list of key elements is missing, the probability of the model omitting details significantly increases, and the evaluation conclusion tends to be biased toward “overall impression.”
3. Evaluation Dimension Design
It is recommended to score from the following five dimensions and retain specific observation records for each:
1. Semantic Fidelity: Does the overall scene match the theme description, and is the style close to the reference?
2. Key Element Integrity: Are all necessary elements in the list present? This is the easiest to quantify and can be checked item by item. A missing iconic sign has a different impact than a slight deviation in a building’s shape; evaluations should differentiate the weights.
3. Structural Rationality: Do structures like floors, doors, windows, and passages conform to spatial logic, and are the proportions of each part coordinated?
4. Interactive Extensibility: Beyond basic functions, does the model reasonably add valuable interactions, such as view switching, annotations, and model downloads? Added functions should be judged on whether they truly serve the use case, rather than simply being “the more, the better.”
5. Delivery Standardization: Are the file format, naming, number of rendered images, and descriptive information complete? Can the project file be opened and edited normally?
It is recommended to use the same scoring standard for each dimension and keep original screenshots or files for subsequent review.
4. Data Representation of Voxel Scenes
The core of a voxel scene is a set of blocks with coordinates and colors. The following example uses pure Python to implement a minimal scene builder and exports the result as JSON, making it easy for any renderer supporting this format to read.
from dataclasses import dataclass, fieldimport json@dataclassclass VoxelScene:name: strvoxels: list = field(default_factory=list) # [x, y, z, [r, g, b]]labels: list = field(default_factory=list) # Text labels and their coordinatesdef add_box(self, x0, y0, z0, sx, sy, sz, color):"""Add a rectangular voxel block at the specified starting point."""for x in range(x0, x0 + sx):for y in range(y0, y0 + sy):for z in range(z0, z0 + sz):self.voxels.append([x, y, z, list(color)])def add_label(self, text, x, y, z):self.labels.append({"text": text, "pos": [x, y, z]})def to_json(self):return json.dumps({"name": self.name, "voxels": self.voxels, "labels": self.labels},ensure_ascii=False,)scene = VoxelScene("Detective Agency")scene.add_box(0, 0, 0, 8, 6, 1, (200, 180, 150)) # Foundationscene.add_box(0, 0, 1, 8, 6, 3, (230, 220, 200)) # First-floor wallsscene.add_label("Agency Sign", 4, 0, 5)scene.add_label("First-floor Cafe", 2, 0, 1)# Key element checkrequired = {"Agency Sign", "Second-floor Stairs", "First-floor Cafe"}found = {label["text"] for label in scene.labels}print("Missing elements:", required - found)with open("scene.json", "w", encoding="utf-8") as f:f.write(scene.to_json())
The “key element check” part in the example can be directly used as an automated evaluation check. By performing a set difference operation between the list and the actual output, missing items can be identified. This method cannot replace manual observation, but it ensures that every item on the list is checked.
Note that the number of voxels grows cubically with size. A solid cube with an edge length of 100 has about 1 million blocks. Therefore, in actual projects, priority should be given to using mesh merging, hollow structures, or block loading to control the data volume.
5. Minimal Implementation Idea for Interactive Webpages
Interactive pages usually include the following basic capabilities:
- Rotation, scaling, and translation of the model, implemented by standard 3D interaction control logic;
- Scene annotation, anchoring text labels to 3D coordinates that change with the view;
- View focusing, pre-saving several key views for smooth transitions upon clicking;
- File download and multi-image switching, making it easy to display render results and project files.
During implementation, it is recommended to separate the data layer from the presentation layer. The data layer only outputs the JSON structure from the previous section, and the presentation layer is responsible for reading and rendering. This way, even if the rendering method changes, the scene data does not need modification.
Providing operation instructions when first opening the page is an easily overlooked detail that affects user experience. For pages requiring 3D interactions, a brief prompt can significantly reduce the user’s learning curve.
6. Common Problems and Risks
Missing Key Elements: This is the most common issue. The generated result may be reasonable overall but missing a specific iconic component. The countermeasure is to explicitly list necessary elements in the prompt and check item by item after delivery.
Semantic Ambiguity: If the theme name has a specific cultural or work background, the model may only restore part of it. The prompt should supplement specific descriptions of appearance and layout to reduce reliance on the model’s implicit understanding.
Proportion Distortion: When floor heights, door widths, and character sizes are uncoordinated, the visual will be noticeably distorted. You can provide reference dimensions in the prompt or use a unified proportional scale during the checking phase.
Interaction Over-design: Too many supplementary functions will increase maintenance costs. When evaluating, distinguish between “useful additions” and “decorative clutter.”
Non-editable Project Files: If the delivered project file cannot be opened normally or its structure is chaotic, no matter how high the quality of the rendered image is, it cannot meet subsequent modification needs. Delivery standardization should be assessed as an independent dimension.
7. Practical Suggestions and Checklist
- State the theme, style, size, list of key elements, and delivery format in the prompt.
- Define the scoring standard before testing to avoid conclusions being dominated by a single impression.
- Check each key element item by item and keep screenshots and files.
- Generate multiple times with the same prompt to observe the stability of the results.
- Open the interactive page in different browsers to check if basic functions are available.
- Organize test records into a table for horizontal comparison between different versions.
By mastering the above process, the evaluation of 3D generation capabilities shifts from subjective feelings to reproducible engineering practices. This method is also applicable to other types of generative tasks, such as scene design, product modeling, and data visualization.
