Evaluating LLMs in 3D Creation and Spatial Reasoning: A Practical Analysis
This article analyzes the 3D creation and spatial reasoning capabilities of large language models (LLMs) through a practical "bare-bones" testing approach. By prompting an LLM to generate a voxel-based 3D miniature landscape, we evaluate its ability to autonomously construct spatial structures, handle abstract visual instructions, and proactively generate interactive 3D code. The analysis also covers prompt engineering strategies for 3D generation and explores the integration of LLMs with cloud-native services for enterprise-level deployment.
In recent years, the boundaries of Large Language Model (LLM) capabilities have been expanding at an unprecedented pace. Historically, model evaluations focused on flat, 2D dimensions like code generation, natural language understanding, and logical reasoning. As technology evolves, evaluation metrics are shifting toward higher-dimensional spatial understanding and 3D creation. Mainstream LLMs now attempt to generate engineering files for professional 3D modeling software and interactive code for frontend 3D engines, marking a transition from “text understanders” to “spatial builders.”
1. Background and Evaluation Objectives
A recently released LLM marketed its 3D creation and spatial reasoning abilities as core features, offering dedicated 3D content generation channels to output 3D models, interactive webpages, or 3D printing data. To objectively assess these claims, we designed a “bare-bones” experiment without relying on any preset skills. Conducted on an AI integration platform without external plugins, we used pure natural language prompts to drive the LLM in constructing a 3D asset with a specific visual style. This approach isolates the model’s internal spatial imagination and code generation quality from external toolchain interference.
2. Prompt Design and Practical Case
The test objective was to generate a 3D voxel miniature landscape themed around a famous building from an anime series. Voxel modeling, using blocks as basic units, tests the model’s grasp of macroscopic building structures and its ability to stack details at the microscopic level. We input a structured prompt:
[Role Setting] You are a senior 3D technical artist.[Task Objective] Use the Python API of a professional 3D modeling software to create a 3D voxel miniature landscape themed around a "detective agency from an anime".[Style Requirements] Inspired by the anime building, highlight the agency as the visual focal point, using pure voxel block language, emphasizing the stacked geometric structures.[Deliverables]1. Two PNG format multi-angle views rendered out.2. An editable 3D modeling software engineering file.
The prompt defined the modeling language (voxel), visual center (agency building), and delivery format (image + engineering file), but left specific spatial details blank to test the model’s spatial reasoning and autonomous completion capabilities.
3. Deliverable Analysis: Highlights and Missing Elements
The model successfully delivered two rendered views and a compliant engineering file. Overall, it captured the essence of voxel modeling; the main framework, door/window proportions, and stacking logic were acceptable. However, a critical omission was identified: the model missed the building’s most recognizable visual element—the prominent sign on the wall.
This highlights a typical pain point in current LLM 3D spatial reasoning: the lack of ability to map abstract instructions like “visual highlight” to concrete features. While the model understands general building structures, it often ignores iconic IP details to reduce generation complexity unless given explicit coordinate positioning and text descriptions. Under strict criteria, missing such a key anchor prevents a perfect score.
4. Proactive Compensation in Interactive Webpages
Despite the flaws in the pure modeling output, the model’s performance in the interactive layer was surprisingly good. Without an explicit request, it proactively generated interactive webpage code based on a mainstream frontend 3D engine. This code cleverly compensated for the static file’s limitations, adding features like multi-angle camera switching, dynamic lighting adjustments, and local detail zooming.
This “proactive compensation” mechanism indicates progress in understanding the user’s latent intent to “view and control the 3D model.” The model acts less like a rigid instruction executor and more like a collaborative partner with product thinking.
5. Prompt Iteration and Engineering Strategies
Addressing the missing sign from the first round, we iterated the prompt and explicitly requested the interactive webpage:
[Supplementary Instructions]1. On the second floor of the building's front facade, use contrasting colored voxels to spell out the text "Detective Agency" as a sign.2. In addition to the original engineering file, output a WebGL-based HTML interactive page supporting mouse drag-to-rotate and scroll-to-zoom.
With the supplementary instructions, the model successfully generated the sign structure at the specified coordinate layer and output the expected webpage file. This iteration provides key insights into 3D prompt engineering: in 3D contexts, prompts must go beyond “what to draw” and specify “where to draw” (coordinate systems and spatial relations), “how to draw” (materials and rendering pipelines), and “how to present” (interaction logic).
6. Conclusion and Cloud Synergy Outlook
Overall, current LLMs demonstrate considerable application potential in 3D creation and spatial reasoning. Their ability to generate voxel models and interactive code offers new productivity tools for game development and metaverse asset construction. However, spatial reasoning still suffers from long-distance dependency breaks and the loss of critical details, requiring developers to guide and correct via refined prompt engineering.
Furthermore, large-scale 3D asset generation and real-time rendering demand higher computing power. In enterprise applications, combining LLM generation capabilities with cloud-native services like Baidu AI Cloud is key to engineering implementation. For example, using Baidu AI Cloud’s elastic GPU compute instances to accelerate model inference and rendering, leveraging Object Storage Service (BOS) for stable hosting and distribution of massive 3D engineering files, and integrating container platforms for automated 3D generation pipelines can effectively bridge the gap between AI generation and business application. With dual breakthroughs in algorithms and computing power, LLMs will inevitably deliver more mature and reliable performance in 3D creation.
