LLM Capabilities and Limits in 3D Modeling: A Practical Assessment

1. Background: The Gap Between Text Generation and Spatial Modeling

Large language models were originally designed for natural language understanding and generation. In recent years, however, developers and designers have increasingly attempted to apply them to 3D modeling, parametric design, BIM component generation, and game scene construction. These tasks differ fundamentally from pure text generation: the model must understand not only nouns but also spatial relationships, geometric constraints, boolean operation sequences, and coordinate system transformations.

A common scenario in practice: a user wants the model to generate 3D geometry that can be imported into modeling software based on an abstract description. For example:

“Please use common 3D modeling software to build the ‘flat-beam hall’ described in Yingzao Fayuan, with the beam frame following flat-beam construction methods and the column grid divided by central and secondary bays.”

This looks like a single instruction, but it implies a large set of domain rules: the cross-sectional form of flat beams, the ratio of column diameter to bay width, mortise-and-tenon or lap joints, and the roof curvature method. If a model only has language capability without geometric reasoning, it will typically produce incorrect structures at the very first step.

More common problems appear during the modification phase. The first version of a model can often show a rough outline, but once the model is asked to “lower the eave height,” “change the columns to square,” or “adjust the beam cross-section ratio,” many models become unstable: they either ignore earlier constraints, modify unrelated components, or repeatedly generate syntax errors or non-closed geometry. The core issue is that the model lacks continuous tracking of 3D scene state and constraint propagation capability.

Therefore, evaluating whether a model is suitable for 3D modeling collaboration should not focus only on first-version generation quality, but also on its overall performance in multi-turn modification, local constraints, geometric parameter feedback, and script executability.

2. Evaluation Dimensions: What Qualifies as “Usable for 3D Modeling”

From an engineering perspective, a large language model’s capability for 3D modeling collaboration can be assessed across five dimensions.

2.1 Instruction Parsing and Domain Terminology Understanding

Architectural components, furniture structures, and mechanical parts all have extensive specialized terminology. Terms such as “flat-beam hall,” “crescent beam,” “roof curvature,” “inward lean,” and “column rise” require the model to recognize these terms and map them to correct geometric features. If a model interprets a “flat beam” as a simple rectangular box, it has only captured the literal meaning without establishing domain-specific semantic associations.

2.2 Spatial Relationship and Constraint Expression

The model must be able to express geometric relationships such as “A is directly above B,” “C and D are symmetric along the X axis,” or “E’s inner diameter equals 0.6 times F’s outer diameter.” More critically, when a user later modifies a dimension, can the model identify which components should change in response? This is essentially the “constraint propagation” capability in parametric modeling.

2.3 Script Executability and Stability

One of the primary ways LLMs assist 3D modeling today is by generating scripts—for example, Python scripts that call a modeling software’s API, or geometry description files that can be imported. The generated script must be syntactically correct, use existing commands, keep parameters within reasonable ranges, and avoid boolean operations that produce non-manifold or empty results. Otherwise, users waste significant time on trial and error.

2.4 State Retention Across Multi-Turn Modifications

This is currently the weakest link. Users expect the model to remember the overall structure of the current model and keep other parts unchanged when modifying a local component. In practice, the following problems frequently occur:

• Regenerating all code loses the existing structure;
• Modifying component A accidentally changes component B;
• Fixing one error introduces new errors;
• Repeated adjustments never satisfy the original constraints.

This indicates that the model lacks an explicit scene state manager, and the generated code tends to be a one-shot process rather than a sustainable, iterable parametric system.

2.5 Error Localization and Self-Repair

When the 3D software reports an error, can the model quickly determine whether it is a syntax issue, a geometry issue, or a logic issue, and provide a targeted fix? This is also key to evaluating engineering usability. A model that only apologizes but cannot correct itself has limited value in real modeling tasks.

3. A Representative Test: Traditional Timber-Frame Hall

To illustrate these issues concretely, this article uses a general test case: asking the model to generate a 3D model of a hall that follows traditional timber-frame construction methods. Since no specific product is discussed here, the focus is on task design itself.

3.1 Task Description Design

The instruction given to the model should not be a single short sentence but should contain sufficient structural information. For example:

Please use a 3D modeling software’s scripting interface to generate a simple traditional timber-frame hall:

1. Three bays across the front, with the central bay 4.2 m wide and the secondary bays 3.6 m wide;
2. Five purlins in depth, with a front-to-rear eave column distance of 6 m;
3. Column height 4.5 m, column diameter 0.3 m;
4. The beam frame uses flat-beam construction, with the main beam depth approximately 1/10 of the span;
5. The roof is a flush gable roof with a 0.9 m eave overhang;
6. First generate three layers for the column grid, beam frame, and roof surface, and name them separately.

The value of this description lies in translating vague domain language into executable structural parameters. If the model can correctly understand and generate a script, it at least demonstrates basic parsing capability.

3.2 First-Version Output Evaluation

First-version output typically exhibits the following problems:

• Column grid coordinates are correct, but beams and columns are not accurately aligned;
• The roof surface is generated as a single flat plane without roof curvature;
• The boolean operation order between components is wrong, causing some geometry to disappear;
• Layer divisions do not meet requirements;
• Units are inconsistent, mixing meters and centimeters.

These errors show that the model still has clear weaknesses in “understanding geometric topology relationships.” It can break a concept into component names but struggles to maintain spatial consistency between components.

3.3 Modification Phase Evaluation

When the user further requests “change the central bay width to 4.8 m while keeping everything else unchanged,” model performance varies dramatically.

In the poorer case, the model regenerates all code and changes the roof, column height, and other parameters that were not requested. In the slightly better case, the model can identify the variable corresponding to “central bay width,” but when that variable is modified, the related beam length, purlin length, and roof width do not update accordingly.

This shows that the model lacks an internal representation of a “parameter dependency graph.” A mature 3D modeling assistance system should treat “central bay width” as a primary parameter and automatically derive other dimensions based on parameter relationships between components.

4. Engineering Insights: Building a More Reliable Human-AI Collaborative Modeling Workflow

4.1 Convert Natural Language Descriptions into Structured Constraints

Rather than letting the model improvise, it is better to break requirements into structured constraints. For example:

Project: Traditional timber-frame hall
Component: Column
  Type: Eave column
  Quantity: 6
  Cross-section: Circular
  Diameter: 0.30m
  Height: 4.50m
Constraints:
  - Eave columns in two rows along Y direction, spacing 6.00m
  - Three columns per row along X direction, spacings 3.60m, 4.20m, 3.60m

Structured descriptions can significantly reduce model ambiguity and also help locate parameters during subsequent modifications.

4.2 Require the Model to Output Parametric Scripts Instead of One-Off Geometry

A more recommended approach is not to let the model directly generate point, line, and surface coordinates, but to first generate a parameter table and component dependency relationships, then generate the modeling script. For example:

main_span_mm = 4200
secondary_span_mm = 3600
column_height_mm = 4500
column_diameter_mm = 300
beam_height_mm = main_span_mm / 10

As long as the model maintains these parameters and their relationships, modifications are more likely to change only the primary parameter while automatically updating associated components.

4.3 Introduce External Geometry Validation and Error Feedback

It is difficult for the model alone to handle complex geometry reliably. In practice, after the modeling script executes, geometry checking logic can be invoked—for example, checking whether components intersect, whether dimensions exceed limits, and whether layers conform to standards—and the results can be fed back to the model for repair. This “generate–execute–validate–repair” loop is more reliable than single-pass generation.

4.4 Limit the Scope of Each Modification

During multi-turn modifications, present only a small number of clear adjustment requests to the model each time, and require the model to state in its output which components and parameters are affected. For example:

This round only adjusts the central bay width from 4200mm to 4800mm. Please first list the affected components, then provide a differentiated modification plan. Do not regenerate the entire model.

Although this prompting approach cannot completely eliminate errors, it can significantly reduce the probability of unrelated components being damaged.

5. Conclusion: How Far From Stable Usability?

In practice, large language models can already handle the initial transformation from description to first-version geometry in 3D modeling tasks, particularly for simple components, regular arrays, and parametric shapes. However, when the task enters the “modify–maintain–propagate–repair” stage, model performance remains unstable.

The core bottleneck is not “inability to model” but the lack of explicit spatial memory, geometric constraint propagation mechanisms, and verifiable state management. Models are better at generating one-off scripts than maintaining an iterable, rollback-capable, locally updatable parametric scene.

Therefore, a more reasonable positioning at this stage is to treat large models as “collaborative generators” and “script drafters” in the 3D modeling workflow, supplemented by structured constraint input, script execution validation, parameter dependency management, and human confirmation mechanisms—rather than treating them as “automatic modeling engines” that can independently complete complex architectural modeling.

For developers and design teams, a worthwhile investment is building an intermediate representation layer that uses a general data structure to describe components, parameters, constraints, and topological relationships. The model would then work on that representation layer rather than directly manipulating every detail of a specific modeling application. This architecture reduces vendor-specific dependencies and facilitates migration across different 3D modeling environments.

When a model can remain stable across multi-turn modifications, accurately understand geometric constraints, and continuously repair errors based on validation feedback, it will truly have practical value for complex 3D modeling collaboration. Until then, setting realistic expectations, optimizing task decomposition, and introducing external validation mechanisms are the keys to improving success rates.