Why 3D Scientific Understanding Is Difficult: From CAD Generation to Engineering Reasoning

As AI systems improve at generating 2D images and interacting with software interfaces, they are moving into more demanding 3D applications, including architecture, mechanical design, medical imaging, and scientific visualization. Many models can now create parts from natural-language descriptions, assemble structures, and generate multi-component assemblies. But a key question remains: does a generated object represent a valid engineering structure, or does it merely look convincing?

The distinction matters. In many visual tasks, an output may be considered good if its overall shape, color, and layout are approximately right. In engineering and science, a misplaced hole, an incorrect connection, or an inaccurate distance measurement can invalidate the result. To reason about 3D objects, a model must handle geometry, topology, spatial orientation, scale, and domain-specific rules together.

1. From Drawing Objects to Understanding Them

Generative models can produce impressive CAD-like results. Given a text description, they may construct a turbine, gear, bracket, or building component, generate multiple parts, and even present an exploded-view animation. These outputs can be visually striking, but visual completeness does not guarantee engineering validity.

At minimum, an assembly needs to be checked for the following:

1. Dimensional constraints
   Lengths, radii, thicknesses, hole diameters, and spacing must meet design requirements. Visual proportions alone are not sufficient.

2. Correct topology
   Whether parts are connected, intersecting, nested, or separated by a clearance determines whether an assembly can function as intended.

3. Valid motion relationships
   Spatial reasoning is needed to check whether rotating parts interfere, sliding components have enough travel, and hinged structures have the expected degrees of freedom.

4. Manufacturability
   Before a model can become a product, it must be checked for features such as machinable holes, adequate wall thickness, and internal cavities that can be released from a mold.

5. Physical meaning
   In mechanical, biological, and chemical contexts, spatial relationships often carry specific meaning. A model must understand not only what an object resembles, but why its components are positioned as they are.

CAD generation is therefore primarily a construction capability. Scientific 3D understanding requires a complete cycle of observation, measurement, reasoning, and verification.

2. What Professional 3D Benchmarks Measure

A newer class of cross-disciplinary 3D evaluations attempts to reproduce how specialists inspect scientific objects. These benchmarks typically cover fields such as biology, chemistry, medical imaging, architecture, and mechanical CAD. They use hundreds of questions based on real 3D data to assess whether a model can independently interact with and analyze a scene.

Unlike conventional image question-answering, these tasks do not necessarily present a single static image. The input may be an interactive 3D scene that can be rotated, zoomed, sectioned, measured, or selectively hidden. A model must plan how to inspect the scene, choose suitable views and operations, and use the resulting measurements to answer the question.

Example tasks include:

• Hiding most of a protein structure while retaining a molecule and nearby ions, then determining the distance between them.
• Slicing or navigating medical imagery to identify the direction and level of an abnormality.
• Checking an architectural model for spatial conflicts among rooms, floors, and walls.
• Analyzing contact, relative position, and movement direction between parts in a mechanical assembly.
• Using molecular or structural geometry to determine whether a scientific property holds.

The challenge is not simply recalling a fact. A model must gather evidence through a sequence of operations. If it guesses from an object’s name or from general knowledge without locating it in the scene, occasional correct answers will not translate into reliable performance across many tasks.

3. Why a High CAD Score May Still Fall Short

Some models have achieved near-perfect results on CAD tests focused on interface operations or geometric appearance. Yet on 3D evaluations that emphasize scientific relationships and spatial reasoning, the top score has been around 0.53, with other models also failing to reach a consistently passing level. This is not a contradiction: the two types of tests measure different capabilities.

3.1 Visual similarity is not structural understanding

A model may generate something that looks like a typical turbine without understanding the constraints among blade count, axial alignment, sealing structures, and direction of rotation. The assembly may appear complete while parts intersect internally or lack actual, computable connections.

3.2 2D projections can hide 3D errors

From a fixed viewpoint, two intersecting parts may appear merely to touch. A component inside the model may also look correctly positioned. Rotating the scene, inspecting cross-sections, and comparing multiple views may be necessary to reveal errors along the depth axis.

3.3 Measurement is part of the reasoning process

If a task asks for the shortest distance between two objects, the model must identify the objects and choose the right reference points, coordinate system, and measurement method. Estimating from appearance alone is vulnerable to viewpoint, occlusion, and perspective distortion.

3.4 Domain knowledge cannot replace spatial verification

A model may know general facts about a chemical structure or understand the roles of walls and floors in a building. That knowledge does not establish the actual relationships in the current scene. Scientific tasks often require evidence from a particular object, not a restatement of a general definition.

3.5 Errors compound across multiple operations

A 3D task may involve locating, hiding, rotating, zooming, measuring, comparing, and answering. Selecting the wrong object at any stage can undermine everything that follows. Even when individual operations succeed fairly often, the completion rate for a long sequence can drop substantially.

4. Five Levels of 3D Scientific Reasoning

A useful way to analyze model capabilities is to divide 3D tasks into five levels.

Level 1: Scene recognition
The model identifies the objects in the scene and classifies them. For example, it may need to distinguish a protein, a ligand, and an ion, or identify walls, doors, beams, and pipes.

Level 2: Spatial localization
The model determines objects’ relative directions, depth, and positions, and understands relationships such as inside, outside, adjacent, overlapping, and passing through.

Level 3: Geometric measurement
This includes lengths, angles, radii, volumes, distances, and areas. Measurements must be derived from the scene’s actual coordinates rather than inferred from language or appearance.

Level 4: Relationship reasoning
The model determines whether objects touch, conflict, or contain one another, and whether a structure satisfies a specified scientific condition.

Level 5: Domain interpretation
The final answer must explain the result using relevant mechanical, architectural, biological, or medical knowledge. A number or a yes-or-no response may not be enough.

Many current models perform reasonably well at the first two levels, but often decline at geometric measurement, relationship reasoning, and domain interpretation. This helps explain why demonstrations can look smooth while performance under rigorous scoring remains limited.

5. Designing More Reliable 3D Evaluations

Organizations and research teams evaluating their own models should not rely only on rendered outputs or visual demonstrations. They need a reproducible validation process.

5.1 Define object-level ground truth

For every 3D object, record a unique identifier, category, coordinate bounds, topology, and key dimensions. For assemblies, also record constraints between parts, degrees of freedom, and acceptable tolerances.

5.2 Break tasks into verifiable steps

Instead of asking only whether a model is correct, assess whether it:

• Found the target object.
• Selected an appropriate viewpoint.
• Identified occlusion relationships.
• Made an accurate measurement.
• Evaluated the relevant geometric constraints.
• Reached a conclusion consistent with domain rules.

This makes it possible to determine whether a failure came from perception, measurement, or reasoning.

5.3 Evaluate operation traces as well as answers

A correct final answer does not necessarily indicate a reliable process. The model may have guessed correctly by chance. Evaluation should also record whether it operated on the right objects, used an appropriate measurement method, and maintained a consistent scene state.

5.4 Use tolerances and graded scoring

Engineering measurements involve error, so evaluation should not be limited to fully correct or fully incorrect. Scores can reflect different error ranges, with additional penalties for topology errors, object confusion, and unit mistakes.

5.5 Prevent leakage from textual cues

Object names in a question should not give away the answer. Otherwise, a model may rely on prior knowledge without analyzing the 3D data. A more robust design uses neutral identifiers and requires the model to confirm object properties from the scene.

6. Practical Implications for Engineering Workflows

In production settings, 3D generation tools are better suited to creating initial drafts and exploring design options than to replacing engineering review. A more dependable workflow combines model generation, rule-based validation, and human review.

The model can first create an initial structure from requirements. A geometry engine or rule system can then check for collisions, clearances, dimensions, and topology. Finally, qualified professionals can review materials, manufacturing methods, load conditions, and safety limits. High-risk workflows should also preserve version history, operation logs, and rollback capability.

A system can separate 3D work into several modules:

• Scene parsing: Reads objects, hierarchies, and attributes.
• Interaction: Performs rotation, zooming, hiding, sectioning, and measurement.
• Geometric validation: Checks distances, intersections, bounding boxes, and solid validity.
• Domain rules: Applies constraints from architecture, mechanical engineering, or other scientific fields.
• Audit: Stores evidence, operation traces, and final conclusions.

This architecture is more dependable than relying on a large model’s output alone and makes failures easier to diagnose.

Conclusion

The real challenge in 3D AI is not generating an object that looks right. It is finding objects in complex space, measuring them, verifying their relationships, and producing an explainable conclusion grounded in scientific and engineering rules. Current models have made clear progress in visual presentation and initial modeling, but significant gaps remain in precise localization, geometric calculation, multi-step interaction, and domain reasoning.

Evaluations of 3D AI should therefore go beyond demo videos, rendering quality, or isolated CAD scores. More useful metrics include object identification accuracy, measurement error, topology accuracy, reliability of operation traces, and whether conclusions can be verified. Only when systems move beyond generation to inspection, explanation, and reviewability will they be ready to support engineering and scientific production workflows.