LLM-Powered CAD: Script Generation and Physical AI in Practice
LLM-powered CAD is best treated as an upper orchestration layer on top of the geometry kernel, not a replacement for it. This article explains how the interaction model moves from click-and-drag to describe-generate-verify-iterate, walks through building a parameterized bracket generator with a Pydantic spec and CadQuery, and shows how multimodal understanding, RAG and knowledge graphs, a unified 2D+3D+cloud architecture, and Physical AI fit together. It also gives measurable acceptance metrics, a risk checklist, and a decision table for choosing a technology path, with emphasis on starting from high-frequency, low-risk tasks.
1. What Is LLM-Powered CAD? From a Drafting Tool to an Intelligent Design Assistant
Key point first: LLMs do not replace the geometry kernel. They work better as an upper orchestration layer. They are suited to repetitive modeling, parameter tuning, template generation, and bulk parameter edits, and their gains should be validated through pilot tasks rather than promised as a blanket efficiency increase.
The core of traditional CAD is a deterministic geometry kernel. Designers build geometry, constraints, and assembly relationships feature by feature through menus, toolbars, command lines, or a parametric feature tree. The strength lies in accuracy and stability, but tasks such as bulk modification, reuse of similar parts, parametric derivation, and design knowledge retrieval still depend heavily on human experience.
LLM-powered CAD, script generation, parametric modeling, and multimodal CAD introduce a new interaction style for exactly these tasks. Designers describe intent in natural language; the model converts that intent into parameters, constraints, and scripts, and the CAD engine is called to build the model. The interaction paradigm shifts from “click and drag” to “describe – generate – verify – iterate”.
A more accurate statement is that LLMs have the potential to improve design efficiency, and this potential must be validated by measuring pilot tasks. Establish a baseline first: the same task completed by designers through the traditional workflow, or completed with the existing older scripts. Record the task type, part complexity, number of changes, and number of participants. Measurable indicators include task completion time, first-pass success rate of script generation, geometric correctness rate, human intervention rate, rework rate, and delivery lead time.
2. How Do You Generate CAD Scripts with an LLM? Doing Parametric Modeling
Key point first: scripted parametric modeling is the programmable handle of LLM-powered CAD. The recommended implementation path is: natural language requirement → parameter schema → CAD API → parameter validation → geometry validation → export and rollback.
Almost every mainstream CAD system provides a programmatic interface. Enterprises can describe the modeling workflow in Python, VBA, C#, C++, or a vendor API. Take the open-source Python geometry kernel CadQuery as an example: it is built on OpenCASCADE and is well suited to demonstrating parametric generation logic. If an enterprise uses SolidWorks, CATIA, Creo, or NX, map the flow below onto the corresponding API: box maps to an extrusion or base feature, subtract maps to a boolean cut, and constraints map to sketch constraints or assembly constraints.
Example runtime and dependencies: Python 3.10+, cadquery>=2.5, pydantic>=2.0, and pytest for regression tests. Code example:
from pydantic import BaseModel, Field, model_validatorimport cadquery as cqclass BracketSpec(BaseModel):length: float = Field(gt=0)width: float = Field(gt=0)thick: float = Field(gt=0)radii: list[float]positions: list[tuple[float, float]]@model_validator(mode="after")def check_spec(self):if len(self.radii) != len(self.positions):raise ValueError("Number of hole radii does not match the number of hole positions")if any(r <= 0 for r in self.radii):raise ValueError("Hole diameter must be greater than 0")half_l, half_w = self.length / 2, self.width / 2for r, (x, y) in zip(self.radii, self.positions):if abs(x) + r > half_l or abs(y) + r > half_w:raise ValueError("Hole position exceeds the boundary or edge margin is insufficient")for i in range(len(self.positions)):for j in range(i + 1, len(self.positions)):(x1, y1), (x2, y2) = self.positions[i], self.positions[j]distance = ((x1 - x2) ** 2 + (y1 - y2) ** 2) ** 0.5if distance < self.radii[i] + self.radii[j]:raise ValueError("Hole positions overlap")return selfdef build_bracket(params: dict):spec = BracketSpec(**params)base = cq.Workplane("XY").box(spec.length,spec.width,spec.thick,centered=(True, True, False),)for r, (x, y) in zip(spec.radii, spec.positions):cutter = (cq.Workplane("XY").center(x, y).circle(r).extrude(spec.thick + 2).translate((0, 0, -1)))base = base.cut(cutter)return base.val()
Export and rollback strategy:
try:model = build_bracket(params)cq.exporters.export(model, "bracket.step")except Exception as err:raise RuntimeError("Generation failed, previous model version is retained") from err
In engineering practice, generate into a temporary directory first and atomically replace the official file only after geometric validation passes. This prevents a failed model from overwriting a usable version.
The value of an LLM here is converting natural language requirements into executable scripts. For example, a designer types “design a bracket 200 long, 120 wide, 10 thick, with a Ø8 hole at each end”, and the model can produce a parameter object, call the CAD API, run the modeling step, and adjust parameters when the result is abnormal. Typical failures include overlapping holes, constraint conflicts, missing parameters, and insufficient edge margin. Address these with two layers of checks: the model side performs parameter schema validation and geometry self-checks, while engineers on the human side review critical dimensions, assembly relationships, and manufacturing requirements.
3. How Do You Make a Model Understand Sketches and Drawings? Doing Multimodal CAD
Key point first: the key to multimodal CAD is not generating geometry directly, but first extracting geometric intent, dimensions, and constraints from sketches, drawings, point clouds, or rendered images, and then entering the parametric modeling pipeline.
CAD work is inherently multimodal. Designers express requirements through text, sketches, 2D engineering drawings, 3D models, point clouds, and rendered images. A text-only LLM struggles to interpret these inputs fully, so three capabilities are needed: multimodal encoding, cross-modal alignment, and generation validation.
Implementation usually includes the following steps:
- Multimodal encoding: map images, text, and point clouds into a shared semantic space.
- Entity recognition: identify features such as holes, slots, chamfers, plate thickness, patterns, and assembly relationships.
- Parameter extraction: convert sketches or drawings into structured parameters such as length, width, hole diameter, and position.
- Geometry generation: call the CAD API to create the model.
- Automated validation: check geometric plausibility, interference, wall thickness, hole diameter, and tolerances.
To build this capability, enterprises can rely on cloud-based general-purpose LLM services, object storage, and containerized inference platforms to handle model training, deployment, scaling, and log auditing, rather than building all the underlying infrastructure themselves.
Multimodal scenarios also need safeguards against misinterpretation. A hand-drawn sketch, for example, may lack dimensions; the model should not fill them in on its own, but should flag the missing items and request human confirmation. For critical parts, keep a human-in-the-loop mechanism of “AI suggestion + engineer confirmation”.
4. Why Don’t General-Purpose LLMs Understand Enterprise Standards? Using RAG and Knowledge Graphs
Key point first: general-purpose LLMs do not inherently understand enterprise modeling standards, material libraries, tolerance standards, or design conventions. To make the output usable, the output must be grounded in enterprise industrial data and turned into reusable design knowledge assets.
Industrial data accumulation can proceed along three lines:
- Historical model corpus: turn existing CAD files, scripts, parametric templates, and change orders into a searchable knowledge base.
- Constraint knowledge graph: structure rules such as minimum wall thickness, interference checks, material compatibility, hole-to-edge distance, and machining accessibility.
- Retrieval-augmented generation (RAG): before generating a script, the model first retrieves similar historical cases, design standards, and a standard-parts library, which reduces generation out of thin air.
The value of RAG is that the model answers based on private enterprise knowledge rather than relying on generic memory. This both raises the professionalism of the output and lowers compliance risk. Enterprises should also establish a knowledge update mechanism that version-manages standard changes, process changes, and error cases, so that outdated standards do not contaminate new generation results.
5. How Do You Build an Integrated 2D + 3D + Cloud Design Architecture?
Key point first: the goal of coordinating 2D, 3D, and cloud is not to stack technologies, but to form a closed design loop with a unified entry point, unified data, unified permissions, and unified versions.
An integrated architecture typically includes three capability layers:
- 2D and 3D collaboration: 2D engineering drawings and 3D models are linked in both directions, so one change syncs automatically and version divergence is avoided.
- Cloud collaboration: object storage, message queues, version management, and a permission system support multi-user collaboration, centralized control, and automatic backups.
- Elastic compute: heavy compute tasks such as large-scale assemblies, simulation, and rendering are scheduled to a container platform and scaled on demand.
Cloud collaboration should not stop at elastic compute; it must also meet a minimum engineering bar: observable cost, least-privilege permissions, traceable versions, encrypted data, and auditable operations. An LLM can serve as the unified entry point: users issue instructions in natural language, and the model orchestrates 2D drafting, 3D modeling, simulation, reporting, and collaboration tasks.
6. How Does Physical AI Connect Digital Design to the Physical World?
Key point first: Physical AI is not about AI replacing designers. It moves CAD from “describing shape” to “predicting behavior, guiding manufacturing, and feeding back for optimization”.
The technical path of Physical AI splits into three layers:
- Digital modeling layer: AI generates precise, manufacturable, simulatable geometry.
- Physical simulation layer: simulation of mechanics, thermal behavior, fluid flow, electromagnetics, and similar disciplines validates how the model performs under real physical conditions.
- Virtual-physical mapping layer: the model is connected to sensors, manufacturing equipment, test data, and the operating environment, forming a closed loop of “design – manufacture – feedback – optimize”.
Physical AI has clear boundary conditions. Simulation accuracy depends on material parameters, boundary conditions, mesh quality, and validation data; calibration data must be traceable, otherwise model output cannot be trusted; sensor noise affects closed-loop decisions, so filtering, confidence evaluation, and outlier removal are needed; manufacturing tolerances change the actual assembly result, so the model should support tolerance analysis; and closed-loop data interfaces must meet requirements for timestamps, quality codes, equipment protocols, and data consistency.
In engineering practice, Physical AI is best introduced from a single product or a single process step, for example strength prediction for structural parts, injection molding defect analysis, weld distortion compensation, or production line assembly feedback. Only when simulation, sensors, manufacturing execution systems, and CAD data form stable interfaces can Physical AI move from demonstration to production.
7. What Acceptance Metrics and Checklists Are Needed for Engineering Rollout?
Key point first: AI-enabled CAD should start with high-frequency, low-risk, easy-to-verify repetitive tasks, and iterate based on metrics.
Suggested pilot tasks include bulk parameter edits, template generation, standard-part selection, engineering drawing annotation assistance, report output, and basic interference checks. Establish a baseline before the pilot, such as manual effort under the traditional process, success rate of existing scripts, number of rework cycles, and number of human review rounds.
Recommended rollout checklist:
| Check item | Acceptance criterion | Suggested method |
|---|---|---|
| Generation success rate | Share of scripts or models that run successfully on the first attempt | Statistics per task batch |
| Geometric correctness rate | Share that passes the geometry kernel, interference, wall thickness, and dimension checks | Automated validation + manual spot checks |
| Human intervention rate | Share of tasks that require manual modification of parameters, scripts, or models | Log number of modifications and reasons |
| Parameter compliance rate | Share meeting constraints such as minimum wall thickness, hole-to-edge distance, and material rules | Rule engine checks |
| Rework rate | Share of generated results returned or redone | Compare against the baseline |
| Access control | Least privilege, role isolation, and operation auditing are in place | Permission matrix + log audit |
| Data masking | Sensitive drawings, customer information, and process parameters are removed | Masking rules + spot checks |
On risk control, focus on four types of problems:
- Geometric correctness risk: LLMs can produce hallucinated scripts, so CAD kernel validation, constraint solving, and automated testing must be introduced.
- Professional accuracy risk: use RAG, rule engines, and expert review together as constraints, to avoid output that violates industry standards.
- Data and permission risk: industrial data involves intellectual property, so private or hybrid deployment, access control, and audit logs should be used.
- Human-in-the-loop risk: critical design decisions must still be confirmed by engineers; position AI as an efficiency assistant rather than an unmanned black box.
8. One Checklist: How Should an Enterprise Choose a Technology Path?
Before a full rollout, enterprises can use the following checklist to judge the applicable path:
| Technology path | Suitable scenarios | Main benefits | Main risks |
|---|---|---|---|
| Traditional CAD | One-off complex designs, scenarios with strong human experience | High accuracy, strong controllability | A lot of repetitive work |
| AI-assisted scripting | Bulk parameter edits, template generation, standard-part reuse | Shorter time on repetitive tasks | Generation errors, constraint conflicts |
| RAG augmentation | Industries with strong standards and abundant historical cases | Better compliance with professional standards | Outdated knowledge, permission leaks |
| Physical AI | Digital twins, manufacturing feedback, simulation closed loops | Connects design with the physical world | Inaccurate simulation, complex data interfaces |
The selection principle should be: first validate script generation and parameter validation on low-risk tasks, then use RAG to strengthen domain knowledge, and finally advance Physical AI once stable data and a simulation foundation are in place.
Conclusion
The combination of LLMs and CAD is shifting the design process from manual, feature-by-feature construction to intent-driven generation. Scripted parametric modeling provides the programmable handle, multimodal understanding handles mixed text-and-graphics inputs, industrial data accumulation gives AI professional judgment, 2D + 3D + cloud coordination forms a unified design entry point, and Physical AI further connects digital models to physical manufacturing.
For enterprises, the real focus of adoption is not chasing concepts but building a verifiable, rollback-capable, auditable engineering system: start with high-frequency, low-risk tasks, define baseline metrics, complete the permission and masking mechanisms, and then gradually extend to complex assemblies, simulation, and physical closed loops. Only then can the practical value of LLM-powered CAD be released in a steady way.
