Code-Driven 3D Video Production: From a Single Prompt to an Editable Short Film

In recent years, generative AI has matured rapidly in text, image, and software development. Video generation is also beginning to move from “generating a clip” toward “generating a complete project.” One representative direction is the use of intelligent coding models to automatically write 3D scene scripts from natural-language descriptions, completing modeling, animation, camera work, lighting, rendering, and post-production compositing to output a short film with a narrative structure.

The value of this approach is not merely “letting a model generate video.” It reorganizes the traditional 3D production pipeline into engineering tasks that can be described by code and executed automatically. Instead of clicking through modeling, lighting, and keyframe setup one step at a time, creators can first describe the story and visual goals, and then have the model convert those ideas into scene structures and executable code.

1. From Video Generation to Project Generation

Traditional video generation models usually output pixel sequences directly. They are easy to start with, but they have limitations in character consistency, object relationships, camera continuity, and later editing. If a generated result does not meet expectations, it often has to be regenerated, making it difficult to replace just one object or adjust a single shot.

Code-driven 3D video generation takes a different approach: the model first creates an editable scene project, and a rendering engine then outputs the video. The project typically contains the following objects:

• Scene geometry, such as desks, walls, windows, and lamps;
• Reusable character or prop models;
• Camera, lighting, and material definitions;
• Character actions, object motion, and timelines;
• Shot changes, depth of field, exposure, and color parameters;
• Audio tracks, subtitles, and end credits.

The final video is therefore the result of executing the project, not the only artifact. After generation, creators can continue to modify colors, shot duration, action pacing, or environmental layout. This editability is a major advantage of procedural 3D production.

2. How a Prompt Is Broken Down into Production Tasks

Natural-language descriptions are often abstract. For example: “Create a nighttime short film at a work desk, with a character moving near a device, and the view outside the window gradually shifting from dusk to dawn.” For a system to execute this, the prompt needs to be broken into multiple layers.

1. Story Layer

First, define the narrative mainline, including time changes, spatial relationships, and event order. For example:

1. A shot shows the complete environment;
2. A device lights up and the image enters the main plot;
3. A character enters from the edge of the frame;
4. The character interacts with objects on the desk;
5. Ambient light changes over time;
6. The camera gradually pulls back and ends on a sunrise or title screen.

The story layer determines what the video is about and provides the basis for shot planning.

2. Scene Layer

Next, list the entities in the scene and assign attributes to each entity:

• Geometry;
• Size and position;
• Material and color;
• Whether it participates in animation;
• Whether it casts shadows;
• Whether it needs collision or occlusion relationships.

For desk-based scenes, basic geometry can be used to build the main forms first, followed by bevels, materials, and textures to improve realism. For a short film, models do not necessarily need cinematic precision, but silhouettes must be clear, proportions must be consistent, and there should be no obvious clipping in the main shots.

3. Time Layer

The time layer turns a static scene into video. It usually contains keyframes and state changes, such as:

• A device screen lights up at a certain frame;
• A character moves from the desk edge to the center over a period of time;
• Buttons or indicator lights change briefly;
• Light outside the window shifts from warm tones to cool tones and then to morning light;
• The camera follows a preset path to push in, orbit, and pull back.

The model needs to convert these descriptions into an explicit timeline. Otherwise, different events may occur at the same time, leaving the image without pacing.

3. Automatic Modeling Is Not Unconstrained Generation

Intelligent models can write programs to create objects, but 3D scenes still follow clear engineering rules. A stable generation pipeline usually needs a “scene specification.”

Coordinate and Scale Specifications

All objects should use a unified coordinate system and unit of length. For example, define the ratio between desk height, device size, and character height to avoid problems such as an oversized character or an undersized keyboard. Each object in the scene should preferably have an independent name so it can be modified later through code.

Material and Lighting Specifications

Materials should be reused according to object category. Metal, plastic, glass, wood, and emissive screens have different reflection characteristics and should not be distinguished by color alone. Lighting also needs to consider the relationship between key light, fill light, rim light, and ambient light.

In a “dusk–night–morning” scene, time changes should not be expressed only by changing the background color. The following should also be adjusted in sync:

• Light source intensity;
• Light source color temperature;
• Brightness outside the window;
• Shadow softness;
• Reflections from the screen onto surrounding objects;
• Camera exposure and overall tone.

Only when multiple factors change together will the audience perceive a natural passage of time.

4. Camera Design Determines the Quality of the Final Film

High-quality 3D models do not guarantee that a video will look good. Whether a short film creates a complete viewing experience depends heavily on camera design.

Build a Shot List

Before formal generation, a shot table can be created first:

Shot	Duration	Framing	Main action	Visual purpose
1	4 sec	Wide shot	Show the space	Establish the environment
2	5 sec	Medium shot	Device lights up	Introduce the event
3	6 sec	Close shot	Character activity	Emphasize the subject
4	5 sec	Close-up	Object interaction	Provide detail
5	8 sec	Orbit	Show relationships	Enhance spatial sense
6	7 sec	Long shot	Light changes	Advance time
7	6 sec	Pullback	Scene resolution	Create an ending

The number of shots should not increase without limit. Too many shots make the generated code more complex, increase rendering time, and can easily fragment the pacing. For a short film under one minute, a dozen or so shots can already form a relatively complete visual narrative.

Camera Paths Need to Be Explainable

Camera movement should preferably use explicit paths rather than random jitter. Common approaches include:

• Pushing forward along a straight line;
• Making a small orbit around the subject;
• Gradually pulling back from a local close-up;
• Observing from a fixed position first, then slowly panning;
• Using focal length changes to create spatial depth.

Camera speed should match event intensity. When a character is only moving slightly, the camera should not rotate quickly. When an important action occurs, a brief close-up can also reinforce attention.

5. Character Animation and Object Interaction

Character motion is the part most likely to expose problems in a 3D short film. Simple translation and rotation can be achieved with keyframes, but complex actions also need to consider center of gravity, contact points, and occlusion relationships.

For example, when a character walks near a desk, at minimum the following must be handled:

1. The height relationship between the character’s feet and the desk;
2. Whether the movement path passes through other objects;
3. Whether the action speed changes abruptly;
4. Whether the character’s orientation remains continuous when turning;
5. Whether hand movements match the device position;
6. Whether the character maintains spatial consistency after a shot change.

For short films, low-complexity but stylistically consistent actions can be prioritized. Rather than pursuing highly detailed gait, it is more important to ensure that the character’s size, color, orientation, and positional relationships remain stable across different shots.

6. Rendering and Cost Control

Full 3D rendering is often more time-consuming than generating scripts. Rendering cost is mainly affected by the following factors:

• Output resolution;
• Total video frames;
• Samples per frame;
• Ray tracing complexity;
• Material reflections and transparency;
• Shadow quality;
• Scale of computing resources used.

In engineering practice, a staged strategy can be used. The first stage uses low resolution and low samples for quick previews to check camera, motion, and composition. The second stage performs medium-quality tests only on key shots. The third stage outputs the final version.

If the video frame rate is 24 frames per second, a one-minute video requires processing more than 1,400 frames. Even if single-frame rendering time is short, the overall task accumulates quickly. Therefore, parallel processing across multiple computing devices is a common approach. Different shots or different frames can be assigned to multiple compute nodes and then merged in timeline order.

Cloud computing resources are suitable for short-term, peak-style tasks, especially for individual developers or small teams. Actual costs need to be evaluated separately based on resolution, rendering duration, device specifications, and concurrency. They cannot be judged by video length alone.

7. Audio and Post-Production Compositing

If only visuals are generated, the final film often feels incomplete. Sound can be composed of three types of elements:

• Ambient sound, such as wind, room tone, and distant city noise;
• Action sound, such as key presses, device startup, and object movement;
• Music or ambience, used to control emotion and pacing.

Sound does not necessarily require complex recording. Basic sound effects can also be synthesized programmatically and then mixed using automation tools. The key is audio-visual synchronization: when a device lights up, there should be a corresponding startup sound; when a character acts, there should be subtle spatial feedback; when the shot enters nighttime, music density can be moderately reduced.

The post-production stage also includes color grading, subtitles, opening and ending credits, and loudness normalization. It is important to note that color grading should not mask modeling or lighting problems. The correct approach is to first ensure reasonable exposure in the original image, then apply stylistic treatment.

8. Current Limitations

“Generating a 3D short film from one sentence” may appear highly automated, but several practical limitations remain.

First, complex narratives still require human decomposition. A model can generate shots and code, but it may not accurately understand implicit emotional pacing and directorial intent.

Second, consistency is difficult to maintain in long videos. The longer the video, the more likely character, material, lighting, and spatial relationships are to drift. Short films are therefore better suited to current automated 3D pipelines.

Third, code needs debugging. Automatically generated scripts may contain object naming conflicts, coordinate errors, incompatible rendering parameters, or overlapping animation timelines. Developers should keep logs, execute in stages, and add validation for key objects.

Fourth, the final result still depends on aesthetic judgment. Models are good at executing explicit rules, but composition, pacing, style, and narrative focus still require human oversight.

9. An Engineering Workflow Suitable for Implementation

A reusable production workflow can be summarized as follows:

1. Use natural language to define the story, style, and target duration;
2. Automatically generate a scene list and shot list;
3. Create basic geometry, materials, lighting, and characters;
4. Generate an independent animation script for each shot;
5. Preview at low resolution and check spatial relationships;
6. Fix issues in motion, camera, and lighting;
7. Batch-execute high-quality rendering;
8. Composite audio, subtitles, and opening and ending credits;
9. Output the final film and save the editable project file;
10. Archive prompts, scripts, and rendering logs.

The core of this workflow is not to eliminate human involvement entirely, but to free people from repetitive operations and shift their focus to requirement definition, quality review, and creative control.

Overall, intelligent coding is moving 3D video production from “manual operation of software” toward a new model of “describing goals, generating projects, and executing automatically.” It is especially suitable for product concept demonstrations, educational animation, advertising samples, game trailers, and personal creation. As models improve their understanding of spatial structure, timelines, and visual language, future video production may no longer center on a single generation result, but on a complete digital scene that is editable, reusable, and continuously iterable.