Code as an Interface: AI-Generated 3D Video and the Aesthetic Barrier

1. From Writing Scripts to Producing a Short Film

Recently, a number of surprising examples have appeared in developer communities: users provide only a natural-language description, and the model generates thousands of lines of scripts. These scripts directly call the programming APIs of professional 3D software and complete modeling, materials, rigging, keyframe animation, lighting, and final rendering without touching the graphical interface. The final output is a playable short film.

The part that deserves attention from an engineering perspective is not that “the model can write very long code.” Long code itself is not difficult. The real difficulty is that the target system called by this code is a highly complex, stateful, and parameter-sensitive professional toolchain. In the past, this kind of work relied heavily on the operator’s software proficiency and aesthetic judgment. It was a typical “hands-on” skill. But once it is expressed as code, the executor of the entire pipeline changes from a human operator into a program that can be batch-scheduled and scaled elastically in the cloud.

In other words, AI has not swallowed the craft of 3D design. It has swallowed the part that has already been interface-driven.

2. Why “Code” Becomes the Most Universal Interface Layer

2.1 Interfaces Have Already Converged

Over the past decade or more, mainstream professional software has gradually become programmable. This includes script interpreters, command-line invocation, headless execution modes, and scene file formats that can be driven by external programs.

This means rendering is essentially reduced to a pure function:

Input: scene description (geometry / materials / lighting / camera / timeline)
Output: pixels + metadata (render time, VRAM usage, warning logs)

A pure function is the form most easily handled by machines. It is stateless, repeatable, parallelizable, and assertable.

2.2 3D Modeling Is Naturally Parametric

A 3D scene can be compressed into a finite parameter space: numerical values for coordinates and rotations, physical properties of materials, color temperature and intensity of lights, focal length and aperture of cameras. These quantities are mathematically continuous and composable, which makes them well suited to language-model output.

Compared with vague goals such as “does it look good enough?”, instructions such as “place the key light 45 degrees to the left front of the subject, use a cool color temperature, and set roughness to 0.35” can be written down directly.

2.3 Executable Code Means Free Feedback

When code runs, it produces errors. Rendering produces timing data. Screenshots expose clipping, out-of-bounds objects, and overexposure. This feedback does not require manual labeling. It naturally forms a closed optimization loop, which is an advantage that pure image-generation paths are unlikely to obtain in the short term.

2.4 Low Migration Cost

A production pipeline expressed as code can move from a single machine to the cloud. Intermediate artifacts can be stored in object storage, rendering tasks can be sharded through message queues, and compute capacity can scale elastically through container platforms. Failure rates and per-frame cost can be observed through logs and monitoring.

Engineering capability is directly stacked onto creative capability.

3. Why Aesthetics Is a Blind Spot in Benchmarks

Most evaluation systems favor decidable answers: whether unit tests pass, execution success rate, exact-match rate, or feature coverage rate. Aesthetics falls on the opposite side.

• No gold standard: the same scene can have many equally reasonable solutions, and there is no single correct answer.
• Conflicting objectives: composition, rhythm, material realism, and color consistency often require trade-offs.
• Subjective but consensus-driven: judgment depends on the viewer’s experience, yet it can be perceived highly consistently by a group of people.
• Benchmark drift exists: once viewers have seen higher-quality output, works of the same level may suddenly seem mediocre. This is not because the work quality has dropped, but because the viewer’s perception threshold has risen.

Therefore, the statement that “Model A has better aesthetics than Model B” is often not a model capability problem in practice, but a missing evaluation standard problem. The controllable approach is to engineer aesthetics: break it into constraints that can be written down, rules that can be checked, and sample sets that can be compared.

4. Turning Aesthetics into Executable Constraints

4.1 Start with a Reference Specification

Do not simply provide a reference image. First translate it into structured parameters: focal length, aperture, color temperature, number of key and fill lights, material roughness range, depth-of-field range, and the proportion of empty space in the frame. Once the specification is explicit, the model has a target it can align to.

4.2 Drive Intermediate Representations, Not GUIs

Let the model produce a structured scene description rather than an anthropomorphic “click-and-drag” sequence. Intermediate representations are easier to validate, easier to diff, and easier to roll back:

{
  "camera": {"focal_mm": 35, "aperture": 2.8, "target": "hero"},
  "lighting": [
    {"type": "key", "azimuth_deg": 45, "elevation_deg": 35, "cct_k": 5600, "power": 1.0},
    {"type": "rim", "azimuth_deg": 200, "elevation_deg": 55, "cct_k": 7200, "power": 0.6}
  ],
  "materials": {"hero": {"base_color": "#2E3B4E", "roughness": 0.35, "metallic": 0.1}},
  "timeline": {"fps": 24, "duration_s": 8, "keys": 6}
}

4.3 Generate in Stages Rather than One Shot

Use a staged workflow: layout → materials → lighting → animation → rendering. Each stage produces an artifact that can be independently validated. Low-resolution previews are used for rapid trial and error. Once the result is confirmed, the process moves into high-sampling final rendering. This can reduce rework cost to a fraction of the rendering cost.

4.4 Let Automation Replace Human Inspection

The following is a schematic skeleton for automated driving and validation:

def build_and_render(scene_spec: dict, preview: bool = True):
    scene = compile_scene(scene_spec)      # structural validation + schema validation
    assert_in_frame(scene)                 # whether the subject is cropped out
    assert_no_clipping(scene)              # whether clipping or out-of-bounds occurs
    if preview:
        return render(scene, samples=32, res=(960, 540))
    return render(scene, samples=512, res=(1920, 1080))
def iterate(spec, judge, max_rounds=5):
    best, best_score = None, -1e9
    for _ in range(max_rounds):
        frame = build_and_render(spec, preview=True)
        score, notes = judge(frame, spec)   # rule-based scoring + difference notes
        if score > best_score:
            best, best_score = frame, score
            snapshot(spec)                  # save the latest passing snapshot
        spec = apply_patch(spec, notes)     # targeted patch rather than full rewrite
    return build_and_render(best, preview=False)

The key design principle is that targeted patching takes priority over full rewriting. A full rewrite can randomly degrade parts that were already correct and makes it difficult to attribute the cause of the problem.

5. Engineering Implementation Checklist

• Determinism: fix random seeds, fix version numbers, and put scene files under version control to ensure that the same input can reproduce the same output.
• Interface allowlist: generate calls only for verified APIs, and validate schemas against nonexistent interfaces to suppress API hallucination.
• Stage-by-stage assertions: execute long scripts by stage, assert immediately after each stage, and avoid errors surfacing only at the end of the pipeline.
• Failure rollback: if any round of validation fails, roll back to the latest passing snapshot rather than continuing to add changes on top of incomplete output.
• Resource governance: store intermediate artifacts in object storage, shard rendering tasks through message queues, let container platforms scale compute elastically, and continuously observe failure rate, average per-frame cost, and retry rate through logging services and monitoring alerts.

6. The Boundary: What Gets Absorbed Is Not Judgment

The conclusion needs to be stated accurately. Work that can be expressed in code, called through interfaces, and automatically verified is being taken over by models at an accelerating pace. But goal definition, style trade-offs, and judgment about what is “good” remain on the human side, and their value is amplified, because when production speed is no longer the bottleneck, evaluation standards become the bottleneck.

For teams, the actionable conclusions are three:

1. Make creative workflows as interface-driven and parameter-driven as possible.
2. Turn aesthetic preferences into versionable specifications and rules.
3. Automate validation rather than relying on manual visual review every time.

What is truly being “swallowed” is never the industry itself, but the parts of the industry that have not yet been written as code.