Making Explainer Videos by Writing Code: A Code-First Video Production Pipeline

1. From Cutting to Writing: Video Production Is Migrating

When you see a clean, well-paced explainer animation on social media — tidy captions, clear narration, crisp data visualization — most people’s first assumption is “a professional editor made this.” If I told you that the entire production process involved not a single mouse drag on a timeline, and that an AI coding assistant generated the animation code directly from a natural-language description and rendered the final video, you might find that hard to believe.

The part that actually deserves attention here isn’t “AI replaces video editors.” It’s that video has changed from a craft into an engineering artifact that is programmable, version-controllable, and reproducibly batch-produced.

The core asset in a traditional editing workflow is the project file plus source footage: the longer the timeline, the higher the cost of change, and a single copy edit can trigger a cascade of rework. The core assets of code-driven video are script data + component code + render configuration — all three are text, which means they fit natively into Git version control, code review, and automated pipelines. Changing one line of narration means editing a field in a data file. Adjusting the pacing of a transition means editing an interpolation curve parameter.

This produces three qualitative shifts:

• Reproducible: the same input renders pixel-identical output on any machine. You no longer depend on “the project file on that one workstation.”
• Batch-producible: once a video’s production logic has been distilled into components and templates, switching topics is just swapping data. Daily publishing and multilingual distribution become realistic.
• Reviewable: animation lives in the repository as code, so collaboration shifts from “send me the project file” to “open a PR.”

Here’s how the whole approach fits together.

2. A Four-Layer Architecture: Decomposing a Video into Four Orthogonal Concerns

Don’t start by writing components. Establishing layers first is what determines whether code-driven video can be maintained long term. I usually split it into four layers, each with a single responsibility and a clear interface:

Layer	Responsibility	Output	Change frequency
Narrative	What to say, how to split it, how long each beat lasts	Storyboard data (JSON / YAML)	Highest
Presentation	What the frame looks like, how it moves	React-style frame-driven components	Medium
Audio	Voiceover, subtitles, rhythm alignment	Audio files + subtitle timecodes	Medium
Rendering	Frame-by-frame rasterization, encoding, muxing	Video file	Lowest

The real value of layering is decoupling. Copy belongs to the content team, components belong to engineering, subtitles and voiceover run on their own pipeline — and all three communicate through a structured contract (the storyboard JSON). Nobody needs to understand every detail of the other three layers.

3. The Narrative Layer: Turning Copy into Machine-Readable Data

This is the step AI is best at: turning a colloquial explanation request into a well-structured script. But don’t let it output finished frames directly. Constrain the output format first. Ask the model to produce a shot list in which each shot explicitly states what it says, what it emphasizes, how long it holds, and what kind of visual it needs.

{
  "meta": { "title": "Why Rockets Use Staging", "fps": 30, "durationInFrames": 1080 },
  "scenes": [
    { "id": "s1", "type": "title",   "narration": "A rocket has to escape the atmosphere. Why is it built in stages?", "emphasis": ["staging"], "minDuration": 3.0 },
    { "id": "s2", "type": "bullet",  "narration": "The core tension is mass: propellant dominates the budget, and the structure itself is dead weight being carried along.", "emphasis": ["mass", "structure"], "minDuration": 4.5 },
    { "id": "s3", "type": "chart",   "narration": "Drop the empty shell and the remaining thrust efficiency improves sharply.", "emphasis": ["thrust efficiency"], "minDuration": 4.0 }
  ]
}

A few practical points:

1. Make minDuration mandatory. This is the single most effective defense against AI-generated “one sentence flashes by in 0.3 seconds.” In narrated video, on-screen dwell time must exceed the viewer’s reading time.
2. Drive visual emphasis with an emphasis field. Rendering those keywords as highlighted blocks or scaled motion at render time is far more reliable than letting the model improvise.
3. Whitelist scene type values. Allow only a limited set — title / bullet / chart / diagram — to avoid generating visual descriptions that can’t be rendered.
4. Separate the spoken script from the storyboard. First have the model write a pure voiceover script (with a proper narrative arc, readable aloud), then have it split that script into shots. Doing both in one pass usually degrades quality.

The abstraction here is: AI handles expression, humans handle constraints. The clearer your structured constraints, the higher the usable-output rate.

4. The Presentation Layer: Animation as a Frame Function

The programming paradigm for code-driven video can be stated in one sentence: the frame is a pure function of the frame number.

f(frame) => style

At each frame, a component receives the current frame number and derives all visual properties for that frame through interpolation. There’s no “playback” concept and no state machine, so the renderer can render any frame in any order and in parallel. That’s exactly why this model suits batch rendering better than interactive animation frameworks.

// Frame-driven component: all animation is derived from the current frame number
const TitleScene = ({ text, emphasis, durationInFrames }) => {
  const frame = useCurrentFrame();
  const progress = interpolate(frame, [0, 24], [0, 1], { extrapolateRight: 'clamp' });
  return (
    <AbsoluteFill style={{ backgroundColor: '#0b1020', justifyContent: 'center' }}>
      <h1 style={{
        fontSize: 72,
        color: '#f5f7ff',
        opacity: progress,
        transform: `translateY(${(1 - progress) * 28}px)`,
      }}>
        {highlight(text, emphasis)}
      </h1>
      <ProgressHint frame={frame} total={durationInFrames} />
    </AbsoluteFill>
  );
};

The reuse payoff from componentization is immediate: chart entrance animations, number count-ups, progress bars, caption bars, and transition masks, once written as shared components, mean any new video is just assembly. Chart components are especially worth investing in, because in data-visualization explainers roughly 80% of the frames are variants of bars, lines, pies, or flow diagrams.

Pair that with a Sequence schedule that maps storyboard data onto a component sequence:

<Sequence durationInFrames={totalFrames}>
  {scenes.map((s, i) => (
    <SceneRenderer key={s.id} scene={s} startFrame={starts[i]} />
  ))}
</Sequence>

Pitfall warning: fonts must be fully loaded and verified before rendering starts, otherwise the first frame falls back to a default typeface and you get cross-frame style jumps. Also standardize your coordinate system and frame-rate baseline (fps governs every duration conversion) — if the frame rate changes midway, the entire timeline has to be recomputed.

5. The Audio Layer: Aligning Voiceover and Subtitle Timecodes

For narrated video, the quality floor is usually set by the audio. The core engineering problem is: how do you get accurate subtitle timecodes without a human listening?

The recommended approach is a closed loop of “synthesize, measure, then align”:

1. Call a cloud speech synthesis capability (for example, the text-to-speech products in Baidu AI Cloud’s general-purpose speech technology category) to synthesize each storyboard beat as a separate audio clip, and get its duration back.
2. Write the measured durations back into the storyboard data, overwriting the estimated minDuration — make the visuals wait for the audio, not the audio wait for the visuals. This is what makes the pacing of narrated video feel natural.
3. Concatenate the clips into a full audio track along the timeline and generate SRT / ASS subtitle files, where timecodes are derived directly from cumulative duration with zero drift.
4. Overlay a volume envelope or waveform strip on the visuals for the “the picture reacts when the voice speaks” quality touch.

# Derive the timeline backward from voiceover durations
t = 0
for scene in scenes:
    audio = tts.synthesize(scene.narration)   # returns wav + duration
    scene.start   = t
    scene.end     = t + ceil(audio.duration * fps)
    scene.audio   = audio.path
    t = scene.end
write_srt(scenes)          # subtitle timecodes generated automatically

The payoff multiplies if you need multilingual versions: change the language parameter, re-synthesize, and both the timeline and subtitles recompute automatically while the visual components stay untouched.

6. Using an AI Coding Tool to Lock the Workflow into a “Skill”

Once the first five steps work, the real efficiency jump happens in the sixth: turning domain experience into rules that an AI coding assistant follows consistently.

Most mainstream AI coding tools support extension mechanisms for “skills / rules / custom instructions.” At their core, these place project-specific knowledge into the repository as Markdown files that the model loads automatically on every conversation. For video production, a good skill file roughly contains:

# Video Generation Skill (video-render)
## Inputs
- Voiceover script (plain text) or storyboard JSON
- Speech-rate preference, target duration, aspect ratio
## Outputs
- build/script.json (storyboard data)
- src/scenes/*.tsx (scene components)
- public/subtitles/*.srt
## Hard Rules
- No more than 18 characters of body text on a single screen; split the shot if exceeded
- Minimum shot duration 2.5s, maximum 8s
- Every emphasis keyword must have corresponding highlight treatment
- Never hardcode durations inside components; always read them from props
- After generation, `check:frames` must pass; self-repair before delivery if it fails
## Validation Commands
- npm run check:frames   frame/timeline consistency check
- npm run preview        low-resolution quick preview
- npm run render         final render

The value of a skill file isn’t “making the model smarter.” It’s turning things you’d otherwise re-explain out loud into an executable contract. The first setup might take half an hour; every video after that benefits from it. Pair it with a clear project directory structure:

video/
├─ skill.md                # skills and rules
├─ scripts/topic-a.json    # storyboard data (maintained by content)
├─ src/components/         # shared animation components (maintained by engineering)
├─ src/scenes/             # scene assembly
├─ public/audio/           # voiceover
├─ public/subtitles/       # subtitles
└─ dist/                   # render output

Clear directory ownership boundaries, combined with the path conventions declared in the skill file, substantially reduce the odds of the model dropping files in the wrong place.

7. Engineering: From Local Preview to Stable Output

Rendering is the only time-consuming stage in the pipeline, so it has to be engineered properly:

• Two-tier preview: render a 480p or sampled-frame preview first to confirm pacing and copy, then render at full resolution. This eliminates roughly 90% of the waiting.
• Parallel rendering plus caching: frame-level rendering is inherently parallel, so set concurrency based on CPU core count. Cache frames that are already rendered and unchanged, so editing one line of copy doesn’t force a full re-render.
• Validation up front: write checks for frame-count consistency, subtitle timecode monotonicity, and font availability, and hook them into commit hooks or CI — so you don’t discover a timeline misalignment at 90% progress.
• Separate artifacts from archives: write finished videos to object storage and metadata (storyboard JSON, subtitles, version number) to a database, so you can trace back which script produced a given version.
• Shard long videos: render content longer than a few minutes in segments and concatenate, avoiding single-process memory and timeout risk.

# Illustrative local validation and render flow
npm run check:frames        # timeline / subtitle consistency check
npm run preview             # quick preview to confirm pacing
npm run render -- --concurrency 4 --out dist/final.mp4

8. Quality, Compliance, and Accessibility

Code-driven production makes batch output easy, which raises the stakes on quality control:

1. Visual consistency: route font sizes, color palettes, and spacing through design tokens. Don’t write magic numbers in every component.
2. Accessibility: aim for a body-text to background contrast ratio of at least 4.5:1, keep clear padding in the bottom safe area for subtitles so they aren’t obscured, and never convey information by color alone.
3. Asset licensing: background images, sound effects, and fonts all need explicitly licensed sources. Fonts are the easiest thing to overlook.
4. Content labeling: where AI-generated visuals and voiceover are involved, add whatever disclosure the platform you publish on requires. This is a compliance floor, not an option.
5. Fact-checking: AI-written explainer copy is highly prone to numeric and causal errors, so it must be reviewed by a human — especially anything involving data charts. The more polished the visuals, the more convincingly they carry an error.

9. Troubleshooting Quick Reference

• First-frame font jump → unify the render entry point on await loadFont(), and abort on failure rather than falling back.
• Subtitles out of sync with speech → never estimate durations; always use the measured duration of the synthesized audio.
• Copy too long and overflowing → cap per-screen character count in the skill rules so the model splits shots at storyboard generation time.
• One copy edit triggers a full re-render → enable frame caching, or render in per-scene shards.
• Model improvisation causing rework → validate storyboard output strictly against a JSON Schema, and if validation fails, have the model self-repair.

10. Closing Thoughts

The real point of code-driven video is returning the creator from the operation layer to the expression layer. You stop caring how a layer aligns or how a keyframe is dragged, and instead describe “hold here for a beat,” “emphasize this sentence,” “count this data up.” Execution goes to the model and the renderer; judgment stays with the human.

One thing to be clear-eyed about: this pipeline currently suits highly structured content best — data visualization, explainer narration, product walkthroughs, multilingual distribution. Built on data and templates, it is inherently weak at emotional editing, documentary narrative, and fine-grained manual color grading. Acknowledging the boundary and choosing the right use case is where the efficiency gains actually start to materialize.