Building a Short-Form Video Generation SaaS with AI in Three Hours: Task Orchestration in Practice

    This article explains how to build a short-form video generation SaaS prototype in three hours, focusing on the engineering work behind the model call: task orchestration, asset management, asynchronous execution, failure retries, resource costs, and delivery experience. It covers MVP scope, system architecture, task state design, pipeline decomposition, prototype scheduling, cost and daily capacity estimation, and the production hardening needed before launch.

    Short-form video generation is not just about calling a model API. What really determines whether a product can ship is often task orchestration, asset management, asynchronous execution, failure retries, resource costs, and delivery experience.

    For e-commerce teams, content marketers, and small and medium-sized businesses, a common requirement is: upload product images, enter a few selling points, and let the system automatically generate product promotion videos suitable for short-form video platforms. Users do not want to understand complex video editing workflows, nor do they want to manually handle voiceover, subtitles, transitions, and cover creation. They care more about three questions:

    • Is the total cost per video controllable?
    • How many tasks can the system process per day?
    • Can the entire process from images to a finished video be completed on a simple page?

    Therefore, when building this type of system, the goal should not merely be “make a video that can play.” The goal should be to build a complete automated production pipeline. This article breaks down the process based on a practical prototype plan.

    1. Define the Minimum Viable Product First

    The first version of a short-form video generation SaaS should not pursue complex features. The core workflow can be narrowed down to the following five steps:

    1. The user uploads one or more product images.
    2. The user fills in the product name, core selling points, and target audience.
    3. The system generates a video script, voiceover text, and shot plan.
    4. The backend asynchronously handles image processing, speech synthesis, subtitle generation, and video stitching.
    5. The user views task progress on the web page and downloads the finished video.

    This is essentially a typical system that follows the pattern of “input assets — generate intermediate results — compose final result — deliver file.”

    The first version can temporarily skip complex account systems, team collaboration, asset marketplaces, and detailed operational analytics. Instead, it should prioritize validating three things:

    • Whether users are willing to upload assets and wait for generation;
    • Whether the generated results are basically usable;
    • Whether the time and resource consumption per task are acceptable.

    Only after these three questions are validated is it necessary to expand into membership plans, batch tasks, and multi-template management.

    2. Overall System Architecture

    A basic version can use a frontend-backend separated structure, mainly including the following modules:

    1. Web Frontend

    The frontend handles asset upload, parameter input, task creation, status polling, and result preview. The page does not need to be complex, but it must clearly show task status, for example:

    • Waiting for processing;
    • Generating script;
    • Generating voiceover;
    • Composing video;
    • Succeeded;
    • Failed.

    If a task takes a long time, the frontend should not keep blocking and waiting for the API response. Instead, it should receive a task ID immediately after creating the task, and then obtain processing progress through polling or real-time notifications.

    2. Business Service

    The business service handles user request validation, task creation, permission checks, asset records, and result queries. It should not directly perform time-consuming video processing work; otherwise, when multiple users submit tasks at the same time, API threads can be occupied for long periods.

    The service layer needs to maintain at least the following data:

    1. Task ID
    2. User ID
    3. Original asset URL
    4. Video template
    5. Script text
    6. Voiceover file URL
    7. Subtitle file URL
    8. Final video URL
    9. Current status
    10. Failure reason
    11. Created time
    12. Completed time

    3. Object Storage

    Images, audio, subtitles, and videos are all file-based data and are suitable for storage in object storage. The database should only store file identifiers, access URLs, or metadata, not large binary content directly.

    This design has several benefits:

    • It reduces pressure on the business database;
    • It supports resumable uploads for large files;
    • It makes it easy to set temporary access permissions;
    • It allows lifecycle policies to be configured for different file types;
    • It facilitates subsequent integration with content moderation and distribution workflows.

    4. Message Queue and Task Workers

    Video generation is a typical asynchronous task. After a user submits a request, the business service writes the task to the database and publishes it to a message queue. Task workers read tasks from the queue and complete script generation, voiceover generation, subtitle generation, and video composition in sequence.

    This architecture decouples “user request speed” from “video processing speed.” Even if video generation takes tens of seconds or longer, the user-facing API can still return quickly.

    5. Logs and Monitoring

    During the prototype stage, teams often focus only on the success path. After the system is running, they discover that failures are difficult to diagnose. Therefore, at minimum, the system should record:

    • The start and end time of each task;
    • The time spent in each processing stage;
    • External service call results;
    • File download and upload failure information;
    • Retry counts;
    • Video composition commands and exit status;
    • The final failure reason.

    Logs are not only for troubleshooting but also for later cost analysis and capacity planning.

    3. A Task State Machine Is More Reliable Than a Simple Field

    Many initial systems use only a Boolean field to indicate whether a task is “completed.” This makes it impossible to determine the current progress after a task is interrupted. A more reliable approach is to design an explicit state machine:

    1. CREATED
    2. -> SCRIPT_GENERATING
    3. -> AUDIO_GENERATING
    4. -> SUBTITLE_PREPARING
    5. -> VIDEO_RENDERING
    6. -> SUCCEEDED

    Any stage can enter:

    1. FAILED

    If tasks support retries, a task can return from the failed state to the corresponding stage instead of starting over from the beginning. For example, if voiceover generation fails, there is no need to regenerate the script.

    The pseudocode is as follows:

    1. def process_task(task):
    2. try:
    3. update_status(task, "SCRIPT_GENERATING")
    4. script = generate_script(task.input_data)
    5. save_script(task.id, script)
    6. update_status(task, "AUDIO_GENERATING")
    7. audio = generate_audio(script)
    8. save_audio(task.id, audio)
    9. update_status(task, "SUBTITLE_PREPARING")
    10. subtitle = build_subtitle(script, audio)
    11. save_subtitle(task.id, subtitle)
    12. update_status(task, "VIDEO_RENDERING")
    13. video = render_video(task, audio, subtitle)
    14. save_video(task.id, video)
    15. update_status(task, "SUCCEEDED")
    16. except Exception as exc:
    17. record_error(task.id, str(exc))
    18. update_status(task.id, "FAILED")

    Production environments should also add idempotency controls. A task worker may consume the same message repeatedly due to network exceptions. Without idempotency design, this can generate duplicate files or duplicate charges. You can set a unique business key for each stage and check whether the stage result already exists before execution.

    4. How to Split the Video Generation Pipeline

    1. Script Generation

    Script generation needs to constrain the output format explicitly. It is not enough to ask the model to “write an ad copy.” Structured fields are more suitable:

    1. {
    2. "title": "Video title",
    3. "opening": "Content that attracts attention in the first two seconds",
    4. "scenes": [
    5. {
    6. "duration": 4,
    7. "visual": "Show the product appearance",
    8. "voiceover": "Voiceover content",
    9. "caption": "On-screen caption"
    10. }
    11. ],
    12. "ending": "Call to action"
    13. }

    Structured results are easier for downstream programs to consume and avoid the need for manual parsing after script generation. The prompt should clearly specify video duration, tone, audience, prohibited expressions, and output format.

    Content compliance also needs attention. Product promotion must not automatically generate absolute promises, false efficacy claims, or unverifiable comparative conclusions. The system can add rule checks after script generation to block sensitive words, exaggerated descriptions, and efficacy claims lacking supporting evidence.

    2. Voiceover Generation

    The voiceover stage needs to focus on text length, speaking rate, pauses, and audio duration. When script and audio duration do not match, subtitles and visuals can become misaligned.

    Common approaches include:

    • Limiting script word count based on target duration;
    • Splitting long text into sentences;
    • Saving the start and end time of each sentence;
    • Normalizing audio loudness;
    • Setting a reasonable volume balance between background music and voiceover.

    If the system supports multiple voices, voice selection should be a template configuration rather than hardcoded in the code. This way, adding new styles later only requires adding configuration.

    3. Subtitle Generation

    The simplest subtitles can distribute time evenly by sentence, but the result is usually not natural enough. A better approach is to generate a timeline based on speech segmentation results:

    1. 00:00.000 --> 00:02.300
    2. This is a product suitable for everyday use
    3. 00:02.300 --> 00:05.100
    4. Lightweight and easy to store, suitable for many scenarios

    After the subtitle file is generated, font, font size, outline, position, and safe area also need to be considered. Short-form videos are usually watched on mobile devices, so subtitles must not be placed too close to the edge of the frame or cover the main subject of the product.

    4. Video Composition

    Video composition can be abstracted into multiple media layers:

    • Background canvas;
    • Product images;
    • Zoom and pan animations;
    • Voiceover audio;
    • Background music;
    • Subtitles;
    • Opening and ending;
    • Watermark or brand mark.

    The first version does not need complex visual algorithms. Fixed templates can be used to complete “slow image zoom, shot switching, subtitle overlay, and audio mixing.” The advantage of templates is that they are stable, predictable, and easy to process in batches.

    5. How to Schedule Three Hours of Prototype Development

    If the goal is to complete a demonstrable version in a short period, it is recommended to proceed in the following order:

    First Hour: Complete the Main Path

    First implement the upload page, task creation API, task records, and result display. At this stage, a fixed template can be used. The key is to ensure that a user can submit one complete task.

    Second Hour: Integrate Asynchronous Processing

    Split script generation, audio processing, and video composition into background tasks, and add status queries and error records. Even if the interface is simple at this point, make sure tasks do not fail because of request timeouts.

    Third Hour: Complete Interactions and Exception Handling

    Add upload format validation, task progress display, failure retries, result preview, and download functionality. Run at least one round of testing on critical paths, including empty files, oversized images, invalid text, processing timeouts, and duplicate submissions.

    “Completing in a short time” here should be understood as completing a verifiable prototype, not directly reaching production-level stability. Before a real launch, systematic improvements to security, performance, and cost are still required.

    6. How to Estimate Cost and Daily Processing Capacity

    The cost per video is not just the model invocation fee. It should also include:

    • Text generation cost;
    • Voiceover generation cost;
    • Video rendering compute resources;
    • File storage cost;
    • Network traffic cost;
    • Extra consumption caused by failed retries;
    • Content moderation and monitoring resources.

    The following formula can be used for estimation:

    1. Total cost per video =
    2. text processing cost
    3. + audio generation cost
    4. + video rendering cost
    5. + storage cost
    6. + traffic cost
    7. + failed retry cost

    Daily processing capacity depends on the slowest stage and the achievable parallelism:

    1. Daily processing volume ≈ available processing time × concurrent executions ÷ average time per task

    This formula can only be used for preliminary estimation. In practice, queue backlog, resource peaks, failure rate, and rate-limiting policies also need to be considered. For batch tasks, it is recommended to set a maximum concurrency limit to avoid exhausting compute resources when a large number of tasks are submitted at once.

    7. Moving from Prototype to Production

    Before launch, at least the following capabilities should be added:

    Security

    • Restrict uploaded file types and sizes;
    • Perform content detection on images, audio, and video;
    • Use temporary authorization URLs for file access;
    • Prevent script injection and path traversal;
    • Apply rate limiting to user requests;
    • Block sensitive content and illegal promotional claims.

    Stability

    • Automatically reclaim timed-out tasks;
    • Retry failed tasks by stage;
    • Trigger alerts when the queue is backlogged;
    • Back up critical data regularly;
    • Clean up temporary directories during file processing;
    • Set timeouts and circuit breakers for external dependencies.

    Observability

    Metrics such as task success rate, average duration, duration of each stage, failure types, resource utilization, and queue length should be established. Only by continuously collecting this data can you determine whether the system bottleneck comes from text generation, audio processing, or video rendering.

    Conclusion

    The core value of a short-form video generation SaaS is not completing a single text generation or video composition task. It is organizing multiple capabilities into a stable, traceable, and scalable automated production pipeline.

    A qualified minimum system should have clear task states, reliable asynchronous execution, structured intermediate results, controllable file lifecycles, and complete failure handling mechanisms. Developers can first use fixed templates to complete a prototype, then gradually add template management, batch generation, user quotas, data analytics, and content moderation.

    When the system can reliably answer “What is the cost, how long does it take, what happens if it fails, and how many videos can be processed per day,” it has truly moved from a demo to an operable product.