Building a Video SaaS with an AI Coding Agent: Boundaries and Risks
This article presents the findings and practical boundaries of a single test in which an AI coding agent built a short-video generation platform from scratch within several hours. It introduces a specification-driven development workflow, design considerations for context management, concurrency, caching, and retries, and a clear division between tasks suitable for AI assistance and those requiring human oversight. It also covers unverified assumptions, deployment checks, and common troubleshooting paths. The recommendations are reference patterns rather than measured benchmarks and should be validated before reuse.
Test Scope and Boundaries
This article discusses the approach and conclusions from a single test. The test asked an AI coding agent to build a short-video generation platform from scratch within several hours. The platform was designed for cross-border e-commerce sellers: users upload product images, the system processes them automatically, and users download the generated videos.
The test focused on three areas:
- Whether the agent could maintain coherent context during a long-running task
- Whether the system decomposition was reasonable
- Whether engineering details were handled rigorously
This article does not provide comparable quantitative metrics or industry averages. The design diagrams and templates are reference structures compiled for this discussion, not measurements from a specific run. Readers should validate them independently before applying them to their own projects.
A Three-Phase Specification-Driven Development Template
Specification-Driven Development (SDD) divides the work into research, architecture, and development phases. Each phase produces documentation that is reviewed by a human before the next phase begins. The goal is to make the model reason and plan before writing code. Open-ended conversations can easily drift, while engineering work requires convergence.
Recommended contents for the research document
- Business goals and non-goals
- User roles, including administrators, regular users, and background workers
- MVP scope and explicit exclusions
- Acceptance criteria, with at least one verifiable result for each feature
- Risk assumptions, such as third-party API availability and the source of media assets
Recommended contents for the architecture document
- Module boundaries, including authentication, media upload, the rendering pipeline, task queues, resource management, and frontend presentation
- Data models for the four primary entities: users, media assets, tasks, and generated videos, including their key fields and states
- API contracts covering the path, request fields, response fields, and error codes for each endpoint. The frontend and backend should use the same contract.
- Task queue, cache, and retry configurations, as described in the next section
- Failure degradation behavior. For example, when an external visual generation API is unavailable, the task should move to a pending-retry state and the user should receive an understandable message.
Recommended contents for the development document
- Task breakdown, with each task mapped to a module and its inputs and outputs clearly defined
- Acceptance cases covering at least the normal flow, missing fields, timeouts, and duplicate submissions
- Integration steps: start with API integration, followed by end-to-end integration
- Rollback points, with a revertible version marker for each milestone
A human review gate should be placed at the end of every phase. The review should verify that:
- The documentation is complete
- Field names are consistent between the frontend and backend
- The acceptance cases can actually be executed
Design Patterns for Context, Concurrency, Caching, and Retries
The values below are starting points for design, not optimal values established through testing.
Context management
- At the beginning of each phase, provide only the documents for the current phase and the relevant API contracts. Do not repeatedly paste the entire conversation history.
- Keep field naming conventions in a shared data dictionary and update that file whenever a field changes.
- When the context approaches its limit, ask the agent to generate a phase summary before starting a new session.
Concurrency control
- Keep rendering concurrency below the quota of the external API. A reasonable starting point is one or two concurrent tasks per user.
- When the same asset is submitted repeatedly, use the asset hash for deduplication to avoid duplicate charges.
- Restrict the task state machine to valid transitions. For example, a completed task should not be allowed to move directly back to the queued state.
Caching and retries
- Uploads and preprocessing results can be cached. The cache key should include the asset hash and the version of the processing parameters.
- Use exponential backoff when external API calls fail. A reasonable starting point is a maximum of three retries with an initial delay of several seconds.
- Retries must include an idempotency key. Otherwise, the same task could generate multiple videos and incur multiple charges.
- Tasks that exceed the retry limit should enter a dead-letter queue for manual handling or scheduled processing.
Observations and Unverified Items
The conclusions in this article fall into three confidence categories.
Observed
The test focused on three types of problems: context overflow, inconsistent frontend and backend fields, and broken logic between modules. Phase gates and a shared data dictionary can help identify these problems at an early stage. However, the article does not report how frequently each problem occurred, so the findings should not be treated as probability estimates.
Not verified
The proposed concurrency levels, retry count, and backoff interval were not validated through load testing. The test did not cover queue behavior under heavy load or the actual handling time for dead-letter tasks.
Dependent on conditions
Whether an AI agent can independently complete an entire system depends on the clarity of the requirements, the quality of the external API documentation, and the team’s code review capabilities. The less precise the requirements, the more human oversight is required.
Which Tasks Are Suitable for AI Assistance?
| Area | Recommendation | Notes |
|---|---|---|
| Boilerplate code and CRUD endpoints | Delegate to AI with human review | The structure is predictable; review should focus on field consistency |
| Initial data model drafts | Delegate to AI with human confirmation | State transitions require manual verification |
| Third-party API integrations | Human-led, AI-assisted | Official documentation and quota details must be checked |
| Billing, permissions, and secret management | Must have human oversight | Errors can have significant consequences |
| Rendering parameters and cost strategy | Must have human oversight | These involve business trade-offs |
| Content compliance rules | Must have human oversight | These involve legal requirements and platform policies |
Pre-Launch Checklist
Confirm each item before launch:
- Store API keys only in environment variables or a secrets management service. Do not place them in code, prompts, or logs.
- Set per-user and system-wide usage limits, along with budget alerts.
- Trigger an alert when the queue backlog exceeds a threshold, and make it possible to pause the acceptance of new tasks.
- Use uploaded assets only for the current processing request, define a clear retention period, and support user deletion.
- Redact personal information such as phone numbers and email addresses in logs.
- Record the source of media assets and provide an avenue for copyright infringement complaints.
- Apply basic compliance checks to generated content, with the ability to block prohibited terms and sensitive imagery.
- Preserve a rollback version for every release. Back up the database before migrations and verify that the backup can be restored.
Uploaded assets, account credentials, and user data must not be placed in prompts or public logs. This rule should be enforced during development rather than added only before launch.
Common Errors and Troubleshooting Paths
- Field mismatches: Check the data dictionary first, then the API contract, and finally compare the frontend and backend type definitions.
- Queue blockage: Check the number of dead-letter tasks and the duration of individual tasks. Confirm whether any task is stuck during an external API call without a configured timeout.
- Third-party API rate limiting: Check the response codes, verify that retries use idempotency keys, reduce concurrency, and then restore it gradually.
