Multi-Hour AI Coding Agent Test: Can AI Independently Deliver a Video Generation SaaS?
This article documents a hands-on experiment in which an AI coding agent is tasked with building a complete short-video marketing SaaS platform over a multi-hour autonomous run. It explains why the boundary of AI programming is shifting from snippet generation to full system delivery, introduces a spec-driven development (SDD) methodology with a three-stage research → architecture → development workflow, and walks through the platform's four-layer architecture and key modules: asynchronous task management with state machines and idempotent deduction, phone-code authentication, credit-based billing with pre-freezing and refunds, and an external-service adapter layer with observability. It also shares empirical findings on where long-running agents excel and where they fail, along with practical recommendations for keeping them on track. Readers will learn how to organize long-horizon AI tasks, control spec drift, and collaborate with agents to deliver reliable, operable systems.
From “Writing Code” to “Delivering a System”
Most coding agents today showcase the visual polish of the pages they generate, and only a few vendors have started emphasizing task planning, long-horizon execution, and systems-engineering capability. They claim a model can decompose system-level requirements like a senior architect and keep its context coherent across automation runs lasting several hours.
This shift points to a key question: the capability boundary of AI programming is expanding from “generating code snippets” to “delivering complete systems.” The difference between the two is not simply additive:
| Dimension | Snippet-Style Code Generation | Long-Horizon System Delivery |
|---|---|---|
| Task duration | Seconds to minutes | Hours |
| Context | Current file and conversation | Dozens of files, interface contracts, design decisions |
| Cost of failure | Fixable with local edits | Errors accumulate and may surface all at once later |
| Required capability | Syntax and API knowledge | Requirement decomposition, architecture design, trade-offs |
The difficulty of long-horizon tasks lies in this: the model must keep remembering “why things were designed this way” across a large number of tool calls. Once it loses an early constraint, it can write code that conflicts with the established architecture, and the rework cost later becomes very high. This article tests how a long-horizon agent performs in real system development and distills a reusable methodology from the experience.
Topic Selection: Delivering a Video Generation SaaS End to End with AI
The test target was a “short-video marketing auto-generation platform.” The core scenario: a cross-border e-commerce merchant uploads product images, and the system automatically composes a promotional video with subtitles and voiceover, delivered as an out-of-the-box service aimed directly at end merchants.
This topic was chosen deliberately. Many prior tutorials focus on workflow orchestration, teaching users to string together various nodes to produce videos. But frontline merchants actually care about return on investment: what a single video costs, how many can be produced per day, and whether there is a platform they can use with one click. For most merchants, the learning curve itself is a hidden expense. So the product positioning was defined clearly as “delivering the result directly into the user’s hands,” rather than “teaching the user to build it themselves.”
From a technical standpoint, this topic also covers the typical hard parts of long-horizon development:
- Integration: connecting to external video generation services, object storage, and content moderation capabilities;
- Asynchrony: video generation takes anywhere from tens of seconds to several minutes, requiring task queues and scheduling;
- Reliability: task status tracking, failure retries, and timeout handling;
- Business closure: phone-code login, token authentication, and credit deduction and refund.
This is exactly the part of AI programming that is most often underestimated: the real barrier is not “writing pages” but organizing scattered capabilities into a system that stays state-consistent and operable. In a cloud environment, these capabilities break down into general-purpose components such as object storage, message queues, container platforms, logging services, and monitoring and alerting. This is precisely the range of foundational capabilities that products like Baidu AI Cloud provide.
Methodology: Spec-Driven Development and Phased Prompting
Why Spec-Driven Development Is Needed
If you simply hand requirements to an agent and tell it to “build a platform from scratch,” the result is often tens of thousands of lines of hard-to-maintain code generated in one shot. Once it errors at runtime, the model lacks structured context and can only patch locally, eventually falling into a loop of “fix one thing, break three.”
The idea behind Spec-Driven Development (SDD) is to distill requirements, architecture, interfaces, and data models into explicit specification documents before any code is written. The development phase treats the spec as the standard, and the spec becomes “compressed context” passed across stages.
In long-horizon tasks, the value of this approach is amplified:
- The model can look back at the spec at any step, reducing design drift;
- Developers can catch errors during spec review, avoiding carrying them into the coding phase;
- Each completed stage solidifies its artifacts, naturally forming recoverable checkpoints.
A Three-Stage Workflow
The experiment used a three-stage workflow of “Research → Architecture → Development.” At the end of each stage, the agent was required to output a structured document, and after human confirmation it could proceed to the next stage. The prompt skeleton is as follows (illustrative):
[Role] Senior System Architect + Full-Stack Engineer[Goal] Build a short-video marketing auto-generation SaaS platform from scratch[Working Method] Advance strictly through the following stages; at the end of each stage, output a document and wait for confirmation:1. Research: list core user roles, key business flows, feature list and priorities (MVP scope)2. Architecture: provide the rationale for tech choices, system layering, module responsibilities, data model (ER) and API contracts3. Development: implement module by module, add unit tests after each module, and output run and verification steps[Hard Constraints]- All time-consuming operations (video generation, content moderation) must be asynchronous, and the API must return a task ID immediately- Authentication uses phone-code login + token authentication; any deduction with financial semantics must be idempotent- External dependencies must be wrapped behind an adapter layer; direct calls to third-party APIs in business code are forbidden- Database changes are managed via versioned migration scripts- Critical paths (ordering, charging, task creation) must have tests- Do not invent nonexistent APIs or parameters; when unsure, list open questions first
Each stage has its own emphasis:
- Research answers “what to build.” The agent outputs user roles (merchant, operations, admin), core flows (register/login → upload assets → place order and charge → generate task → download result), and an MVP feature list, turning a vague idea into verifiable requirements.
- Architecture answers “how to organize.” It requires the rationale for tech choices, module breakdown, a database ER diagram, and API contracts. The API contracts are frozen as interface documentation, and subsequent development follows them, avoiding rework caused by frontend and backend making divergent assumptions.
- Development answers “how to implement.” Modules are implemented in batches, each with tests and a self-check list, keeping the frontend, backend, database scripts, and configuration in sync.
Engineering Details of Process Management
A long-horizon agent runs for hours, and developers cannot watch over it the whole time, so process management matters just as much:
- Stage checkpoints: require a commit after each completed module, with a commit message describing the change and the corresponding requirement item, so issues can be rolled back quickly;
- Context compression strategy: when the conversation nears its context limit, require the agent to first update a “decision log and progress checklist,” then start a new session to continue, distilling easily lost design decisions into files;
- Externalized constraints: write the hard constraints into a rules file at the project root (such as
PROJECT_RULES.md) so the agent reads them automatically at each stage, rather than relying on a single prompt; - Minimal viable path first: prioritize getting the main flow of “login → upload → order → asynchronous generation → download” working end to end, then add enhancements such as rate limiting, monitoring, and an admin console once it is verified.
Architecture: Four-Layer Decoupling for Long-Running Tasks
After the architecture stage, the platform was divided into four layers:
- Access layer: web console, admin console, and API gateway, responsible for request routing, rate limiting, and basic protection;
- Business layer: users and authentication, asset management, orders and credits, and task management, all stateless services for easy horizontal scaling;
- Asynchronous layer: message queue, task scheduler, generation executor, and scheduled jobs, responsible for smoothing peaks and handling retries for time-consuming operations;
- Infrastructure layer: relational database, in-memory cache, object storage, logging service, and monitoring and alerting.
The complete flow when a user submits a generation request is as follows:
Console uploads image → object storage (returns asset reference)↓Submit generation request (with asset ID, video parameters)↓Business layer: verify token → pre-freeze credits → write task (PENDING) → enqueue message↓ (API returns task ID immediately)Scheduler/executor: consume message → atomically claim task → call generation adapter → poll generation service↓Success: transcode output written to object storage → task set to SUCCESS → confirm credit deductionFailure: retry/degrade by error type → if limit exceeded, set to FAILED → unfreeze credits↓Console polls task status → downloads result on success
Clear, decoupled responsibilities across the layers were a principle repeatedly emphasized in this design. For example, the business layer does not call the generation service directly; it goes through the adapter layer, leaving room to swap models later.
Implementation of Key Modules
Asynchronous Tasks: State Machine, Claiming, and Backoff Polling
Video generation typically takes tens of seconds to several minutes. With a synchronous request, a connection timeout at the gateway or load balancer would abort the request. Meanwhile, the generation service has limited concurrency, and burst traffic would directly overwhelm the upstream. So the pattern of “message queue buffering + executor consumption + state-machine management” was adopted.
The task state transitions are as follows:
PENDING --create task--> QUEUED --executor claims--> RUNNING↑ ││ ┌────┴─────┐│ success │ │ failure (retryable)│ ↓ ↓└──retry limit not exceeded──┘ SUCCESS FAILED (terminal)↑TIMEOUT ──┘
The claiming logic uses a conditional database update to achieve mutual exclusion (illustrative):
-- Claim succeeds only when the task is in the pending stateUPDATE gen_taskSET status = 'RUNNING', worker_id = :workerId, started_at = NOW()WHERE id = :taskId AND status = 'QUEUED';-- A claim is successful only if the affected row count is 1
The message queue provides reliable delivery, and messages are re-enqueued when the executor fails to consume them. But if the executor crashes after successfully calling the upstream but before writing back the result, a retry would cause duplicate charges. So two mechanisms were designed:
- Idempotency key: on each submission, generate and persist a business idempotency key from “user ID + asset ID + parameter hash,” and on retry, first check whether a successful record already exists;
- Reconciliation job: periodically scan tasks stuck in RUNNING for a long time, verify their true status against the upstream generation service, and correct local inconsistencies.
For polling, the external generation service returns only a “processing” status until the task completes, so the executor pulls results with exponential backoff (e.g., 3s → 6s → 12s, capped at 30s) to avoid high-frequency requests. The frontend console queries task status every 2–3 seconds and informs the user of the current stage through progress text.
Authentication: Phone-Code Login and Token Verification
The login flow is: the user submits a phone number → the server checks send frequency and generates a verification code → after verification passes, it issues an access token and a refresh token.
Key security points include:
- Verification codes have a time-to-live (e.g., 5 minutes) and a maximum number of attempts, and are invalidated immediately after a successful check;
- Phone numbers are rate-limited by IP and device dimensions to prevent SMS bombing and malicious abuse of the API;
- Access tokens have a short lifetime; refresh tokens are bound to the device and can be actively revoked;
- All endpoints requiring login pass through a unified auth middleware, so authentication logic is not reimplemented in business code.
Illustrative code (framework-agnostic):
// Auth middleware: parse and verify the token, then attach user identity on successasync function auth(req, res, next) {const token = (req.headers['authorization'] || '').replace(/^Bearer\s+/i, '');if (!token) return res.status(401).json({ code: 'UNAUTHENTICATED' });try {// Signature and expiry check; can work with a token blacklist for forced logoutconst payload = verifyToken(token);if (await isTokenRevoked(payload.jti)) {return res.status(401).json({ code: 'TOKEN_REVOKED' });}req.user = { id: payload.sub, phone: payload.phone };next();} catch (e) {res.status(401).json({ code: 'INVALID_TOKEN' });}}
Credit Billing: Pre-Freezing, Idempotent Deduction, and Refund on Failure
Billing is the module with the strictest consistency requirements in the entire system. Rather than charging only after generation succeeds, a two-phase model of “pre-freeze at order time, confirm deduction on success, unfreeze on failure” ensures the user’s balance ultimately matches actual consumption.
- Accounts and ledgers: the account table stores available and frozen credits; the ledger table records every freeze, deduction, unfreeze, and top-up, including the business order number and idempotency key, so any balance change is traceable;
- Concurrency safety: deductions use a conditional atomic update to prevent over-deduction and overselling.
-- Complete deduction and freezing in one shot when available credits are sufficientUPDATE credit_accountSET available = available - :amount,frozen = frozen + :amount,version = version + 1WHERE user_id = :userIdAND available >= :amount;
- Idempotency guarantee: the idempotency key has a unique constraint at the database level, and duplicate requests that trigger a conflict directly return the first result;
- Refund on failure: when generation fails, credits are unfrozen and a refund entry is written.
Illustrative flow:
async function settleTask(task) {const entry = await findCreditEntry(task.idempotencyKey); // idempotency key + unique constraintif (entry && entry.status !== 'PENDING') return entry;if (task.status === 'SUCCESS') {return confirmDeduction(entry, task.consumedSeconds); // confirm deduction by actual duration}if (task.status === 'FAILED' || task.status === 'TIMEOUT') {return unfreeze(entry); // full unfreeze; partial compensation configurable}return entry;}
External Capability Adaptation and Observability
The generation service may be affected by model versions, rate limits, and price changes, so it is uniformly wrapped behind an adapter layer: define three standard methods, submit / queryStatus / cancel, and let different upstreams implement their own adapters while the business layer depends only on the interface. This way, swapping or layering generation capabilities requires no changes to business code.
For observability:
- Assign a trace ID that spans the full task lifecycle, chaining the entire “submit → schedule → call upstream → write back result” process in the logging service;
- Set core metrics in monitoring and alerting: task success rate, average generation time, queue backlog, and abnormal credit fluctuations;
- All access to user assets uses temporary signed URLs from object storage with an expiration, avoiding public access to resources.
Test Observations: What Long-Horizon Agents Do Well and Where They Falter
Areas That Stood Out
- Scaffolding and infrastructure: repetitive engineering code such as project initialization, database migration scripts, auth middleware, and message queue integration was generated with high quality and little rework;
- Contract consistency: after API contracts were frozen, backend endpoints and frontend calls stayed largely consistent, significantly lowering cross-layer integration cost;
- Self-repair: after a test run failed, the agent could locate and fix the relevant files based on the error output, without step-by-step human prompting.
Failure Points to Watch For
- Implicit assumptions: on details left unspecified, the agent filled in gaps on its own, for example defaulting to “no refund on failure” or “assets never expire,” which did not match business expectations;
- Over-engineering: introducing complex abstraction layers unrelated to current needs during the MVP phase;
- Spec drift: in the later stages, slight changes appeared in field names and status enums agreed on early, requiring spec documents and tests to guard against them.
Recommendations
- During the requirements phase, write business rules as decidable items, for example “fully refund credits when generation fails” and “automatically clean up assets older than 7 days”;
- After each module is completed, require the agent to run tests and paste the results, and fix and rerun if they fail, forming an “implement → verify” loop;
- Shift the review focus from “is the code correct” to “are the constraints being followed,” spot-checking whether key constraints from the spec are actually reflected in the code.
Conclusion
The most valuable takeaway from this experiment is not “AI can write a platform in a few hours,” but that what truly determines delivery quality is how humans organize the task. Spec-driven development breaks a long-horizon task into verifiable stages of “research → architecture → development,” using documents to fight context forgetting and tests to constrain the model’s free play. As the capability of coding agents moves from generating code to delivering systems, the developer’s core value shifts accordingly: defining the problem, designing constraints, and verifying quality.
At the same time, engineering fundamentals are far from obsolete. Classic distributed-systems problems like idempotency, state machines, authentication, and observability remain the bedrock of system reliability. The difference is that they no longer need to be hand-written from scratch by one person, but are completed through collaboration between humans and agents. Using AI to quickly turn an idea into a runnable product is becoming the standard way of working for developers.
