Long-Horizon Agentic Coding, Tested: Building a Video Generation SaaS Platform in 3 Hours
A hands-on evaluation of long-horizon agentic coding tools by using them to build a production-grade, multi-tenant video generation SaaS platform. The article explains why complex, multi-service requirements expose the real limits of AI coding tools, then walks through a spec-driven development (SDD) methodology, the design of five core subsystems (auth and multi-tenancy, a data layer with row-level security, asynchronous task orchestration, the generation pipeline, and billing), six concrete pitfalls hit during the build, and a seven-point acceptance checklist for judging whether long-horizon output can be trusted.
1. Why Use a “Video Generation SaaS” to Stress-Test Long-Horizon Tasks
Most people evaluate AI coding tools with a task like “can it build a good-looking landing page.” That test barely discriminates between tools — it’s a single file, stateless, with no external dependencies, so almost any model can deliver it in minutes.
The real dividing line shows up when a requirement grows from a single page into a commercially viable multi-tenant service. Complexity here doesn’t grow linearly; it spreads exponentially along the boundaries between systems.
A video generation platform that can actually charge users and run touches at least five chains at the same time:
- Access and authentication: phone-number verification-code login, stateless token issuance and renewal, multi-device session management;
- Data layer: multi-tenant isolation, row-level security policies, soft deletes and audit fields;
- Asynchronous orchestration: a single render takes minutes while requests arrive in seconds, so the two must be decoupled;
- Generation pipeline: asset validation, model invocation, failure retries, result storage and transcoding;
- Billing and risk control: credit pre-deduction, refunds on failed tasks, concurrency quotas, abuse detection.
This is no longer a “write code” problem — it’s a “system architecture” problem. That’s exactly why it deserves a long-horizon (multi-hour) automated test.
The business motivation is worth a separate note. Early on I liked sharing how to build various automation orchestration workflows. Later I noticed a counterintuitive fact: the vast majority of merchants running cross-border e-commerce don’t want to learn how to orchestrate workflows at all. They only care about ROI — does a single video cost a few cents or a few dollars? Can it reliably produce a thousand a day? Is there a platform where you just upload assets, click, and get a video?
The conclusion is straightforward: in the AI era, rather than teaching someone to fish, it’s better to hand them the fish directly. That’s also why I changed the test from a “tutorial” to a “product” — the engineering completeness a product demands is naturally an order of magnitude higher than what a tutorial requires.
2. Methodology: Spec-Driven Development (SDD)
2.1 Core Idea: Shift from “Describing Code” to “Constraining the Artifact”
The traditional way of prompting is “help me write an upload endpoint.” In a long-horizon task this is bound to fail, because once the context exceeds the window, early agreements get silently forgotten.
Spec-driven development moves the emphasis earlier: first produce research conclusions, then architectural decisions, and only then the implementation. The output of all three stages must be persisted to files (requirements spec, architecture notes, task list), and every subsequent round of development treats these files as the single source of truth.
2.2 Four Elements the Prompt Should Contain
- Role and delivery standard: specify a “senior backend architect” and make clear “production-grade, Demo-level implementations not accepted”;
- Non-negotiable technical constraints: technology-stack boundaries, naming conventions, directory structure, prohibited practices (e.g., “no direct database access from the interface layer”);
- Acceptance criteria: every requirement needs an observable way to verify it — for example, “accessing a task’s detail while not logged in returns 404 rather than 403” (to avoid revealing whether a resource exists);
- Phased stop points: require that the output of each stage be produced and the process pause, waiting for confirmation before moving to the next stage.
Point 4 is the most easily overlooked, but it determines whether the task stays controllable. The most common cause of long-horizon failure isn’t insufficient capability — it’s the absence of checkpoints, which lets errors accumulate while no one is watching.
2.3 Context Engineering: Feed in Segments, Not All at Once
Dumping the “entire project” to the model at once always performs worse than splitting it by subsystem. My approach: the architecture file stays resident globally, while each subsystem’s implementation injects only the relevant module code when needed. This is essentially using engineering means to simulate how a person manages working memory.
3. Design Points for the Five Subsystems
3.1 Auth and Multi-Tenancy
- Phone-number verification-code login; tokens use a stateless standard (JWT-style), with short-lived access tokens plus long-lived refresh tokens;
- Every business table carries an
owner_id, and queries must include the tenant condition; - Key point: don’t filter tenants only in the application layer — add a fallback at the database layer.
3.2 Data Layer and Row-Level Security (RLS)
Row-level security policies in a relational database turn “tenant isolation” from a convention into a constraint. Even if some code forgets to write a where condition, the database still rejects unauthorized access.
create table projects (id uuid primary key default gen_random_uuid(),owner_id uuid not null,name text not null,status text not null default 'draft',created_at timestamptz not null default now());alter table projects enable row level security;create policy projects_owner_isolation on projectsusing (owner_id = current_setting('app.user_id')::uuid)with check (owner_id = current_setting('app.user_id')::uuid);
The corresponding connection layer needs to inject the session variable on every request:
async with pool.acquire() as conn:await conn.execute("select set_config('app.user_id', $1, true)", str(user.id))rows = await conn.fetch("select * from projects") # automatically subject to RLS
3.3 Asynchronous Task Orchestration: State Machine + Idempotency
Video generation takes minutes and must never be awaited synchronously. The core is three things: an explicit state machine, idempotent consumption, and observable retries.
The task states should be kept to queued → running → succeeded | failed | canceled, with no other intermediate states allowed. The consumer must check “current state + attempt count,” otherwise redelivery causes the same task to be rendered twice, directly resulting in double credit consumption.
async def consume(msg):task = await repo.get_task(msg.task_id)# Idempotency gate: if the state doesn't match or the attempt count is stale, acknowledge and dropif task.status != "queued" or task.attempt != msg.attempt:return ack(msg)async with uow():task.mark_running()try:artifact = await render_pipeline(task.payload)await storage.put(artifact)task.mark_succeeded(artifact.url, cost=artifact.billed_cost)except TransientError as e:task.retry_later(e, backoff=exponential(msg.attempt))except PermanentError as e:task.mark_failed(e)await billing.refund(task.owner_id, task.hold_amount)return ack(msg)
Credit deduction must be a three-stage process — pre-deduct, settle, and refund on failure — and the pre-deduction must happen in the same database transaction as task creation. If you create the task first and charge afterward, high concurrency will inevitably produce a negative balance.
3.4 Generation Pipeline
The pipeline itself is stateless, which makes horizontal scaling easy:
- Asset validation (format, resolution, duration, size) → store in object storage;
- Call the video generation capability and return the external task identifier immediately after submission;
- Poll or use a callback to get the result, and route retries by error type on failure;
- Transcode the result, add a watermark, write back an accessible URL, and trigger a notification.
For external model calls you must set an overall timeout budget, not just a per-request timeout. Otherwise an external task stuck in an intermediate state will permanently occupy concurrency quota.
3.5 Billing and Risk Control
- Handle the cost of a single video as “estimate + settlement difference,” using a conservative estimate;
- Apply token-bucket rate limiting per user to prevent a single account from flooding the queue;
- Record the actual cost of each generation to build an auditable unit-economics model — this is what determines whether the business can run long-term, not just whether it can run at all.
4. Six Pitfalls I Actually Hit
1. Interface drift caused by context compression. In the mid-to-late stage of a long-horizon task, an interface signature defined earlier gets “reinvented” once, and the frontend calls no longer match the backend implementation. The fix is to freeze the interface contract into a machine-readable spec file and run a contract check at the end of every round.
2. Database schema drift. The model directly edited the model definition file but didn’t generate the corresponding migration script — it runs locally but fails to start in a clean environment. You must require that any schema change come with a migration script, and that migrations are forward- and rollback-capable.
3. Duplicate consumption of asynchronous tasks. Early versions had no attempt-count check, and queue redelivery caused duplicate generation. An idempotency gate is a necessity, not an optimization.
4. Long-lived connections cut off by the gateway. Initially I used long polling to wait for generation to complete, which failed in bulk under the reverse proxy’s default timeout. Minute-level tasks should return a task identifier for the client to poll, or switch to Server-Sent Events (SSE), rather than leaving the HTTP request hanging.
5. Frontend mocks inconsistent with real data. During UI development I used fake data, and the field structures didn’t match during integration, causing a large amount of rework. Having the frontend integrate against the real interface contract from the start actually lowers the rework cost.
6. Incorrect cost and credit accounting. The first implementation charged on “generation success,” so failed tasks burned compute for free without being counted. Cost accounting must cover failures and retries too.
A note worth making: the roughly three-hour figure is a measured result for this one specific project under good network conditions and ample compute, and it does not constitute any general performance conclusion. Compute was tight that day, and the actual span was stretched to nearly a full day. This has nothing to do with model capability, but it’s precisely the most realistic constraint when putting long-horizon tasks into practice.
5. Acceptance Checklist: How to Judge Whether Long-Horizon Output Is Trustworthy
The biggest risk in long-horizon delivery is that it “looks complete.” I recommend verifying against seven hard metrics:
- One-click startup in a clean environment, with migrations that are forward- and rollback-capable;
- Tenant isolation has a database-layer fallback, not just application-layer filtering;
- Asynchronous tasks have an idempotency gate and attempt-count checks;
- Billing is a three-stage pre-deduct + settle + refund with clear transaction boundaries;
- All external calls have an overall timeout budget and failure routing;
- Critical paths are covered by automated tests, not just manual verification;
- Minimal observability: structured logs, task success rate, and per-task cost all queryable.
On the deployment side, there’s no need to build all the infrastructure yourself: a container platform to host service instances, a message queue to carry asynchronous tasks, object storage to hold assets and artifacts, and logging and monitoring services for observability are all reusable general-purpose capabilities (for example, on a platform like Baidu AI Cloud, these are offered as standard categories). Save your energy for the business logic itself.
6. Conclusion: Where the Real Capability Boundary Lies
The biggest takeaway from this round of experiments wasn’t “how much code a given tool can write,” but a shift in how requirements are expressed: as the delivery horizon stretches from minutes to hours, the developer’s core work moves earlier — from “writing the implementation” to “writing the spec, setting constraints, placing checkpoints, and verifying the artifact.”
At the same time, don’t be overly optimistic. Where long-horizon tasks need cross-system tradeoffs, business-priority judgments, or choices among several imperfect options, a person is still needed to decide. Its capability boundary is clear: within a range that has an explicit spec and verifiable criteria, it can replace a team in delivering a from-scratch-to-running engineering implementation; where the spec is fuzzy and exploration is required, human judgment is still indispensable.
Verifying this isn’t hard: pick a system-level requirement you’ve always wanted to build but shelved because you were “short on hands,” split it into three stages — research, architecture, development — write clear acceptance criteria, and then let it run.
