Hands-On: Testing AI Coding Long-Horizon Capabilities by Building a Full-Stack Video Generation SaaS

    This article examines how AI coding tools are shifting from code completion toward long-horizon system engineering. To put the claim to the test, the author has an intelligent coding agent build a multi-tenant, short-video generation SaaS platform for cross-border e-commerce from scratch in roughly three hours. It explains why video generation SaaS makes a demanding stress-test topic, how Spec-Driven Development (SDD) keeps a long-horizon agent on track through Research, Design, and Development phases, and walks through the core pieces: multi-tenancy with row-level security, asynchronous task queues with status polling, and consistent billing via a credit ledger. It closes with lessons on context drift, happy-path bias, and the sensitivity of long-horizon agents to infrastructure stability.

    First, the conclusion: AI coding is moving from “code completion” to “long-horizon system engineering”

    Over the past two years, most people’s understanding of AI coding has stayed at the level of “completing a line of code” or “generating a function.” But recently, the focus of a new wave of intelligent coding tools has clearly shifted. The headline selling points are no longer “does the generated page look good,” but rather Agentic behavior, Long-Horizon Tasks, and System Engineering—in other words, whether a model can act like a senior architect, autonomously decomposing system-level requirements and staying contextually coherent, on track, and without forgetting across long stretches of continuous, multi-hour automated work.

    There is a school of thought online claiming that these tools can now rival top-tier models on complex engineering tasks. Rather than argue about it, I decided to put it to the test myself. So I designed a high-intensity experiment: have an intelligent coding tool build a multi-tenant, short-video generation SaaS platform for cross-border e-commerce from zero, and observe whether it could complete the entire journey from research to a runnable system in roughly three hours under normal compute availability.

    I should stress that this is not as simple as “generating a good-looking demo page.” Between a production-grade SaaS and a demonstration prototype sits an entire body of systems engineering.

    Why “video generation SaaS” is the stress-test topic

    The choice of topic itself comes from a counterintuitive business observation.

    In the past I was keen to share all kinds of automation workflows, teaching people how to “craft” a product-marketing short video step by step. But later I realized that most cross-border e-commerce operators don’t actually want to learn the workflow itself. What they truly care about is only the input-output ratio: does a single video cost a few cents or a few dollars? Can it reliably produce a thousand videos a day? Is there a platform where you upload product assets, click once, and get a finished video?

    This judgment points to a clear conclusion: in the AI era, handing users the method (“teaching someone to fish”) is often less effective than delivering the result directly to the user (“giving someone the fish”). So a natural topic emerged—build a short-video generation platform where users upload image assets, the system automatically assembles a marketing video, and the “fish” is delivered straight to the user.

    Supporting this product form is a systems-engineering effort with extremely high technical density.

    The complexity of a production-grade SaaS rises exponentially

    Placing the single action of “generating a video” under production scrutiny immediately spawns a chain of tightly coupled engineering problems:

    • Backend services: How do you design a well-structured, extensible server (for example, based on a Python asynchronous web framework) and organize routing, dependency injection, and domain layering?
    • Database and multi-tenancy: How should the core table structures be designed? How do you achieve logical isolation of user data, and go further with Row-Level Security (RLS) to enforce tenant data isolation at the database layer, preventing unauthorized access?
    • Task scheduling: Video synthesis usually takes several minutes and cannot be awaited synchronously within a request thread. How do you introduce an asynchronous task queue and design a reliable status polling/callback mechanism so users know their task’s progress?
    • Authentication and billing: How do you implement phone-number verification-code login? How do you issue and validate JWTs? How do you design a credits system that pre-deducts on task creation, rolls back on failure, and settles on completion?

    The key difficulty is that these problems are strongly coupled with one another. Billing logic depends on task state, task state depends on queue reliability, and the queue in turn depends on authentication to determine “whose task it is.” This is no longer a “write code” problem but a “system design” problem. By conventional wisdom, it is hard to polish all these pieces to a production-grade level without a team of around five people.

    This time, I used only a single intelligent coding tool.

    Constraining agent behavior with Spec-Driven Development (SDD)

    To keep a long-horizon agent from going off the rails over several hours, the most effective approach is not repeated verbal reminders but giving it an executable spec. I adopted the SDD (Spec-Driven Development) approach, forcing the agent to proceed strictly through three phases—Research → Design → Development—with each phase required to produce a written artifact before moving to the next.

    A reusable spec skeleton looks roughly like this:

    1. # Spec document (specs/platform-spec.md)
    2. ## 1. Research phase
    3. - Target users and core scenario: batch generation of marketing short videos for cross-border e-commerce sellers
    4. - Key constraints: multi-tenant isolation, usage-based billing, asynchronous long-running tasks, horizontal scalability
    5. - Candidate technology choices and rationale for trade-offs (include a comparison; do not give a single conclusion)
    6. ## 2. Design phase
    7. - System layering: access layer / business service layer / asynchronous task layer / data layer
    8. - Data model: users, assets, tasks, credit ledger entries, video artifacts
    9. - Key flow sequence: upload assets → create task (pre-deduct credits) → enqueue → generate → write back status → settle
    10. - Failures and compensation: task timeout, generation failure, credit rollback
    11. ## 3. Development phase
    12. - Split implementation order by module; connect the main flow first, then fill in edge cases
    13. - Self-test after each module is delivered before moving to the next

    The value of this spec is that it chops the “long-horizon task” into short milestones with clear acceptance points. The agent must “stop and report” at each phase, and the context is explicitly compressed into documents rather than an infinitely growing conversation history. This significantly reduces common long-task problems like “forgetting requirements” and “repeatedly overturning decisions.”

    Breaking down the key pieces

    1. Multi-tenancy and row-level security

    The most easily overlooked security boundary in a multi-tenant SaaS is “reading someone else’s data through broken access control.” Beyond application-layer tenant filtering, a more robust approach is to push isolation down to the database layer:

    1. -- Enable row-level security on the task table
    2. ALTER TABLE generation_task ENABLE ROW LEVEL SECURITY;
    3. -- Only allow reading rows whose tenant_id matches the current session context
    4. CREATE POLICY tenant_isolation ON generation_task
    5. USING (tenant_id = current_setting('app.current_tenant')::uuid);

    The benefit is that even if a query forgets to include the tenant condition, the database will reject cross-tenant data as a backstop, turning security from “relying on the programmer’s memory” into “relying on a database constraint.”

    2. Asynchronous task queue and status polling

    Video generation time is uncontrollable and must be made asynchronous. A typical flow is:

    1. Request thread: create task → pre-deduct credits → write task row (PENDING) → publish message → return task_id immediately
    2. Worker process: consume message → call the video generation model → upload result to object storage → update status (SUCCEEDED/FAILED)
    3. Frontend: poll the task status endpoint with task_id (or receive a callback), and show the finished-video link on completion

    There are two easily overlooked design points here. First, credits should be “pre-deducted” at task creation rather than deducted at completion, otherwise you get bad debt where “generation succeeds but the user’s balance is insufficient.” Second, queue consumption must be idempotent, because messages may be delivered more than once, and duplicate credit deductions would create a reconciliation disaster.

    3. Authentication, credits, and deduction consistency

    The billing system is about “money” and has the lowest fault tolerance. The core principle is to make deduction a closed loop that keeps a ledger, can be rolled back, and can be reconciled:

    1. 1. Create task: write a PRE_AUTH (pre-authorization/hold) record to the credit ledger table
    2. 2. Task succeeds: write a SETTLE record, converting the held amount into an actual deduction
    3. 3. Task fails/times out: write a REFUND record, releasing the held amount
    4. 4. Reconciliation job: periodically verify that "held + settled + refunded" equals the opening balance

    As long as every change of a cent is recorded in the ledger, the system gains the ability to reconcile and to recover from failures. This is also where intelligent coding tools tend to “only write the happy path” in long tasks—you need to explicitly require them in the spec to fill in the failure and compensation branches.

    Post-mortem: lessons from the pitfalls

    This experiment was not all green lights. A few typical pitfalls are worth recording:

    • Context drift: In long tasks, the agent sometimes “forgets” field naming or interface conventions defined early on, causing later modules not to line up with earlier ones. The fix is to lock contracts (data models, interface signatures) into the spec document, rather than letting it improvise from memory.
    • Happy-path dependency: By default, it is more likely to get the main flow working while ignoring edge cases like timeouts, retries, and rollbacks. You need to proactively ask “what happens if it fails?” during review.
    • Rhythm interruption from compute congestion: When many users flood in at once, the platform’s response slows or even queues up, and long tasks may be interrupted midway. This reminds us that long-horizon agents are actually quite sensitive to infrastructure stability, and ample compute plus a stable calling experience is a prerequisite for truly finishing the “three hours.”

    Conclusion

    The biggest takeaway from this stress test is that the focus of competition in AI coding has shifted from “can it write this piece of code” to “can it independently complete a long-horizon, system-level engineering task.” When an agent can proceed autonomously for hours at the pace of “Research → Design → Development” and deliver a runnable system with multi-tenant isolation, asynchronous tasks, and a closed-loop billing system, its meaning to developers is no longer “typing faster,” but redrawing the capability boundary between individuals and small teams.

    Of course, the stronger the tool, the higher—not lower—the bar it sets for people: developers need to understand architecture better, understand how to decompose problems better, and better understand how to turn vague requirements into clear specs. The better the spec is written, the higher the quality of the long-horizon agent’s output—and that may be where the real substance of this “next level” lies.