Evaluating and Safely Accepting Long-Horizon Code Generation
Long-horizon code generation is moving beyond function completion toward making continuous changes across real codebases. This article presents an evaluation framework centered on legacy-system understanding, cross-module consistency, build and test reliability, permission boundaries, failure handling, and human security review. It also provides reusable acceptance criteria for deciding whether generated code is ready to merge.
Long-horizon code generation has moved from “complete this function” to continuously advancing a real engineering change. In practice, the key question is not whether a single response looks impressive. It is whether the model can understand an existing system, maintain consistency across modules, and leave the code in a compilable, testable, and reversible state. Benchmarks are useful for comparison, but they cannot replace acceptance testing against systems with historical complexity and security constraints.
Three Questions an Evaluation Should Answer
First, does the model understand constraints rather than merely the wording of a requirement? Legacy projects often include constraints around directory boundaries, authentication methods, data ownership, prohibited external dependencies, and established error-code conventions. A requirement may say, “Import user images and reference them in an animation scene.” The actual challenge is determining who may import the images, where they are stored, and whether the reference operation can bypass authorization.
Second, can the model make sustained progress across a long context instead of reinventing the architecture at every turn? Long-task failures often do not appear in the first line of code. The model may forget a state machine midway through the task, place a temporary workaround on the main execution path, or assign different meanings to the same field in the frontend and backend.
Third, does “runnable” mean “ready to deliver”? Passing tests only shows that behavior matches expectations for the covered cases. If permission isolation, malformed input, concurrency, and rollback are not covered, a green automated test suite can create false confidence.
Task Design: Legacy Modification and Greenfield Delivery
Two substantially different task types reveal capability boundaries more effectively than a single algorithm problem.
Incremental Changes to a Legacy System
Incremental legacy-system work is useful for evaluating code reading and constraint compliance. Prepare a small project with existing layers such as an API layer, task state management, a storage abstraction, and a rendering or preview module. The requirement should be limited to adding “user asset import and controlled referencing.”
The task definition should explicitly state that:
- Assets must be isolated by user.
- Ownership must be checked before an asset is referenced.
- Internal paths must not be exposed directly to the frontend.
- Failures must be retryable and must not leave incomplete records.
The existing code does not need to be perfect. It should, however, contain realistic naming inconsistencies and one easy-to-miss authorization gap. This helps reveal whether the model follows old habits or proactively adds the missing validation.
Greenfield Implementation
A greenfield task evaluates how completely the model can turn source material into a product. Provide only a learning-objective document and a short product description, including several knowledge points, an ordered sequence of levels, completion conditions, and a review flow for incorrect answers. Do not provide an existing project skeleton.
A satisfactory result should include a playable minimum loop, repeatable completion checks, and basic tests. It should not consist only of mockups or pseudocode.
Time-box the two task types separately and record the number of human interventions for each. Interventions include correcting an incorrect assumption, adding an overlooked security rule, or pointing out an untested path. The intervention count is itself a quality signal.
Observe the Process, Not Only the Final Diff
Record progress by phase instead of waiting until the end to inspect the commits.
Context Understanding
Before changing code, does the model identify module boundaries, data flows, and invariants that must not be broken? If it immediately performs a large rewrite without explaining compatibility with existing interfaces, the subsequent risk is high.
Cross-Module Progress
Check whether contracts—request and response fields—match the persisted state, asynchronous tasks, and presentation layer. All components should use the same state enumeration and semantics. One of the most common long-task defects is that the frontend treats an operation as complete while the backend still considers it in progress.
Runnable State
The project should build, start, and complete one primary workflow in a clean environment. Something that appears to work only inside the model’s temporary directory does not qualify as passing.
Testable State
Check whether the model adds tests for the primary path, rejection paths, and exceptional paths. A test suite containing only successful cases provides very little evidence for a security-sensitive change.
Representative invariants should be encoded in tests rather than left in comments:
Given an asset identifier owned by user AWhen user B requests to bind it to user B's sceneThen the request is rejectedAnd neither the scene configuration nor storage is changed
Assertions like this represent delivery standards more effectively than checking only whether the API returns 200.
Authorization and Input Handling: What Humans Must Own
A code model can generate validation logic, but it cannot replace the threat model. At minimum, a human should review the following four areas for asset-import functionality.
Identity and Ownership
Are the uploader, resource owner, and referencing user the same principal? If sharing is supported, is there an explicit authorization record rather than reliance on a guessable path?
Input Boundaries
Validate file type and size, normalize filenames, prevent path traversal, and ensure that script content cannot be interpreted as a template. Animation or markup-based scenes are especially vulnerable to turning a supposedly safe “reference” into string concatenation followed by execution.
Least Privilege
A rendering or preview process should not be able to traverse every user directory. Signed URLs or indirect references should be short-lived and revocable.
Failure Semantics
If a partial write fails, task state, stored objects, and references must either roll back together or enter an explicit failure state. Otherwise, the system may leave behind orphaned files that can later be referenced by another user.
If an evaluation reveals that the primary workflow succeeds but changing the user identifier allows access to another user’s asset, classify it as a blocking defect. Do not downgrade it as “something to fix later.” The fact that automated tests missed the path does not change its severity.
Reasonable Expectations for Long-Horizon Capabilities
Post-training and richer task environments generally improve a model’s persistence over long sequences and the stability of its tool use. Models may be less likely to lose the goal midway, more willing to run tests, and better able to revise code after a failure. This does not mean that security reasoning is complete.
White-box code review can expose obvious missing checks, but full attack-surface analysis, environment differences, and business-specific security policies still depend on people and established security processes.
A safer operating model is to treat the model as a collaborator that can make sustained code changes, while keeping the permission model, threat checklist, and final acceptance behind human gates. If those gates are not passed, the change must not be merged, regardless of how complete the demonstration appears.
A Reusable Acceptance Checklist
Before delivery, confirm at least the following:
- Requirements’ invariants have been encoded as tests.
- Cross-module states are documented in a comparison table and match the implementation.
- Rejection paths have explicit assertions.
- Secrets, internal paths, and debug switches are absent from default configuration.
- Rollback or failure cleanup has been exercised at least once.
- A human has replayed the critical operations using a second identity.
When these conditions are met, long-horizon generated code can move beyond “it runs” toward “it may be considered for merge.” Without them, scores, demonstrations, and green primary-path tests should not be interpreted as evidence that the code is safe to deliver.
