How 1,000 AI Agents Built an Operating System: The Engineering Behind Long-Horizon Autonomous Development

1. From Writing Code to Delivering Software: The Difficulty Has Shifted

In one publicly reported experiment, a research team started from an empty folder. More than 1,000 agents worked continuously for eight days. Without humans writing the core code, they produced a complete system containing a kernel, disk and network device drivers, a filesystem, a graphical interface, and an installer. The system successfully booted on a laptop.

Such results are easy to interpret as “the model has become stronger.” From an engineering perspective, however, the more important finding is different: models have reached the conditions required to serve as basic execution units. As a result, the center of engineering effort has shifted from “can the model write correct code?” to “how can a large number of execution units cooperate stably over a long time scale?”

When autonomous development is observed through a software engineering lens, the difficulty concentrates in four areas:

• Time scale: A single conversation is measured in seconds, while an engineering cycle is measured in days. Across many rounds, early context is truncated or summarized with distortion, making it very easy for agents to forget the overall architecture.
• State consistency: Multiple execution units may modify the same codebase at the same time. This requires version control, dependency coordination, and conflict resolution. Otherwise, the more tasks are completed, the messier the repository becomes.
• Correctness verification: A system such as an operating system has three simultaneous correctness requirements: compilation, runtime behavior, and hardware interaction. Generation alone cannot confirm whether the delivered artifact is usable.
• Coordination overhead: The cost of communication, synchronization, and repeated computation grows with parallelism. Higher parallelism is not always better.

2. Layered Architecture: The Control Plane and Data Plane Must Be Separated

Scalable autonomous development systems generally use a two-layer structure. This also follows the common distributed-system principle of separating the control plane from the data plane:

• Control plane: task decomposer, orchestration scheduler, state store, quota management, and permission management. It only decides “what to do, who does it, and how to verify completion.” It does not perform the actual coding.
• Data plane: a large number of worker execution units running in containerized sandboxes. Each unit holds an independent code copy and a toolset, including file read/write, compilation, testing, and command execution.

The key is a clear responsibility boundary. The control plane holds the global objective and acceptance criteria, while the data plane holds only local tasks and local context. When a single execution unit fails, the cost of retry is limited to one subtask.

Task Decomposition and the Dependency Graph

Operating system development is naturally hierarchical: boot loading, kernel, memory management, scheduling, drivers, filesystem, system call interface, user-space interface, and installer. A reasonable approach is for the orchestration layer to first build a dependency graph, such as a directed acyclic graph, and then drive parallel execution.

An operating system only becomes debuggable and self-verifiable after it can boot. Therefore, the critical path in this kind of task usually lies in low-level components and integration work. Scheduling strategies should reserve compute and validation resources for the critical path rather than distributing them evenly.

How Execution Units Communicate

Implicit shared-memory-style collaboration is unreliable at large scale. In engineering practice, three explicit mechanisms are more commonly used:

Mechanism	Suitable scenario	Engineering focus
Shared filesystem / object storage	Code artifacts, logs, intermediate results	Directory-level isolation and versioned paths to prevent overwrites
Message queue (task and event streams)	Task dispatch and state-change notifications	Idempotent consumption, dead-letter queues, timeout-based redelivery
Structured handoff documents	Context transfer across stages	Use file contracts instead of natural-language summaries

Isolation and Permissions

Each execution unit should run in an independent container with limits on CPU, memory, and execution time. Network egress should be opened only when needed. Code repository credentials should use short-lived tokens and grant only the minimum necessary permissions.

An autonomous execution system is essentially a remote interface capable of executing arbitrary code. Without isolation, it effectively exposes production-like environments to automated processes.

The following orchestration-layer code is illustrative. It focuses on validation gates and retry budgets rather than specific framework interfaces:

# Core orchestration loop (illustrative; framework details omitted)
def run_task(task, ctx):
    """Execute one subtask and merge only after passing the validation gate."""
    for attempt in range(task.max_retries):          # Retry budget: prevents infinite loops and wasted spend
        workdir = ctx.sandbox.new_workspace(task.id) # Independent sandbox and code copy
        report = ctx.executor.run(
            prompt=build_prompt(task.spec, ctx.global_contract),
            workdir=workdir,
            tools=["fs_read", "fs_write", "shell", "git"],
            timeout=task.timeout,
        )
        checks = ctx.verifier.run_all(workdir)       # Compilation + unit tests + static checks
        if checks.passed:
            return ctx.vcs.merge(task.branch, checks.evidence)  # Merge only with evidence
        ctx.feedback.append(task.id, checks.diagnostics)  # Feed failure diagnostics back to guide the next attempt
    return TaskResult(task.id, status="failed", log=ctx.logs.dump(task.id))

Progressive Collaboration Emerges

It is worth noting that in the experiment, the agents did not start by following a manually designed organizational structure. Instead, they spontaneously formed division of labor and coordination mechanisms during collaboration.

The engineering explanation is this: when the task scale exceeds the capacity of a single context, and the verifier can continuously provide objective feedback, the division-of-labor structure that allows validation to pass faster is naturally selected. This means the focus of system design should be on providing sufficiently good feedback signals, not on prescribing the organizational structure in advance.

3. Context Engineering: Keeping Long-Running Tasks from Drifting

Most failures in long-running tasks are not caused by “not knowing how to write code,” but by “forgetting the constraints.” In practice, context is supplied in layers:

1. Global contract layer: architecture conventions, interface definitions, coding standards, and acceptance criteria. This content is short and stable, and it is injected in every round.
2. Task-local layer: the specification of the current subtask, relevant file fragments, and dependency descriptions.
3. On-demand retrieval layer: codebase indexing and retrieval, used only when needed to avoid stuffing everything into the context.

Three practical lessons deserve emphasis:

• Structured handoff is better than natural-language restatement. Passing interface definition files, test cases, and validation reports across stages is much more reliable than asking an agent to “summarize what was done in the previous stage.” The latter loses details and introduces hallucination.
• Compression must preserve constraints. When summarizing context, prioritize retaining interface signatures, boundary conditions, and acceptance criteria. Process descriptions may be discarded.
• Verifiers are the most efficient context. Compiler errors, test failure stack traces, and remaining proof obligations from formal checkers are signals with very high compression ratios and low misleadingness. Instead of writing longer prompts to describe requirements, it is better to build stronger automated validation.

4. Closed-Loop Validation: The Root Condition for Long-Running Convergence

Whether an autonomous system can converge after dozens of hours depends on the quality of the “generate–verify” loop. The loop needs to satisfy four conditions:

• Deterministic verification: the same input produces the same conclusion.
• Sufficient coverage: real defects can be caught.
• Low cost: verification can be executed frequently.
• Locatable feedback: the feedback can indicate where the problem is.

Formal Verification Is the Strongest Loop

Translating mathematical proofs into code that can be checked line by line by a machine is tedious but highly verifiable labor. It therefore becomes a typical scenario for testing autonomous engineering capability.

One study compared the following conditions on a classic number-theory result:

Condition	Time	Outcome
Internal research model + human guidance + custom framework	11 days	Completed
1,000 agents based on a new model, with only minimal prompts and a list of intermediate results	30 hours	100% completed; action consumption was about one-third of the first version, and proof code was reduced by about 7x
Agents of the same scale using an older-generation model	30 hours	Progress stalled at about 27%, and largely stopped after 18 hours

These numbers illustrate different issues.

The comparison between 30 hours and 11 days shows that for decomposable verification-heavy work, parallelism has an approximately linear leverage effect.

The reduction of action consumption to one-third and the roughly 7x shortening of proof code show that the newer model’s outputs are closer to a machine-checkable form, reducing the back-and-forth cycle of “write something wrong, discover the error, and rewrite.”

The fact that the older-generation model stopped at 27% and stagnated after 18 hours shows that the failure mode of long-running tasks is often “stagnation” rather than “slow progress.” When there is no effective internal self-correction, agents can fall into local loops.

This also points to a core metric for judging whether an automation system can handle long-running tasks: not the output in the first hour, but whether it is still converging at hour 18.

Non-Formal Domains Also Depend on Validation

In physics and hardware-oriented work, validation methods become numerical simulation, dimensional and conservation-law checks, and regression testing on extreme cases.

A typical example is an astrophysics criterion that has been proven under a large-scale approximation. However, at extremely small scales requiring a quantum-theory description, although the computational framework was listed as early as 2003, the derivation was so complex that no one completed it for many years. For this kind of problem, the bottleneck is validation cost rather than a lack of ideas. That makes it well suited to large-scale parallel agents.

5. Cost Model: Why “Cheaper Models + High Parallelism” Can Win

Public information indicates that, compared with the previous generation, a newer intelligent coding model improved output speed by about 30%, reduced API pricing by about 20%, and offered a high-speed tier priced at about twice the standard rate while providing about 2.5x speed. In multiple benchmark tests, it ranked first in all but one. For engineering teams, this leads to a counterintuitive but important conclusion: in high-parallelism autonomous development scenarios, a cheaper but slightly weaker model may perform better overall than a more expensive but stronger model.

The reason lies in the cost structure. The cost of a single request can be approximated as:

Cost ≈ InputTokens × P_in + OutputTokens × P_out

Agent systems have two characteristics in token consumption. On the input side, repeatedly replayed context and tool outputs dominate. On the output side, there is a large amount of trial and error. Therefore, the benefit of a lower unit price is amplified by the number of replays.

For example, if each task averages eight cycles of “execute–verify–correct,” and each cycle replays 50k tokens of context, then context replay accounts for most of the total cost. In that case, a 20% reduction in unit price is almost proportionally transmitted into a reduction in total cost.

This leads to four general optimization strategies:

1. Context reuse: enable caching for stable prefixes to avoid repeated billing and repeated computation.
2. Output-side stop-loss: set maximum retry counts, per-task timeouts, and budget limits to prevent stagnant loops from consuming compute.
3. Model routing: use strong models for high-value steps such as planning and architecture decisions; delegate file retrieval, formatting, and batch mechanical edits to lower-cost models.
4. Validation front-loading: place cheap and deterministic checks, such as compilation, static analysis, and unit tests, before expensive checks such as full test suites or formal verification, so incorrect branches can be eliminated early.

6. Safety Governance: Self-Improvement Capability Must Be Explicitly Constrained

In the case discussed above, the detail that deserves the most attention from an engineering perspective is not the upper limit of capability, but an active restriction: the vendor limited the new model’s recursive self-improvement, or RSI, capability during the training phase, ensuring that its self-improvement level would not exceed the previous flagship model.

The design logic is clear. For a system that can modify its own training and optimization processes, the speed of capability improvement is no longer determined by external evaluation cadence. Safety assessment would lose a baseline.

For practical deployment, at least five layers of defense are recommended:

• Capability-tiered gates: grant graded authorization for autonomous code modification, command execution, network access, and writes to production environments. High-level capabilities should require human confirmation.
• Environment and production isolation: autonomous processes should not directly touch production credentials or data. Outputs must pass through an independent review and release pipeline.
• Auditable traces: retain complete prompts, tool calls, commands, and outputs for every agent, so that any line of code can be traced back to its source.
• Anomaly circuit breakers: set thresholds for cost, duration, failure rate, and resource usage. When exceeded, the system should automatically pause and alert.
• Dependency and supply-chain verification: autonomously generated code may introduce uncontrolled dependencies, so source and license validation should be mandatory.

When deploying this kind of system in the cloud, container platforms can provide isolation and elastic compute. Object storage can host shared workspaces and artifacts. Message queues can handle task and event distribution. Logging services and monitoring alerts can provide observability. These capabilities belong to general infrastructure categories, and specific choices should be evaluated based on the team’s existing technology stack and compliance requirements.

7. A Practical Rollout Path

It is not advisable to jump directly to thousand-agent parallelism. A feasible path can be divided into four steps:

• Step 1: Single agent + strong validation. Select a target with a complete verifier, such as refactoring a module with a full test suite. Get the “generate–verify–correct” loop working, then measure single-task success rate and average retry count.
• Step 2: Small-queue parallelism. Introduce a task dependency graph and sandbox isolation. Start with a parallelism level of 5 to 10. Observe conflict rates and merge success rates. The key is to understand the relationship between task granularity and conflict rate.
• Step 3: Introduce a dedicated verification agent. Assign an independent role to run tests, locate failures, and generate diagnostic reports. This frees coding agents from noise in their context. This step is often the dividing line between success and failure for long-running tasks.
• Step 4: Scale and cost governance. Enable model routing, context caching, quota control, and budget control. Add audit and circuit-breaker mechanisms.

Each step should have clear metrics: task pass rate, average convergence rounds, cost per effective output, and number of human interventions. Without quantitative metrics, scaling ultimately just parallelizes ineffective loops.

8. Conclusion

The fact that more than 1,000 agents delivered a bootable operating system within eight days indicates that an inflection point in the feasibility of autonomous software engineering has arrived. However, for this capability to become real enterprise productivity, what matters is not model parameters alone, but five engineering capabilities:

• Decomposable task modeling
• Isolated parallel execution
• Layered context supply
• Deterministic and inexpensive verification loops
• Auditable safety boundaries

At the same time, the detail that the vendor actively limited the model’s self-improvement capability is a reminder: the stronger the capability, the more important it is for humans to explicitly define the boundaries.

For developers, the most cost-effective direction today is to transform the parts of their business that are “clearly verifiable, decomposable, and highly repetitive” into automated pipelines. The returns from this work do not depend on the capability ceiling of any single model, but on the maturity of engineering design.