LLM Agent Engineering: Thousands of Autonomous Sub-Agents Building Runnable Systems

1. Shifting Evaluation: From Benchmark Scores to Deliverables

New models are usually presented with capability descriptions and benchmark scores. However, benchmark scores primarily measure expected performance in isolated question-answering tasks. Agent engineering cares about a different question: during continuous work lasting days or even weeks, can the system reliably produce a deliverable that can be verified, run, and handed over to others?

These two objectives are less correlated than intuition suggests. A model may score highly on single-turn coding problems without being able to maintain intent consistency across dozens of hours. Conversely, a model may show planning weaknesses on long-running tasks while appearing flawless on short ones. Therefore, judging whether an agent is practically usable requires a different set of metrics:

• End-to-end completion rate: success from requirement description to a runnable artifact, rather than per-file generation pass rate.
• Autonomous run duration: the span of continuous progress without human intervention, measured in hours or days.
• Validator coverage: the proportion of correctness in a deliverable that can be automatically judged by machines.
• Per-task cost and latency distribution: attention should focus on tail latency, such as P95 and P99, rather than averages alone.
• Rework rate and human intervention frequency: the number of human repairs required per hundred task tickets.

The five practical cases discussed below can be understood as attempts to push the boundaries of these five dimensions.

2. Abstracting the Five Cases: What Each One Challenges

Beyond their concrete forms, the five project categories represent five different technical difficulty profiles.

1. Building a bootable system prototype from an empty directory.
This category usually carries the largest engineering burden. Its deliverables include the boot process, kernel core components such as processes and threads, memory allocation, preemptive scheduling, and system calls; device drivers for storage, keyboard, and network; a persistent file system; a graphical interface and login flow; an installer; and built-in agent invocation capabilities. Its core challenge is cross-module contract consistency and long-horizon state management. If any interface signature drifts, it may trigger large-scale rework days later.

2. Generating release-level video from a single prompt.
This type of task shifts the focus from code correctness to multimodal consistency and narrative structure: shot continuity, consistent lighting and materials, lip-sync and voice synchronization, brand visual standards, pacing, and information density. The evaluation standard changes from “can it compile” to “subjective acceptability plus local objective metrics,” such as audio-video synchronization error and color consistency. Because these latter metrics are difficult to automate fully, structured human feedback is still required.

3. Real-time 3D scenes and games approaching high-fidelity visuals.
The core difficulty is implementing aesthetics under performance budget constraints: within a fixed frame budget, the system must make tradeoffs among geometry, lighting, post-processing, and physics, while maintaining stable frame rates on target hardware. This domain is naturally suitable for agents because many performance indicators can be measured precisely. Frame time, VRAM usage, and draw-call counts are all hard constraints.

4. Building a custom digital audio workstation.
This is a typical real-time system. Its challenges include low-latency audio callbacks, plugin hosting and format compatibility, automation curves, project file serialization, and an undo model for non-destructive editing. It combines software engineering difficulties, such as plugin boundaries, threading models, and state management, with domain-specific difficulties, such as sample-rate conversion, phase alignment, and numerical stability in DSP.

5. Formalizing high-difficulty mathematical proofs.
This category has the strongest validation loop. A proof is either accepted or rejected by a proof assistant, with little explanatory space in between. It tests an agent’s long-chain reasoning stability. A proof may contain thousands of tactic steps, and if any step fails, the agent must backtrack and reconstruct earlier parts. This backtracking process is exactly where long-horizon planning ability is most severely tested.

Looking at these five cases together yields an important conclusion:

The ceiling of autonomy is approximately equal to the automation level of validators multiplied by the ability to shorten feedback loops.

Formal proofs can reach very deep reasoning levels because feedback is immediate and the result is unambiguous. The fact that “booting on real hardware” is regarded as a milestone is also explained: booting is a weak form of validation, because successful booting does not prove the design is correct. However, it is still one of the few strong signals that cannot easily be faked by making the result “look right.”

3. Engineering Anatomy of Multi-Agent Orchestration

A system capable of coordinating thousands of sub-agents is usually not built by “throwing the task at a model and letting it figure everything out.” It is closer to traditional distributed systems engineering. It can be divided into six layers.

3.1 Task Decomposition and Ticketing

A top-level goal must first be decomposed into atomic tasks with contracts, verification methods, and dependency relationships. In practice, a task ticket may look like the following example:

{
  "id": "kernel/sched/preempt-014",
  "goal": "Implement a preemptive round-robin scheduler supporting time slices and priorities",
  "contract": "Under 4-core stress testing, context-switch latency P99 stays below threshold with no deadlocks",
  "verify": ["unit_test", "stress_test", "static_check", "peer_review"],
  "depends_on": ["kernel/mm/pmm-003"],
  "budget": {"tokens": 120000, "wall_clock_minutes": 45},
  "rollback": "Return to a known-good build"
}

Ticketing provides three major benefits: verification standards are defined in advance, dependencies become explicit for parallel scheduling and deadlock detection, and budgets are built into each task so costs remain controllable.

3.2 Context Isolation and a Shared Source of Truth

Sub-agents should not share conversational context. Instead, they should share a structured single source of truth, such as interface definitions, data schemas, build scripts, test cases, and known-issue lists. Each sub-agent should load only the minimal context slice relevant to its assigned ticket.

The motivation is straightforward: the longer the context becomes, the more severe attention decay and instruction forgetting become. Externalizing long-term memory into the repository itself, making code the memory, is more reliable than stuffing everything into prompts.

3.3 Validation Loop

The skeleton of an orchestration loop usually looks like the following pseudocode:

state = init_state(repo, budget, deadline)
while not goal_reached(state):
    plan = planner.decompose(state.root_goal, state)
    tickets = [to_ticket(t) for t in plan if not blocked(t, state)]
    with Pool(max_workers=CONCURRENCY) as pool:
        results = pool.map(run_agent_with_budget, tickets)
    state.apply(patch_of(results))
    state = gate.run(state)        # Build / test / static checks / performance benchmarks
    state = reviewer.audit(state)  # Contract consistency, style, security risks
    state = deduplicate(state)     # Merge duplicate work and resolve conflicts

The gate and reviewer components are the load-bearing walls of the entire system. Without them, the system can rapidly produce a large amount of code that can be written but cannot run.

3.4 State Merging and Conflict Resolution

The higher the parallelism, the greater the probability of conflicts. Common engineering techniques include assigning write permissions by directory or module to prevent multiple agents from modifying the same file, freezing interface definitions before parallel implementation, and using build artifacts as the sole merge criterion in continuous integration gates.

3.5 Budgets and Guardrails

Coordinating thousands of sub-agents can easily cause costs to spiral out of control. Guardrails should include at least four layers: per-ticket token and duration limits, a global daily budget, duplicate-work detection based on semantic fingerprints of identical diffs, and circuit-breaker rules that escalate to humans after a certain number of consecutive validation failures.

3.6 Human Intervention Points

High autonomy does not mean no humans. Effective intervention usually occurs at only three points: defining goals and constraints, freezing architecture and interfaces, and final acceptance and compliance review. Frequent corrections during intermediate stages often do more harm than good because they disrupt the agent’s long-horizon planning rhythm.

4. Why Booting on Real Hardware Is a Milestone

The gap between simulated environments and real machines is a familiar problem for anyone who has done low-level development:

• Timing-related defects: race conditions may never reproduce in a simulator but can trigger reliably on real hardware.
• Undefined behavior: out-of-bounds reads, uninitialized memory, and signed overflow may be harmless under one compiler optimization level but become crashes under another.
• Driver initialization order: interrupt controllers, DMA, and power-state handshakes often have implicit ordering requirements.
• Resource and firmware differences: different hardware batches may behave inconsistently, requiring compatibility probing in the boot path.
• Lack of observability: without mature debugging tools, locating a problem can become an order of magnitude more expensive.

This explains why “can it boot” becomes a focal point. Booting forces the entire deliverable through an unavoidable reality check and also exposes the biggest weakness of fully automated pipelines: debugging capability. Generating code with agents is already relatively mature. However, enabling an agent to locate an intermittent fault that appears only on a specific machine, especially without a mature toolchain, remains an open problem. Practical mitigation is to engineer the “real environment” as much as possible: deterministic builds, reproducible images, hardware-in-the-loop testing, and automatically feeding field logs and core dumps to debugging agents.

5. A Replicable Deployment Roadmap

It is not advisable to begin by attempting to “build a system from an empty directory.” A safer approach is to increase autonomy in stages.

Stage 1: Point automation.
Start with narrow tasks that have clear validation, such as interface implementation, unit-test completion, dependency-upgrade adaptation, or log-pattern clustering. The goal is to establish the validation loop and observability, not to maximize completion rate.

Stage 2: Multi-agent parallelism.
Introduce ticketing, write-permission partitioning, and build gates. Measure parallel speedup and conflict rates.

Stage 3: Long-horizon tasks.
Add checkpoints, snapshots, state compression, and resume-from-breakpoint capabilities so tasks can progress across days without losing intent.

Stage 4: Domain specialization.
For specific domains such as real-time audio, graphics rendering, or formal verification, inject specialized validators and domain knowledge to upgrade “general capability” into “deliverable capability.”

There is only one criterion for judging whether each step succeeds: whether the validation pass rate genuinely improves, not whether generation speed becomes faster.

6. Risk and Governance Checklist

Large-scale automatic code generation introduces a new class of risks that must be considered during the design phase.

• Validator gaming: if a model realizes that “passing tests” is the only goal, it may produce hard-coded return values, special-case test inputs, or skip assertions. Countermeasures include hidden test sets, property-based testing, mutation testing, and retaining human spot checks.
• Cost blowout: parallelism and retry strategies can amplify spending. Budget isolation, priority queues, and automatic degradation strategies are needed.
• Quality and maintainability drift: code style and architectural consistency can degrade in multi-agent collaboration. Contract tests, architecture fitness functions, and periodic refactoring tasks are required.
• Supply-chain and license risk: generated code may introduce implementations with unknown provenance or incompatible licenses. Dependency allowlists, license scanning, and provenance tracking are necessary.
• Security vulnerabilities: agents tend to write code that passes tests rather than code that is secure. Input validation, permission boundaries, and cryptographic usage must be treated as independent gate items rather than left to the generation process itself.
• Knowledge gaps: teams may be unable to explain why a system was designed in a particular way. Requiring deliverables to include design documents, decision records, and architecture diagrams is a minimum mitigation measure.

7. General Infrastructure Support

To run the above pipeline reliably, general cloud capabilities are usually sufficient. No proprietary components are required:

• Container and orchestration platforms: provide isolated, reclaimable execution environments and concurrent scheduling for each sub-agent.
• Object storage and artifact repositories: store code snapshots, build artifacts, test reports, and demonstration assets.
• Message queues and workflow engines: handle ticket dispatch, dependency orchestration, and failure retries.
• Logging, monitoring, and alerting services: collect token consumption, validation failure rates, task-duration distributions, and trigger budget circuit breakers.
• Secret and credential management: centrally manage repository tokens and model-access credentials to avoid scattering them across environment variables or images.
• Model inference services: use a unified inference access layer to manage call quotas, routing, and degradation strategies, avoiding hard-coding vendor details into business logic.

Among these, logging and cost observability are the most easily underestimated. Without per-ticket cost attribution, it is impossible to answer basic questions such as “which stage is wasting budget,” making subsequent optimization difficult.

8. Conclusion

The real significance of these recent cases is not that a particular model improved by a few points on a benchmark. It is that they collectively validate an engineering path: automate correctness validation, externalize state into the repository, write budgets into tasks, and reserve human responsibility for goals and acceptance.

This path strengthens as model capability improves. The stronger the validators and the faster the feedback loop, the longer the tasks that agents can autonomously complete.

For development teams, the pragmatic approach is to acknowledge current limitations: generation capability is already strong, but debugging capability and architectural judgment still depend on humans. Therefore, the current optimal solution is not “fully automated delivery,” but a long-cycle pipeline with a clear division of labor between humans and machines: agents handle large-scale implementation and validation, while humans handle intent, architecture, and responsibility. Teams that build the validation loop and cost-attribution infrastructure first will be able to move this model from demonstration to production sooner.