Engineering an Operating System Prototype with Multi-Agent Collaboration

1. Why an Operating System Cannot Be Generated in One Pass

An operating system involves the boot flow, kernel entry, interrupt handling, memory management, process scheduling, device abstraction, file system, system call interface, user-space utilities, the build pipeline, and the test system. These parts are highly interdependent. A single defect can prevent the system from booting, or it can allow the system to boot while introducing stability and security risks.

This kind of system cannot be delivered through a one-time generation process. It requires an engineering workflow that is traceable, acceptable, and capable of rollback.

An agent is not simply a single-turn question-answering assistant. It is an execution unit that can receive goals, split tasks, execute actions, inspect results, fix defects, and advance delivery. Multiple agents form a closed loop through task orchestration, state sharing, validation feedback, and permission control.

Regarding scale, there are public reports of research teams using large agent clusters to carry out prototype operating system experiments and formal verification experiments. However, these cases differ widely in agent count, elapsed time, hardware environment, and the definition of “completion.” There is no unified and independently verifiable standard. Therefore, this article does not use specific numbers as the basis for conclusions. It only borrows the methodological skeleton: parallel execution + strict gates + continuous integration.

Readers often ask three questions:

1. How should task granularity be chosen for multi-agent collaboration?
2. What are common mistakes in multi-agent development?
3. What should be considered when building a validation loop for multi-agent systems?

These questions are answered in the following sections.

2. From Goal Layer to Infrastructure Layer: Architecture and Resource Budget

A layered architecture addresses the problem that even a strong single agent is not enough.

1. Goal layer: Break down “build a prototype operating system that can boot on a laptop” into verifiable goals. For example, the system should be able to boot, enter kernel mode, initialize memory, create processes, output logs, respond to basic commands, and pass smoke tests.
2. Orchestration layer: Generate a task graph and manage dependencies, priorities, and execution order. For example, memory management should be completed before process management, device abstraction depends on interrupt mechanisms, and file system work depends on storage access interfaces. The orchestration layer must also prevent agents from editing conflicting modules at the same time.
3. Execution layer: Assign roles such as architecture planning, module development, code review, testing, integration, documentation, and defect repair. This avoids placing the entire context on a single agent.
4. Validation layer: Apply multiple gates, including compilation, unit tests, static analysis, smoke tests, boot tests, regression tests, and performance tests. Artifacts that fail validation should not be merged into the main branch.
5. Infrastructure layer: Provide code repositories, artifact repositories, message queues, logging, monitoring and alerting, sandboxes, task queues, permission management, and audit records.

Many designs miss the resource budget layer. It is recommended to make this an explicit part of the orchestration layer. Parameters should include the maximum number of concurrent agents, token and compute budgets per task, task queue rate limits, per-task timeouts, circuit-breaker thresholds for consecutive failures, and degradation policies for queue backlog.

Without budget constraints, multi-agent systems simply amplify errors faster.

3. How to Choose Task Granularity

If tasks are too coarse, multiple agents may repeatedly modify the same module. If tasks are too fine, scheduling and integration overhead can become uncontrollable.

The judgment criteria are simple:

1. The task is split along interface boundaries.
2. The task can be independently verified.
3. The task dependencies are clear.

For a prototype operating system, the task list may include:

• Boot and startup
• Memory management
• Processes and threads
• Scheduler
• Device abstraction, starting with simulated devices
• File system
• System calls
• User-space utilities
• Build and artifacts
• Testing and validation

Testing tasks must advance together with development tasks. They cannot be left until the end.

A reusable task template is shown below:

task_id: memory_manager
owner_agent: dev-mem-01
reviewer_agent: review-mem-01
goal: Provide physical page allocation, virtual page mapping, and page reclamation capabilities
inputs: boot_info, page_table_spec
outputs: src/mm/*, docs/mm_api.md, tests/mm_unit.c, evidence/mm_report.json
dependencies: boot_task, interrupt_task
timeout_sec: 5400          # Timeout terminates the task; no zombie tasks are left behind
token_budget: 400k
compute_budget: 30         # CPU-hour
max_retries: 2             # Escalate to human review after exceeding this limit
gate: unit_cov >= 70% and boot_smoke_pass
evidence_path: artifacts/mm/<commit>/
acceptance: |
  After 1e6 allocation/release cycles, RSS fluctuation <= 5%;
  100% of invalid addresses and out-of-bounds mappings are caught;
  No deadlock occurs under 8 concurrent threads;
  No leak occurs after 30 minutes of continuous operation.
rollback: git revert <merge_commit>, revert to mm_api v1

Quantitative acceptance criteria are essential. A statement such as “no serious leaks” is difficult to judge. A statement such as “after 1e6 allocation/release cycles, RSS fluctuation is no more than 5%” is testable and enforceable.

4. Role Division and Conflict Resolution

The main roles are:

• Architecture planning: Define interfaces and data structures, but do not implement all functionality.
• Module development: Implement modules, write unit tests, and produce documentation.
• Code review: Check boundary conditions, exception paths, resource release, and concurrency safety.
• Testing: Create test cases, inject faults, and produce defect reports.
• Integration: Handle merges, dependency conflicts, and build failures.
• Defect repair: Locate problems using logs, stack traces, call chains, and change records.
• Documentation and tracking: Maintain task status, change records, and risk assessments.

The role permission matrix is as follows:

Role	Read source code	Write source code	Merge to main branch	Change gates	Delete data	External network access
Architecture planning	✓	Interface definitions only	✗	✗	✗	✗
Module development	✓	Own modules only	✗	✗	✗	✗
Code review	✓	✗	✗	✗	✗	✗
Testing	✓	Test code only	✗	✗	✗	Read-only mirror sources
Integration	✓	Conflict resolution only	✓	✗	✗	✗
Defect repair	✓	Authorized modules only	✗	✗	✗	✗
Documentation and tracking	✓	Documentation only	✗	✗	✗	✗

Interface conflict handling is as follows:

Conflict type	Handling method
Concurrent modification of the same file	Module ownership plus file-level task locks; late-arriving tasks queue
Interface signature change	The architecture agent makes the final decision; version number increases; the old interface is retained for one iteration
Dependency order reversed	The orchestration layer reorders tasks and blocks downstream tasks instead of forcing parallel execution
Builds break each other	Each agent uses an independent workspace and branch isolation; the integration agent performs unified merging

High-risk actions, such as deleting data, changing boot configuration, adjusting permission policies, formatting operations, or accessing external resources, must go through manual approval or an allowlist.

Multi-agent systems can expand execution scale. They cannot replace engineering governance.

5. Executable Validation Loop and Gate Checklist

Validation gates cannot rely only on abstract variable names. Each gate must clearly define the command, pass criteria, environment version, timeout, and retry limit.

env: Ubuntu 22.04, gcc 12.2, binutils 2.38, qemu 8.0, python 3.11
image_digest: sha256:<locked image>
gates:
  - name: build
    cmd: make clean && make defconfig && make -j$(nproc)
    expect: kernel.elf is produced, and warning count is 0
    timeout_sec: 300
  - name: static_analysis
    cmd: cppcheck --enable=warning,style,performance --error-exitcode=1 src/
    timeout_sec: 240
  - name: unit_test
    cmd: make test-unit        # cmocka / GoogleTest / KUnit
    expect: All tests pass, and line coverage >= 70%
    timeout_sec: 600
  - name: boot_smoke
    cmd: qemu-system-x86_64 -kernel kernel.elf -serial stdio -nographic -no-reboot
    expect: Within 60 seconds, stdout shows "boot ok" / "mm init" / "sched init" / "init ok" in order
    timeout_sec: 90
  - name: stability
    cmd: scripts/boot_loop.sh --runs 100 --fail-fast
    expect: 100/100 successful boots; any failure fails the entire batch
    timeout_sec: 3600
  - name: memory
    expect: Steady-state RSS fluctuation <= 5%, available physical memory >= expected value, no allocation failure logs
  - name: perf_baseline
    expect: Boot time P95 <= 3s, context switches >= 10k/s; alert if regression > 10% compared with the previous version
retry: The same task is automatically retried at most 2 times; the third failure escalates to human review

Integration should follow five steps:

1. Build a minimal skeleton that can be built and booted stably.
2. Freeze interfaces, then develop modules in parallel.
3. Use the integration agent to run boot tests and distinguish boot, memory, interrupt, or dependency-order issues.
4. Harden stability by covering exception paths, boundary conditions, leaks, and concurrency conflicts.
5. Add performance and observability checks, including boot time, memory usage, context-switch frequency, system call latency, and log error rate.

For a prototype system, stable booting and maintaining basic capabilities are more valuable than adding many features that frequently crash.

6. What Formal Verification Can and Cannot Do

The goal of formal verification is not merely to find more sample errors. It is to mathematically prove that a property holds under a given model. Examples include:

• Two processes will never hold the same lock at the same time.
• The memory allocator will not return an out-of-bounds address.
• The state machine will not enter an illegal state.

The workflow is as follows:

1. Convert a natural-language objective into a formal expression.
2. Generate proof strategies, including lemmas, induction hypotheses, and contradiction paths.
3. Explore multiple proof paths in parallel, with the verifier checking each step and invalid paths pruned.
4. Provide failure feedback, distinguishing unclear definitions, insufficient lemmas, type mismatches, or contradictory models.
5. Combine sub-proofs and generate reviewable records.

A simplified process is:

while goal_not_proved:
    lemmas = propose_lemmas(goal)
    proofs = parallel_search(lemmas)     # Multiple agents search in parallel; the verifier decides each step
    if verifier.valid(proofs):
        return assemble(proofs)
    goal = refine_goal(goal, verifier.feedback)

Common tools include Lean, Coq, and Isabelle/HOL. Each has its own library ecosystem and automation strategies. The value of multi-agent systems is that they can explore many paths in parallel and quickly eliminate incorrect directions.

The limitations must be clearly stated:

• Conclusions are valid only when the model semantics and assumptions hold.
• If the model is over-abstracted, the proof may be correct but irrelevant to the real system.
• State-space explosion and search-depth limits prevent many properties from being fully automated.
• Axiomatic assumptions still require manual review.

Formal verification checks the relationship between a model and a property. Testing discovers defects in samples. They complement each other and cannot replace each other.

7. Security, Audit, and Risk Levels

• Sandbox isolation: Agent code, commands, and tests should run in restricted environments. Network access, file writes, privilege escalation, and access to sensitive resources should be limited. Different modules should use independent workspaces.
• Least privilege: Documentation agents should not modify kernel code. Test agents should not modify production artifacts. Repair agents should not bypass validation gates.
• Audit trail: Task inputs, execution logs, code changes, test results, artifact versions, and failure reasons should be fully retained. A retention policy of 90 days online and 1 year archived is recommended for traceability and compliance.
• Checkpoints and rollback: Each successful merge should produce a rollback-capable version. For failures, preserve the failure scene, including source diffs, build logs, serial output, memory snapshots, and reproduction commands. High-impact changes should use staged validation, dual-branch comparison, and manual review.
• Risk levels:
  • L1: Documentation and tests; automatic release.
  • L2: Module-level code; automatic validation plus sampling.
  • L3: Boot configuration, memory protection, and permission models; mandatory manual review.
  • L4: Sensitive areas such as network security or life sciences; compliance review, data masking, manual confirmation, and access control are required.
• Manual review checklist:
  • Are assumptions explicitly listed?
  • Are boundary conditions covered?
  • Are permission changes minimized?
  • Are failure paths recoverable?
  • Has rollback been verified?
  • Is the evidence complete and reproducible?

8. Enterprise Adoption: Four Phases and Quantitative Gates

Phase 1: Documentation and test generation

Generate interface documentation, test cases, boundary descriptions, change summaries, and defect reports.

Acceptance criteria:

• Documentation coverage >= 80%
• Executable test case rate = 100%
• Manual sampling defect rate <= 5%

Phase 2: Module-level development

Select modules with clear interfaces, few dependencies, and sufficient tests.

Acceptance criteria:

• Unit test coverage >= 70%
• Static analysis has zero errors
• First-pass review rate >= 60%

Phase 3: Integration tasks

Perform merging, building, and boot testing.

Acceptance criteria:

• 20 consecutive integration builds succeed with a 100% success rate
• Boot smoke test passes 100/100
• Manual intervention <= 1 time per week

Phase 4: Critical-path audit

Introduce formal verification or expert review for critical state machines, permission models, memory protection, and protocol consistency.

Acceptance criteria:

• 100% of critical properties have proof or review records

The infrastructure can use vendor-neutral solutions. For example:

• Container services for isolated tasks
• Object storage for code artifacts and test reports
• Message queues for task events
• Logging services for execution records
• Monitoring and alerting for failure rates and resource anomalies
• Key management services for credentials

When selecting infrastructure, clarify compatibility conditions and compliance review requirements. Do not bind the system to a specific cloud vendor.

9. Common Mistakes

• Task granularity is too coarse, causing multiple agents to repeatedly modify the same file.
• Task granularity is too fine, causing scheduling overhead to exceed the benefit.
• Parallel development starts before interfaces are frozen, leading to semantic drift during integration.
• Validation gates contain only variable names without commands or thresholds. This is equivalent to having no gates.
• Module ownership and task locks are missing, so agents overwrite each other’s code.
• Rollback paths are never verified, so an incident requires a full rebuild.
• Dependency order is wrong, causing downstream tasks to run before upstream dependencies are ready.
• Retry limits are missing, causing invalid loops to consume excessive compute resources.
• Evidence is not persisted, making failures impossible to reproduce.
• Formal verification is treated as a replacement for testing.
• Unverifiable external cases are used as the basis for decisions.

10. FAQ

How should task granularity be defined for multi-agent collaboration?

The upper limit should be a task that one agent can complete within a single context and that can be independently accepted. This usually corresponds to one module or one cohesive group of interfaces.

Are more agents always better?

No. The benefit comes from parallelism and gate quality. The number of agents is constrained by merge conflicts, integration cost, and compute budget. Beyond a certain point, adding more agents produces negative returns.

What should be considered when building a multi-agent validation loop?

The loop should have executable gates, locked environment versions, bounded retries, reproducible failure scenes, and automatic escalation to human review when thresholds are exceeded.

Can everything be delegated to agents?

No. High-risk actions, permission changes, and conclusions in sensitive domains must be confirmed by humans. Multi-agent systems improve execution efficiency, but responsibility boundaries remain with people.

One-sentence conclusion

The core of using multi-agent collaboration to build complex systems is not increasing the number of agents. It is building an engineering system that is decomposable, orchestrable, verifiable, and rollback-capable. To judge whether a solution is reliable, check six things: Are tasks clear? Are interfaces stable? Is validation automated? Are failures locatable? Are changes rollback-capable? Are high-risk actions controlled?