AI Coding in Practice: What an 8-Day OS Kernel Demo Does and Doesn't Prove

1. Back to the technology: what can 8 days actually produce?

“Use AI to generate an entire operating system in a few days” spreads quickly in engineering communities, but the genuinely useful engineering information is usually far less than the headline suggests. The gap is not about generation speed, fast or slow. It is about the word “operating system” itself: in a demo context and in an engineering context, it does not refer to the same thing.

A prototype that boots in a generic emulator, prints a boot banner, and runs one minimal executable path is not the same artifact as a kernel that supports multitasking scheduling, virtual memory, device drivers, privilege isolation, and fault recovery. What separates them is not line count but specifications, boundaries, tests, regression, security, and maintainability.

So the first job when judging this kind of result is conceptual clarification:

• Bootable is not the same as usable. A working boot path proves that bootstrapping, memory layout, and basic I/O are connected. It does not prove that concurrency, interrupts, preemption, or resource reclamation behave correctly.
• Runnable is not the same as maintainable. Code produced in a single pass usually lacks abstraction layers, module boundaries, and an error-handling strategy, which makes later changes extremely expensive.
• Demoable is not the same as deployable. Demo environments are typically controlled, single-path, and free of real load. Production faces malformed input, concurrent contention, resource exhaustion, and human operator error.

Once these three distinctions are clear, there is a shared baseline for the discussion. The question is not “can AI write code,” but “can AI-generated output enter an engineering process and pass verification.”

2. From the “speed narrative” to delivery metrics

Vendor specifications and pricing, such as input and output token unit prices, price reductions across model generations, and the generation-speed multiplier from a fast mode, are real hard data. But they describe the economics of the call side, not the quality of delivery-side output. When enterprises select tools, these two things are easily conflated.

It helps to split the evaluation into two sets of metrics.

2.1 Call-side metrics (direct cost)

Metric	Meaning	Use
Input token cost per unit	Price per million input tokens	Estimating the cost of injecting context, documents, and code repositories
Output token cost per unit	Price per million output tokens	Estimating the cost of generated code, tests, and documentation
Generation speed multiplier	Throughput gain from a fast mode	Affects interaction feel and the duration of batch jobs
Context capacity	Volume of code and material a single session can hold	Determines whether cross-file refactors are feasible

2.2 Delivery-side metrics (what determines real value)

• First-pass rate: the share of generated changes that compile and pass static analysis without manual correction.
• Verification pass rate: the share of test cases that pass on the first run once code enters testing.
• Rework count: the average number of retry rounds a task needs before it can be merged.
• Regression introduction rate: the share of newly merged code that breaks existing functionality.
• Human review cost: the average review time per merged pull request.
• Unit delivery cost: the total cost of taking one complete feature from proposal to release, including token spend, human review, test environments, and failed retries.

The conclusion is direct: without delivery-side metrics, any judgment about a “capability jump” rests on feeling alone. Teams should establish their own baseline first, then compare new models or versions against it.

3. Four engineering constraints for AI-generated kernel code

When AI coding is applied to system-level code, the risk concentrates in four areas. Understanding these four matters more than remembering any tool name.

Constraint 1: Deterministic toolchains and environments

System code depends heavily on compiler version, linker scripts, target architecture ABI, and inline assembly conventions. When the generation environment and the actual build environment differ, the most common symptom is “the AI said it compiles, but it does not compile on the team’s machines.”

Engineering practice:

• Pin the build environment to a container image or a unified build machine, with version numbers under version control.
• Minimize manual intervention in build scripts; forbid any “fix it by hand locally and it passes” path.
• Restrict the generation step to syntax and contract checks, and leave the final verdict to a single unified build pipeline.

Constraint 2: Correctness needs evidence, not description

Large models explain code fluently and self-consistently, which has nothing to do with whether the code is actually correct. Correctness in system code can only be supported by execution evidence.

A minimal, executable verification ladder:

# Layered verification: a failure at any layer blocks the merge
set -euo pipefail
build_kernel        --target current_arch        # 1. Compile and link
static_analysis     --level strict               # 2. Static checks (memory, out-of-bounds, uninitialized)
unit_tests          --kernelspace                # 3. Unit-level cases
integration_boot    --timeout 120s               # 4. Boot and basic I/O
soak_test           --duration 30m               # 5. Long-running stability observation
fuzz_smoke          --seed "$RANDOM"             # 6. Stress with malformed input
perf_guard          --baseline baseline.json     # 7. Boot and scheduling metrics must not regress

The key point is that every layer must be repeatable and must decide pass or fail automatically. Otherwise the claim “the AI fixed it” can never be falsified.

Constraint 3: Permissions and data boundaries

Handing private corporate code, internal interface documentation, or customer data to an external model service is the most easily overlooked risk surface. Three lines should be drawn explicitly:

1. Data classification: separate public code, internal code, and code containing personal or regulated data, and route each class through a different channel.
2. Minimum disclosure: send only the context fragments strictly required to complete the task, rather than pasting an entire repository.
3. Audit trails and redaction: invocation logs, prompts, and responses must be auditable; keys, tokens, internal network addresses, and customer identifiers must be substituted before anything is sent.

For the internal channel itself, general-purpose building blocks can carry the load: object storage to hold code fragments and knowledge base snapshots, a message queue to absorb peaks and make inference requests asynchronous, a container platform to provide elastic resources for inference and build jobs, and a logging service plus monitoring and alerting to track success rate, latency, and anomalous calls. AI coding assistant products from Baidu AI Cloud can serve as one implementation option for such an internal channel, but the value of channel design lies in clear boundaries, not in being tied to any specific product.

Constraint 4: Sandboxed execution

Any “generate and immediately execute” step, meaning builds, tests, and fuzzing, must run in an isolated environment:

• A separate filesystem and a separate user, with no access to sensitive host paths.
• Default outbound network restrictions, allowing only necessary dependency sources.
• Hard limits on CPU, memory, process count, and execution time.
• Destroyed after use, with resources reclaimed as soon as the task ends.

4. How to design the evaluation: grade “AI writing a kernel”

Rather than arguing about whether AI can write an operating system, design a tiered scorecard that makes the conclusion reproducible.

Level	Capability description	Verdict method
L0	Generates syntactically correct modules that compile	Build passes with no warnings
L1	Generates a minimal kernel that boots	Boot completes on the target architecture
L2	Supports basic multitasking and memory management	Concurrency cases pass with no deadlock
L3	Runs stably for extended periods	No crashes and no sustained memory growth during the run
L4	Maintainable: stable interfaces, complete error handling, complete documentation	Passes independent review; changes stay local
L5	Safe for multi-tenant use	All isolation and permission cases pass

How to use the scorecard:

1. Fix the task set: prepare 20 to 50 real historical tasks, including bug fixes, feature implementations, and refactors, instead of textbook exercises.
2. Fix the baseline: the same machines, the same build environment, the same test cases.
3. Record the process, not only the result: keep each round’s prompt, the diff, the verification output, and the number of human interventions.
4. Cross-review: results are reviewed by people who did not participate in generation, avoiding the circular argument of “the AI wrote it and the AI says it is correct.”

After two or three iterations, the team will have a capability assessment more reliable than any marketing material.

5. Adoption path: put AI in the pipeline, not in the README

The recommended sequence is low risk, verifiable, and gradually expanding.

Step 1: Try it on non-critical paths. Start with unit test generation, log formatting, configuration conversion, first drafts of documentation, and comment backfills for legacy code. These tasks are cheap when they fail and cheap to verify.

Step 2: Bring it under static gates. Feed generated output into the existing static checks, type checks, dependency license scans, and secret scans, and reject non-compliant merges outright.

Step 3: Increase task complexity. Let it handle mechanical cross-file changes, such as unifying error-code style or adding parameter validation to interfaces in bulk, but keep the rhythm of small commits plus fast regression.

Step 4: Turn knowledge into assets. Compile team conventions, directory layout rules, naming habits, and prohibitions into specification documents that can be read programmatically, and use them as context at generation time. The payoff from this step is usually larger than switching to a stronger model.

Step 5: Measure and review. Summarize delivery-side metrics weekly and build a list of which tasks suit AI and which must be done by people, then adjust strategy accordingly.

6. Realistic expectations: translating the narrative into engineering language

Back to the original topic. Generating a bootable operating system with AI in a short period demonstrates the feasibility of generation along a specific path. It does not imply anything directly about replacing systems engineers. What actually determines value is:

• How verifiable correctness is: can pass or fail be determined automatically.
• How tolerable the cost of error is: is the repair cost acceptable when something breaks.
• Maintainability: when someone takes over the code six months later, can it still be changed.

These three points define the applicable boundary of AI coding. Judging from current engineering practice, it is best suited to tasks that are repetitive, clearly patterned, and automatically verifiable, and it requires the most caution for tasks related to security boundaries and concurrency or memory models, as well as tasks involving real data and external dependencies.

For teams, the actionable advice comes down to one sentence: establish your own baseline and verification ladder before evaluating the benefit of any tool. When numbers such as pass rate, rework count, and unit delivery cost are in your own hands, the capability claims and expectation swings in the market become reference information rather than decision inputs.