LLM-Generated UEFI Graphics Programs: A Debugging Retrospective from Dead Code to a Working Build

Why Stress-Test AI Coding Tools in a Firmware Context

Most discussions about LLM code generation focus on scenarios that have an operating system underneath: business backends, frontend pages, or scripting tools. When code is flawed in those places, you usually still get an exception stack, an error line, or an interpreter you can simply run again.

A UEFI (Unified Extensible Firmware Interface) application is a completely different species:

• No operating system. The program runs in a bare environment where boot services are available but everything else is self-provided: no standard library, no runtime, no notion of a process, and no exception stack when it crashes.
• An extremely narrow debugging path. In most cases the only channel is serial console output. Without a visible screen, you can only guess.
• Long feedback loops. One code change means compile, package bootable media, reboot or remount the virtual machine image, then read the output — anywhere from tens of seconds to several minutes per cycle.
• Hardware semantics must be exact. Frame buffer address, scanline stride, pixel format, mode index: get any one of them wrong and the outcome is a black screen, a garbled screen, or a silent exit rather than a readable error message.

That is precisely what makes this a litmus test for code generation. The model cannot pass on “syntactically plausible” code; it has to genuinely understand the semantics of the platform.

Evaluation Design: Two Rounds, and Taking Over Someone Else’s Code

The exercise was deliberately split into two stages instead of asking for a finished product in one shot:

Stage	Goal	Acceptance criteria
Round 1	Move the program from non-runnable to runnable	Boots, enters the main loop, produces output on screen, responds to input
Round 2	Refinement and optimization	Clear structure, stable frame rate, complete error handling, no memory leaks

The more valuable variation was to take over a half-finished codebase generated by a different model. That code was not worthless: it was already split across entry point, main loop, game logic, rendering, and input, the modular intent was obvious, and the naming was reasonably clean. The only problem was the distance between “looks right” and “runs right.”

The point of this setup is that it moves the evaluation from generation ability to code comprehension and repair ability. In real engineering, maintaining someone else’s legacy code takes far more time than building from scratch, and what determines an AI’s actual value on a team is whether it can read unfamiliar code, locate a root cause, and make a minimal change.

Round 1: What AI Code Review Can and Cannot Catch

The first step was to have the model perform a static review and produce a defect list. Its output typically clustered around a few predictable categories:

1. Unhandled protocol acquisition failures. The code calls the locate interface but never checks the returned status, then dereferences a null pointer.
2. Unclear memory ownership. Whether the caller or the callee allocates the buffer, and who is responsible for freeing it, is left ambiguous.
3. Mode switching and resource lifetime. A display mode is set, yet information obtained before the switch keeps being reused.
4. Missing error paths. When initialization fails, the code neither prints the status code nor aborts; it falls straight into the main loop.
5. Boundaries and type widths. Assumptions about character encoding width, index types, and pixel channel order were never confirmed.

An important caveat: static review can surface “structurally suspicious points,” but it cannot replace runtime verification. The two genuinely fatal problems both only showed up once the program was running.

Problem 1: The program exits immediately after start — a dangling pointer

Symptom: the serial console prints the initialization log and “Starting Game Loop,” and then the process ends with no error.

The triage approach was to add checkpoint logs at key nodes, print the return status code at every step, and bisect the failing interval. The failure landed between the graphics output protocol’s mode query call and the mode set call.

The root cause was a misunderstanding of protocol semantics. When the graphics output protocol is asked to query a mode, it writes mode information into a caller-provided buffer and returns a pointer to that information; and that information is no longer guaranteed to be valid after the display mode set call — the underlying structures have already been reallocated or rewritten, so the original pointer refers to invalidated memory.

Illustrative code:

// Buggy version: reusing the old Info pointer after switching modes
EFI_GRAPHICS_OUTPUT_MODE_INFORMATION *Info = NULL;
UINTN InfoSize = 0;
Status = Gop->QueryMode(Gop, ModeIndex, &InfoSize, &Info);
Gop->SetMode(Gop, ModeIndex);
// Info is no longer trustworthy here; dereferencing it produces undefined behavior
// Fix option 1: query again after the switch
Status = Gop->QueryMode(Gop, Gop->Mode->Mode, &InfoSize, &Info);
// Fix option 2: use the mode information and frame buffer description
// from the protocol's current state directly
Gop->Mode->Info;
Gop->Mode->FrameBufferBase;
Gop->Mode->FrameBufferSize;

This kind of defect is very typical: the API call sequence is entirely legal syntactically — what is wrong is the assumption about lifetime. It is also a hotspot for AI-generated code. Models tend to write resource access in the conventional “fetch once, use repeatedly” pattern, whereas the ownership conventions of firmware interfaces are often counterintuitive.

Problem 2: The screen never appears — the wrong protocol identifier

After the crash was fixed, the program ran stably, but the screen stayed completely black.

The root cause was an incorrect global identifier at the protocol location stage. Graphics-related capabilities have left several identifiers behind in the firmware specification and its historical implementations: the current graphics output protocol, a deprecated older drawing protocol, and a protocol aimed at text output. They serve different purposes and expose different interfaces. Using the wrong one does not produce “error and exit”; it produces a failure to locate the interface. The return value was ignored, so every subsequent call was invalid.

Engineering takeaways:

• After locating a protocol, always check the return status; on failure, print the status code and terminate. Never continue execution with a null pointer.
• Confirm the identifier and the interface definition from the specification documents, not from the model’s memory — the model’s training corpus mixes old and new protocols fairly often.
• Print the key capabilities during initialization: the maximum number of supported modes, the current mode index, and the frame buffer base address and size. Once those numbers are correct, later problems are much easier to pin down.

Turning “Black Box Failures” into “Locatable Faults”

The most expensive cost in firmware development is not writing code, it is “not knowing where it went wrong.” The following methods cut localization cost substantially:

1. Tiered serial logging. Separate initialization, per-frame, and error information, and make the per-frame log switchable off; otherwise it will drag rendering performance down.
2. Print all status codes. Output every return value in numeric form rather than a lossy “success/failure” label.
3. Checkpoint instrumentation. Print timestamps and status around key calls, and use bisection to narrow the failing interval quickly.
4. Minimal reproduction. First build a standalone runnable sample that lights up the screen and draws one rectangle, then layer game logic on top, avoiding multiple coupled factors.
5. Take a capability snapshot first. Once mode, resolution, pixel format, and scanline stride are confirmed correct, start writing the renderer.

The easiest trap in the render loop is the scanline stride. The number of bytes in each frame buffer row is usually larger than width multiplied by bytes per pixel, so computing offsets from the width alone skews or misaligns the image:

// Illustrative: convert scanline stride to a pixel address, and handle the pixel format explicitly
UINT8 *Base = (UINT8 *)(UINTN)Gop->Mode->FrameBufferBase;
UINT32 Stride = Gop->Mode->Info->PixelsPerScanLine;
UINT8 *Pixel = Base + (UINTN)y * Stride + (UINTN)x * 4;
*(UINT32 *)Pixel = 0x00FF0000;  // the exact channel order depends on the reported pixel format

You must read the pixel format reported by the protocol before deciding the channel layout; assuming RGB or BGR from experience is a common point of failure.

Involving an AI in Debugging the Right Way

Handing “the error” to a model can produce very different results depending on how you do it. What works:

• Provide complete context: the full serial output, the status codes, and the relevant code snippets — not a paraphrase of “it crashed.”
• Demand a root cause explanation: ask it to explain “why this happens” before producing a patch. A fix that skips the explanation usually just moves the symptom elsewhere.
• Constrain the change scope: explicitly require that only one module be touched, or only one line changed, so you avoid a large generated rewrite that makes attribution by bisection impossible.
• Require verification steps: ask what symptom should be observed after the change, and write the acceptance criteria into the instruction.
• Confirm item by item rather than accepting everything: for each change, ask “which specific semantic requirement in the specification is this based on.” Anything it cannot answer needs human verification.

The boundary that must be held is this: architecture decisions and semantic correctness are the human’s responsibility. Models are good at generating clearly structured, broadly covering code, but they lack reliable intuition about low-level platform ownership conventions, boot sequence constraints, and hardware behavior; those judgments require a developer to consult the specification.

Round 2: From “It Runs” to “It’s Maintainable”

The refinement stage is no longer about whether a feature exists, but about engineering quality:

• Layered decoupling. Split game logic, rendering, and input into modules that do not depend on each other; the logic layer should only output “what to draw” and never touch the frame buffer directly.
• Explicit resources and lifetime. Every allocation and free appears in pairs, and the frame buffer description is re-acquired uniformly after a mode switch.
• Rendering performance. Avoid per-pixel calls into high-level drawing interfaces; prefer whole-buffer copies and page flipping, or batch writes per scanline, to reduce call overhead.
• Input handling. Keys need debouncing and edge detection so that a long press is not recognized as continuous high-frequency triggering; also confirm whether reading the console input protocol can block.
• Build reproducibility. The firmware toolchain involves a specific set of compile and link options: freestanding compilation, wide character type width, disabling relocation optimizations, and specifying the executable subsystem. Any mismatch fails at packaging or load time. These options should be pinned in scripts rather than typed from memory.

If this work needs to be iterated on over the long term, the surrounding engineering matters just as much: put the build pipeline in a containerized environment to guarantee a consistent environment, archive every version’s boot image and build artifacts in object storage, and collect serial output centrally through a logging service so it can be reviewed later. These are all generic cloud building blocks, independent of any particular vendor.

A Note on Evaluation Fairness

A common objection is that “assigning different tasks to two models makes the comparison unfair.” That objection is methodologically entirely valid. To make a comparison convincing, at least the following variables need to be controlled:

• The same task, the same initial code, and the same hardware or virtual machine environment;
• Unified acceptance criteria, with an explicit definition of “runnable” (does it boot? does it produce output? can it interact?);
• Multiple runs with recorded variance, since a single result is heavily influenced by sampling randomness;
• Separately recorded human intervention time, including time spent reviewing code, copying logs, and judging root causes;
• Separated capability dimensions: first-run success rate, repair success rate, and the maintainability of the generated code — these should not be collapsed into a single score.

Conclusion

The core conclusion of this exercise is not new, but it is amplified in low-level development: LLMs can already compress the time from idea to a first runnable version substantially, and they are especially efficient at generating structured, modular skeleton code. In a bare environment with no operating system, however, their grasp of platform semantics is still unreliable, and the typical failure modes cluster around two categories of “syntactically correct, semantically wrong” problems: incorrect resource lifetime assumptions and incorrect interface identifier selection.

Four practical recommendations for developers: treat AI as a collaborator that produces drafts quickly and executes mechanical edits, not as the architecture decision maker; build observability with serial logs and status codes, which is the prerequisite for all efficient debugging; demand a root cause explanation and verification criteria for every change the AI makes; and pin the environment, build, and verification into a reproducible pipeline, so that every iteration’s judgment rests on facts rather than impressions.