LLM-Assisted UEFI Graphics Application Development: A Practical Walkthrough
This article walks through an experiment in which a large language model takes over a firmware graphics project—a Tetris-style game—that a previous model left in a "won't compile" state, then repairs it through two rounds of iteration, first to a "runs" state and then to a "playable" state. It explains why graphics applications in the UEFI firmware environment are a demanding test for LLMs, covering the bare-metal runtime, GOP-based graphics, limited debugging, and an obscure build chain. Through two concrete faults—a crash after mode switching and a black screen caused by mixing up protocol GUIDs—it shows how to pair the model's general engineering strengths with human-supplied firmware domain knowledge, and closes with practical guidance on writing an environment card, asking in layers, feeding logs first, and keeping a constraints checklist on hand.
Foreword: This write-up follows an experiment in which a large language model took over a firmware graphics project that a previous model had left in a “won’t compile” state. Through two rounds of iteration, it went from “runs” to “playable.” For readability, I refer to the two roles as the “successor” (the model taking over) and the “predecessor” (the model that came before). The development approach, troubleshooting logic, and engineering practices described here should be useful to anyone using large language models to assist firmware development.
1. Why graphics applications in a firmware environment make a good test for LLMs
“Have AI write a game” has been one of the most popular ways to test a large model’s coding ability. But if you stay at the level of console text games, the difficulty is manageable: the build environment is ready to go, debuggers work well, and error messages are clear. Move the task into the UEFI (Unified Extensible Firmware Interface) environment—ask AI to draw a playable game directly during the BIOS phase—and the difficulty jumps sharply.
The reason is straightforward: a firmware environment has no operating system acting as a buffer layer. The model has to deal with several constraints at once—platform knowledge, memory management, hardware access, and debugging tools.
Specifically, there are at least four hard points:
The runtime environment is “bare metal.” UEFI applications run in a firmware environment before the operating system loads. There is no standard C runtime library, no ready-made file system, and no process scheduling. Problems like array overruns, null pointers, and wild pointers usually do not raise exceptions the way user-space programs do; instead they trigger a processor exception and a reboot.
Graphics capability depends on protocols. Drawing under UEFI essentially means manipulating a linear framebuffer through the Graphics Output Protocol (GOP). Setting the mode, obtaining the framebuffer base address, and the pixel format—each step has a strict order and lifecycle constraints.
Debugging options are limited. The program runs at the firmware layer, and after a crash there is usually nothing on screen. Developers often rely only on serial output, and serial printing itself is limited by buffer size and output speed.
The build chain is obscure. Going from source code to a bootable UEFI application image involves several stages: the open-source firmware build project, .inf/.dec build description files, the EDK II build system (EDK II is a widely used open-source firmware development framework in the industry), a cross-compilation toolchain, and image packaging. This kind of knowledge is relatively scarce, and models tend to have “fuzzy” memory of it.
There is also an easily overlooked prerequisite: a model only “thinks” within the environment you tell it about. Build commands, firmware version, virtualization platform parameters—if you do not provide any of these explicitly, the model will fill them in on its own, and the guesswork is usually wrong.
My practice environment
| Item | Description |
|---|---|
| Hardware platform | A virtual machine based on the Intel Q35 chipset, simulating a UEFI environment through OVMF firmware |
| Build approach | EDK II build system, loading by manually placing the application (DXE driver) |
| Debugging | Serial log output, as the main observation window |
| Graphics implementation | Calling GOP in the UEFI application, writing the framebuffer directly |
| Input | Polling the Simple Text Input Protocol |
| Test goal | Implementing a directly playable graphical mini-game in the firmware environment |
A project that “looks right” but won’t run
What makes this practice unusual is that the starting point was not a blank project but “second-hand code.” Earlier, another model had tried to write Tetris in the same environment; the result was “spent hours without even getting the build environment up, and ended up without a runnable program.” But the code it left behind “looked the part”: it was split into multiple files by responsibility—entry point, game logic, rendering, input handling—with clear module separation and mostly reasonable naming.
This reflects a common pattern: large language models have a solid grasp of what code should look like, but whether that code actually runs on your machine is another matter. Rather than throwing this code away, it makes excellent test material—having one model clean up after another tests both code-review ability and fault-localization ability.
2. Round one: from “won’t compile” to “runs”
The goal of the first round was clear: forget about experience, forget about performance—just get something on the screen. Throughout, I positioned the successor as a “senior programmer with eyes and hands”: give it enough information and let it look, change, and verify on its own.
Step 1: Make it a reviewer first, not immediately hands-on
Following the engineering principle of “think it through before acting,” I did not start by having the model change code. Instead I gave one instruction:
“Please review this code, list the problems you find, and prioritize them. Focus on build correctness, memory safety, and protocol usage.”
This choice was deliberate. Faced with an unfamiliar codebase, having a model modify directly tends to produce “patch-style fixes”—fix one thing, introduce three. Having the model complete a structured analysis first has two benefits:
- Build a global picture quickly. The model works through the entry point, protocol location, mode setting, render loop, and input handling one by one, forming a “map of call relationships.”
- Turn the problem list into a checklist. Each subsequent fix can be checked off, keeping progress under control and making rollbacks easier.
The successor produced a list of dozens of issues within minutes, roughly falling into three categories:
| Category | Typical problems | Consequence |
|---|---|---|
| Build config | Improper module type declarations, missing referenced-library references, missing DEC declaration for the protocol GUID | Compilation or linking failure |
| Memory safety | Null pointer dereference, unchecked buffer lengths, use after free | Runtime exceptions |
| Protocol usage | Non-standard protocol location, incorrect lifetime of the mode-info pointer | Initialization failure or crash |
The value of this list was not just “finding problems”; it also exposed the model’s way of thinking: it treated the firmware application as “an ordinary C program plus a few special APIs.” This led to two kinds of issues—it is good at “finding” general defects like memory-safety bugs, but not at “understanding” the special nature of the firmware environment, such as protocol dependencies, global variable initialization timing, and firmware variable services.
And the difficulty of “fixing” happened to fall right in that weak area. So the next phase was not “the model says change, I change, done”—instead a human needed to “translate” firmware-specific constraints to the model.
Step 2: The first real fault—crash after mode switch
After the model fixed the build issues one by one, the program finally compiled and loaded. A few lines of output appeared on the screen, and then it dropped straight back to the firmware shell.
I had it emit serial debug output on each line; the output stopped at “game loop start.” This meant the problem was almost certainly after mode setting.
After taking over the investigation, the successor gave its assessment: to determine the resolution, the code saved the pointer to the current mode info before setting the display mode; it then called the set-mode interface, but the render loop still accessed that old pointer. In a firmware environment, after a mode switch this info may be reallocated or updated, so continuing to read it becomes an illegal memory access.
This advice was quite precise. But the successor also cautioned that this was just “an inference based on the code” and had to be verified through serial logs. I added pointer-address and boundary prints at key spots; after rerunning, the logs confirmed the inference: the old pointer address was clearly different from the new one after the switch.
There were two fix approaches:
- Switch to a value copy. Copy the needed info into a local struct before setting the mode, no longer holding a pointer to protocol-internal data.
- Re-fetch after switching. After a successful mode set, re-query the mode info and obtain the new framebuffer base from it.
I chose the second, because it fits the general principle of “treat the protocol’s current state as authoritative” and the code is clearer:
// Old approach: save the mode-info pointer in advance and keep using itmModeInfo = GraphicsOutput->Mode->Info;GraphicsOutput->SetMode(GraphicsOutput, mModeIndex);// ... the render loop keeps reading mModeInfo, which may be stale by now// New approach: re-query after a successful mode setStatus = GraphicsOutput->SetMode(GraphicsOutput, mModeIndex);if (EFI_ERROR(Status)) {DEBUG((DEBUG_ERROR, "SetMode failed: %r\n", Status));return Status;}Status = GraphicsOutput->QueryMode(GraphicsOutput, mModeIndex, &Size, &mModeInfo);if (EFI_ERROR(Status)) {DEBUG((DEBUG_ERROR, "QueryMode failed: %r\n", Status));return Status;}mFrameBuffer = (UINT8 *)GraphicsOutput->Mode->FrameBufferBase;DEBUG((DEBUG_INFO, "Mode=%d, Res=%dx%d, Bpp=%d\n",mModeIndex, mModeInfo->HorizontalResolution,mModeInfo->VerticalResolution,mModeInfo->PixelsPerScanLine));
Two more points in this snippet are worth noting. First, I explicitly told the model to print the resolution and pixels-per-scan-line info after the query; it turned out that nearly every subsequent “can’t find the cause” moment was located using exactly these few log lines. Second, I asked the model to check not “whether the return value is 0” but to use the EFI_ERROR() macro, because UEFI status codes are not simple integers—they are a composite of “severity level + specific value.”
Step 3: The second real fault—the graphics interface doesn’t appear
Once the crash was fixed, a new problem came up: the program neither crashed nor errored, but the screen stayed black. The serial logs showed that the initialization flow had completed and the render loop was running, yet nothing was “drawn.”
The successor produced another list, and one item caught my attention: GOP is an “interface,” and to use it in a UEFI application you must locate it through the protocol database. Locating it requires providing a GUID (Globally Unique Identifier) as the “door number.” EDK II defines multiple GUIDs related to graphics output—for example, the GUID used by “Install Protocol Interface,” the GUID used to “extract images from the HII database,” and the GUID for the “Graphics Output Protocol” itself. Their names are similar but their purposes are completely different. Grab the wrong one and protocol location either fails outright or returns a pseudo-interface that “looks usable but actually does nothing.”
This kind of problem is classic in firmware development. The EDK II framework’s .inf file must declare protocol dependencies (a [Depex] of type CONDITIONAL or PROTOCOL), and the .dec file must export the corresponding GUID declaration. Declaration and reference correspond by GUID name; get one wrong and the build system usually will not complain—it only cares whether “this GUID exists,” not whether “the protocol you fetch with it is the one you wanted.”
This also made me realize that models actually have quite a good “memory” for this “same-family GUID mix-up” kind of problem—but only if you explicitly tell it in the prompt that “this is the protocol-location step.” Once context is missing, it tends to fill in with the “most common pattern,” and the most common pattern is not always the most correct.
The solution had three steps:
- Have the model list every GUID name referenced in the current code;
- Have it compare each one against “which protocol each GUID corresponds to and what its purpose is”;
- Remove the genuinely unrelated GUIDs from the dependency declarations, and add the protocol dependencies actually needed.
After the changes, the black screen was gone, and the game finally appeared on screen.
Round-one summary: the model is good at “general engineering,” humans supply “domain knowledge”
Looking back on round one, the successor’s performance can be summarized in three sentences:
- Strong code-review ability. Faced with unfamiliar code, it can quickly produce a structured, actionable list of improvements—more in line with engineering practice than “diving straight in.”
- Sound fault-localization thinking. Faced with the crash, it proactively considered the “pointer-lifetime” angle and stressed “verify with logs before concluding”—which is impressive.
- Blind spots in domain knowledge. On platform-specific knowledge like build config, protocol dependencies, and firmware variable services, its judgments often “look right but are actually wrong.”
The conclusion is clear: the model is a useful “general engineer” but not a “firmware expert.” The “last mile” of firmware development must be filled in by a human—turning implicit platform constraints into explicit context and feeding it to the model.
3. Round two: from “runs” to “playable”
Once the program ran, the goal of round two was “polish.” The focus this round was not features but interaction experience and code quality.
Step 1: Play first, then list problems
I played it myself for two minutes and noted a few issues:
- The block-fall speed was fixed—too slow a rhythm, lacking challenge;
- Key presses had latency; left/right movement did not feel responsive;
- Visual feedback for line clears and speed-ups was weak—you could not tell you had “scored”;
- Some key combinations responded abnormally in the firmware environment.
I handed these phenomena to the model as-is, and it gave a layered optimization plan:
| Layer | Optimization | Approach |
|---|---|---|
| Interaction | Key response | Lower the input polling interval, add key debounce |
| Interaction | Pace control | Blocks fall faster as the score rises |
| Visual | Feedback | Add a brief highlight frame on line clears |
| Visual | UI polish | Unify the color scheme, add borders, display the score |
| Code | Maintainability | Separate the “game logic” and “rendering” layers |
The most noteworthy point here is “key latency.” The cause was a fairly heavy delay wait in the render loop, which stalled input polling. The successor’s proposed fix was to “pull input handling out of the render loop and turn it into a separate polling thread”—which is entirely reasonable in an OS environment, but under UEFI there is no multithreading, so that advice was directly unusable.
I swapped it for a “fixed timestep + accumulated time” approach: use a timer to record the time delta, and when it accumulates past a threshold, trigger a block drop, while the loop itself keeps spinning fast. This preserved both input responsiveness and logic pacing. This little episode again shows that the advice the model gives is correct in “general programming” but must be taken with a grain of salt in a “firmware environment.”
Step 2: Have it self-review
I had it re-review its own code across four dimensions: “starts up correctly, logic correct, rendering correct, boundary handling.” It found and fixed several problems; the most typical was that after a line clear it did not promptly update the cache for “the next block’s type,” occasionally causing “a block to not be drawn.”
This problem would almost never surface in a general environment—it only shows up when “a block is genuinely missing on screen.” This again shows: AI-generated code must be validated in a real environment; static review can never replace running it.
Step 3: Add playability design
Finally, I had the model add some “non-functional” details:
- Add pause and restart functions;
- Display the current score and cleared-line count;
- Centralize block types, colors, and shape parameters in one place, making them easy to tune;
- Show a prompt when the game ends, rather than exiting silently.
These changes are all “small,” but they are exactly what turns a “program” into a “game.” The model’s execution efficiency is high, but the prerequisite is that you have to proactively tell it “how players will actually play.”
4. Treat AI as a “collaborator,” not a “contract factory”
After two rounds of effort, my view on “using large models for firmware development” has become much clearer. In one sentence: treat it as a senior programmer with broad knowledge and strong execution ability, but lacking environment context.
Around this positioning, four pieces of practical experience are worth expanding on.
1. Task definitions should be “acceptance-testable,” not just “describable”
“Help me write a UEFI mini-game” is a description, not a task. “Given a specific VM configuration, produce a bootable UEFI application that shows a start screen on launch, starts on Space, has blocks fall on a timer, supports left/right movement, rotation, soft drop, and line-clear scoring, and exits on Esc”—that is a task.
A task definition should include: the runtime environment (firmware version, VM configuration, screen resolution), the build approach (build command, target architecture, output path), debugging means (serial-log switch), and acceptance criteria (which behaviors count as passing).
My approach is to write the “environment card” clearly in one shot before the conversation begins:
[Environment Card]- VM: Q35 + OVMF, resolution 1024x768- Build: EDK II build system; output .efi, then manually place it in the firmware application directory- Debug: serial logs, emitted via the DEBUG macro- Graphics: UEFI GOP, linear framebuffer- Input: UEFI Simple Text Input Protocol- Goal: Chinese text input via double-pinyin, Enter to confirm
Carry this card into every conversation and the chance of the model “making things up” drops sharply.
2. Ask in layers; don’t make the model do everything at once
The right order for firmware development is: first get the build to pass, then initialization succeeds, then rendering succeeds, and only then interaction logic. If you dump all the requirements at once, the code it produces often “looks complete” but actually has problems at every layer, and you have no idea where to start troubleshooting.
My approach is “layered questions + verification per layer”:
- Round one: only require the build to pass, to load, and to print logs;
- Round two: only require mode setting to succeed, the framebuffer to be obtainable, and one pixel to be drawn;
- Round three: only require the render loop to run stably and to respond to the exit key;
- Round four: only then introduce the game logic.
This way, each round’s problems are confined to one “observable range,” and when something breaks you can immediately tell which layer is broken.
3. Let the model “see” the scene, not “imagine” it
Serial logs, screen captures, and error messages are the model’s “eyes.” I have made it a habit: whenever I hit a problem, paste the logs to it first, rather than first describing “I suspect it is here.”
A counterexample: I once directly told the model “the screen isn’t refreshing, maybe it’s a render-buffer problem.” As a result it spent a lot of space discussing “buffering strategy,” while the real cause was that mode setting had failed and the width/height had been written as 0. If I had pasted the Res=0x0 log line at the start, the problem could have been located in thirty seconds.
Letting the model see the scene saves more tokens and more time than letting it guess at causes.
4. Solidify “domain knowledge” into context, rather than re-explaining it every time
Firmware development has a batch of “implicit constraints,” such as:
- Protocol GUIDs have multiple “same-family” members with different purposes—they cannot be mixed up;
- A protocol interface pointer may become invalid after a mode switch and must be re-fetched;
- Global variables are not automatically zeroed in a firmware environment and need explicit initialization;
- String operations must use the firmware-provided safe versions, not the standard library;
- Serial printing should not be too fast, or logs will be lost.
I compiled these into a “firmware development constraints checklist” and appended it to every conversation. The effect was immediate: the model’s advice “got less wrong,” and the discussion efficiency clearly improved.
5. Some advice for peers
If you also want a large model to help with firmware or low-level development, here is the sequence I would suggest:
- Write the environment card first. State the platform, build approach, debugging means, and acceptance criteria clearly, and feed it to the model in one shot.
- Advance in layers. Build → init → render → interaction; verify one layer at a time, do not skip steps.
- Logs first. When you hit a problem, paste the logs and then have the model analyze; do not describe guesses first.
- Keep the constraints checklist on hand. Compile the platform-specific “pitfalls” into a list and bring it every time; it saves a lot of back-and-forth.
- Humans set direction, the model does implementation. How to layer the architecture, how to use protocols, how to handle boundaries—these decisions must be made by humans; how to write code, how to fix errors, how to generate boilerplate—these can be safely left to the model.
6. Closing: AI doesn’t “replace firmware engineers”—it “amplifies firmware engineers”
The model’s performance in this practice was commendable: organized code review, sound fault localization, and decisive refactoring. But it did indeed “crash and burn” on build config, protocol dependencies, and multithreading assumptions—and in very typical ways.
These “crashes and burns” are actually valuable. They show that the core difficulty of firmware development has never been “writing code” but “understanding the environment.”
Large models have already driven down the cost of “writing code” dramatically, but “understanding the environment” still requires humans: knowing UEFI’s boot sequence, knowing the dependencies between protocols, knowing when pointers become invalid, and knowing which “looks usable” interfaces are actually traps.
From this angle, the relationship between AI and firmware engineers is not about “who replaces whom” but “who amplifies whom.” The better you understand the environment and the more clearly you express the constraints, the higher the quality of AI’s output; and every compile-error-fix loop the AI saves you, the more time you have to think about the truly important questions—“how should the architecture be designed” and “how should boundaries be handled.”
AI is an amplifier, not a replacement. What gets amplified is always your own engineering ability.
