Evaluating Code Generation in Lightweight LLMs: A Geometric Construction Web Page Case Study
This article presents a method for evaluating the code generation ability of lightweight LLMs. The test asks a model to build a single-file HTML page that animates a neusis construction of a regular heptagon using a two-mark ruler. The article explains why this type of task separates models by instruction-following fidelity, compares a new lightweight model with a previous-generation full-strength model, and outlines evaluation practices and model selection considerations.
1. Background: Lightweight Flagship Models Punching Above Their Weight
In recent years, LLM product lines have become clearly tiered. Within a single technology family, vendors often release a larger, more capable full-strength version (often called the Max tier) alongside a lighter version with lower inference cost and more flexible deployment (often called the Flash tier). Conventionally, lightweight models are seen as cost-effective options that are “good enough.” They are expected to lag noticeably behind full-strength models on complex reasoning and long-chain code generation tasks.
However, recent industry observations suggest that some newly released lightweight flagship models now approach, and in certain dimensions even exceed, the previous generation’s full-strength models on specific high-difficulty tasks. Developers should take this “overperformance” seriously. If a lightweight model can complete tasks that once required a full-strength model at much lower inference cost, model selection strategies for intelligent coding assistants, automated code generation, and AI Agent orchestration will need a fundamental rethink.
This article does not focus on any specific vendor’s products. It treats the question as a general technical problem: how do you design a test task with enough discriminating power to scientifically assess a lightweight model’s real capability in complex code generation? The case study is implementing an animated compass-and-straightedge construction as a single-file HTML web page.
2. Why HTML Generation Works as a Test Vehicle
In LLM evaluation communities, asking a model to “draw a web animation” has long been a popular informal test. To be fair, such tests have a “toy” character. They cannot cover the full range of industrial software engineering, such as architecture design, concurrency handling, or security hardening. They are still valuable for three reasons.
First, it is a single-text output channel for multimodal ability. The model must express geometric intuition (spatial relationships among shapes), algorithm design (animation state machines, interpolation logic), and engineering implementation (Canvas or SVG rendering, event handling) in pure text output. A weakness in any one of these shows up directly on the final page.
Second, results can be verified immediately. Generate an HTML file and open it in a browser. You can judge correctness by eye without building an evaluation environment, so the feedback loop is very short.
Third, instruction following can be examined precisely. This is the most overlooked and most valuable point. A seemingly simple construction request can embed many constraints: simulation rules for the drawing tools, step order, the scope of visible traces, interactive features, and so on. A model may implement the instructions faithfully, or it may assemble a result that merely looks similar. On the surface the two may look alike, but they reflect how faithfully the model follows complex instructions. That is a core metric for engineering deployment.
3. Test Task Design: Regular Heptagon with a Two-Mark Ruler
The core prompt (paraphrased for this article) reads:
Create an HTML web page that animates the complete construction process of a regular heptagon using the “two-mark ruler” (neusis) method. The steps are as follows: draw a square and construct the perpendicular bisector of its bottom edge. Using the bottom-right vertex as the center and the diagonal as the radius, draw an auxiliary circle. Use the two-mark ruler, pivoting it against the bottom-left vertex, and slide the first mark along the auxiliary circle until the second mark passes exactly through the perpendicular bisector of the bottom edge. Afterward, complete the remaining steps following the classical compass-and-straightedge procedure, including completing the perpendicular bisector and determining the circumcenter. When the compass moves, it produces a trajectory, but traces should be left only where necessary. Finally, provide a button that exports the animation as a GIF.
The task has several layers of difficulty:
- Geometric understanding. The model must understand the principle of the two-mark ruler (neusis construction), a non-classical construction method. It is a straightedge with two marks at a fixed distance. By sliding and rotating it, both marks are made to lie on specified curves at the same time. This go
