Evaluating Lightweight Flagship LLMs on Complex Geometry Code Generation

    This article explores the evaluation of lightweight flagship AI models through complex geometry code generation tasks. Unlike traditional HTML tests that often rely on templates, constructing a regular heptagon using a neusis demands rigorous mathematical reasoning, long-chain planning, and strict process adherence. We introduce a four-layer evaluation framework—focusing on results, process, rendering, and engineering—to expose "result-oriented" shortcuts. By comparing a lightweight model with a higher-capacity model, we demonstrate how process fidelity and instruction adherence can make lightweight models highly competitive for constrained engineering tasks, provided that latency and stability are properly managed.

    1. Evaluation Background: Lightweight Models Need “Process-Level” Evidence

    The typical value of a lightweight flagship model lies in balancing capability, cost, latency, and deployment flexibility. For developers, the real concern is rarely whether the model answers one more general question correctly, but whether it can consistently deliver in constrained engineering tasks: for example, generating runnable frontend code, completing complex geometric calculations step-by-step, maintaining state consistency during animations, and ultimately producing verifiable deliverables.

    Traditional coding and style replication tasks often suffer from a specific bias: a model can score points as long as it “looks like it finished.” Particularly in web rendering tasks, a regular heptagon can be achieved through hardcoded coordinates, pre-rotated polygons, or additional auxiliary steps. A correct final screenshot does not mean the model understood the requirements or that its construction process is credible. Therefore, evaluation should evolve from “correct answer” to “correct result + explainable process + verifiable constraints.”

    Complex geometric drawing happens to be an ideal stress test. It simultaneously covers mathematical reasoning, long-chain planning, coordinate system transformations, frontend rendering, animation state management, file exporting, and error recovery. More importantly, the task exposes “result-oriented” opportunistic paths: did the model strictly use the specified construction method? Did it complete the necessary steps? Did it fabricate non-existent steps that alter the method? Or did it merely output a visually similar shape?

    2. Test Task: Constructing a Regular Heptagon Using a Neusis

    This abstracted test requires the model to build a web page that demonstrates the process of “constructing a regular heptagon using a neusis (a two-marked ruler).” The task outline is as follows:

    1. Start with a square and construct the perpendicular bisector of its bottom edge;
    2. Draw a circle centered at the bottom-right vertex E of the square, with the diagonal as the radius;
    3. Using the neusis, keep it passing through the bottom-left vertex D. Slide the ruler so that the first mark moves along circle E until the second mark passes through the perpendicular bisector of the bottom edge;
    4. Complete the omitted compass and straightedge construction steps;
    5. When using the compass, only keep necessary traces and avoid generating irrelevant movement marks;
    6. Finally, provide the functionality to export the animation as a GIF.

    This task seems like simply “drawing a heptagon and animating it,” but the actual difficulties stack across multiple layers.

    First, the geometric construction itself is uncommon. A regular heptagon is not a simple equal division of a circle. The sliding constraint of the neusis involves a coupled condition: “passing through a fixed point, the first mark on the circle, and the second mark on the perpendicular bisector.” If a model merely memorizes “divide the circumference into seven equal parts,” it easily bypasses the method specified in the prompt.

    Second, the process constraints are strict. The task doesn’t just ask for the final heptagon; it requires advancing along the specified construction path and completing the omitted compass and straightedge steps. The model must distinguish between “necessary additions” and “extra operations that alter the method.”

    Third, rendering and animation requirements are high. The web page must not only draw the static result but also organize key traces, auxiliary lines, and the sliding process into an understandable animation, while supporting GIF export. The model must manage system state, frame sequences, coordinate precision, and export timing.

    Fourth, acceptance dimensions are numerous. The final page might be visually correct with smooth animation, but the construction steps may have deviated. Alternatively, the steps might be correct, but the animation has too many traces, the export fails, or the code is unmaintainable. Multi-dimensional acceptance is required.

    3. Why This Test Is More Valuable Than Standard HTML Tasks

    “Web generation tests” are often criticized as toys because many can be solved by piecing together templates, component libraries, or fixed styles. However, by adding process constraints, geometric invariants, animation states, and export capabilities, it becomes a small end-to-end engineering task rather than just a styling exercise. It tests the following capabilities:

    • Long-term planning: Deconstructing points, lines, circles, auxiliary lines, movement constraints, and animation phases before generating code;
    • Mathematical correctness: Ensuring the final polygon is a regular heptagon and that vertices, edges, circles, and auxiliary lines satisfy geometric relationships;
    • Instruction adherence: Adopting the specified construction path strictly without arbitrarily adding steps just to achieve the result;
    • Code organization: Separating construction logic, rendering logic, animation timing, and export logic;
    • Verifiability: Outputting construction logs, step descriptions, or structured intermediate results;
    • Error handling: Detecting issues like export failures, missing frames, incomplete animations, or unclosed circular movements;
    • Visualization quality: Necessary auxiliary lines, readable animation, and clear final state.

    Thus, the value of this task lies not in whether the page looks cool, but in compressing the model’s long-chain reasoning process into an observable, repeatable, and scorable deliverable.

    4. Evaluation Framework: Four-Layer Acceptance Instead of Just Screenshots

    To make the evaluation reproducible, it is recommended to split acceptance into four layers: Result, Process, Rendering, and Engineering.

    4.1 Result Layer: Geometric Invariants Must Hold

    The final shape should at least satisfy:

    • The output polygon has 7 vertices;
    • All side lengths are equal within a tolerance range;
    • The 7 vertices lie near the same circumscribed circle;
    • The angles between adjacent vertices are approximately equal;
    • The relative positions of the center, auxiliary lines, square, and circle match the given construction;
    • The animation retains necessary auxiliary traces at the end without excessive clutter.

    Simple assertions can describe the final acceptance. For example:

    1. function assertRegularHeptagon(points, eps):
    2. assert points.length == 7
    3. sides = distance between adjacent points
    4. assert max(sides) - min(sides) < eps
    5. calculate the circumscribed circle center and radius
    6. for p in points:
    7. assert abs(distance(p, center) - radius) < eps
    8. for adjacent vertices:
    9. assert approximately equal angular spacing

    This part only proves that the “result looks right,” not that the “process is correct,” so it must be combined with the next layer.

    4.2 Process Layer: Construction Traces Must Be Auditable

    It is recommended to require the model to output a structured construction record simultaneously, and then generate the page based on the record. The record doesn’t need to follow a fixed framework but should include step numbers, operation types, input objects, output objects, and whether it constitutes a necessary auxiliary line. For example:

    1. {
    2. "steps": [
    3. {"id": "s1", "op": "point", "name": "D"},
    4. {"id": "s2", "op": "square", "base": "bottom"},
    5. {"id": "s3", "op": "perpendicular_bisector", "of": "bottom_edge"},
    6. {"id": "s4", "op": "circle", "center": "E", "radius_ref": "diagonal"},
    7. {
    8. "id": "s5",
    9. "op": "neusis",
    10. "pass_through": "D",
    11. "mark1_on": "circle_E",
    12. "mark2_on": "bottom_bisector"
    13. }
    14. ],
    15. "render": {
    16. "necessary_traces_only": true,
    17. "export_gif": true
    18. }
    19. }

    During acceptance, distinguish between two types of “extra steps”: one type involves the explicit requirement to “complete the omitted compass and straightedge construction,” which is a legitimate addition; the other involves fabricated operations that alter the construction path just to tune the shape to the correct answer, which should be judged as a process deviation. In other words, you cannot simply score by “number of steps,” but must verify whether each step can be derived from the prompt or classic construction rules.

    4.3 Rendering Layer: Animation and Export Must Be Observable

    For the rendering layer, it is recommended to check:

    • Whether the animation covers key construction phases rather than just displaying the final heptagon immediately;
    • Whether compass traces appear only at necessary positions;
    • Whether auxiliary lines appear and fade out by phase;
    • Whether the final GIF exists, has a reasonable number of frames, and its final frame matches the static page;
    • Whether the export process is triggered after the animation ends;
    • Whether the page maintains proportions and readability across common window sizes.

    Scoring can separate “visually similar” from “structurally correct.” Only when both hold simultaneously can the model be considered to have completed the task.

    4.4 Engineering Layer: Stability, Latency, and Recoverability

    Production environments cannot rely on a single success. The following must be recorded:

    • End-to-end generation time;
    • Time to first valid output;
    • Whether the reasoning stream stalls for extended periods;
    • Whether it outputs only the thought process without the final code;
    • Whether it can continue or retry after a timeout;
    • Whether the generated code is readable, runnable, and free of dangerous executions;
    • Whether the export process blocks the page;
    • Whether there is a fragile implementation relying solely on fixed canvas dimensions.

    For web generation tasks, generated code should also be run in an isolated browser sandbox to prevent arbitrary scripts from accessing sensitive environments. Evaluation should look at not just “can it draw it,” but “is it suitable for the engineering pipeline.”

    5. Insights from an Abstract Comparison Experiment

    In an abstracted comparison, two models with different positioning used the same prompt and similar acceptance criteria. One leaned towards a lightweight flagship positioning, while the other leaned towards higher capability. The observations were highly representative: the lightweight model exhibited intermittent stalls in the reasoning stream, even showing “thinking without outputting” in some phases, leading to significantly longer end-to-end wait times. However, the final completed version strictly followed the specified construction path, and its animation and step organization were closer to the prompt requirements. The higher-capability model quickly produced a visually reasonable regular heptagon but added extra steps not in the prompt, forcibly achieving the final shape by altering the construction path.

    This result indicates at least three things.

    First, model capability cannot be judged solely by the “final screen.” A page may look correct, but that doesn’t mean the process meets the constraints. In geometric drawing, process orchestration, and compliant generation, process fidelity itself is a core metric.

    Second, lightweight models do not necessarily equate to low capability. In tasks with clear boundaries, explicit acceptance criteria, and step-by-step requirements, lightweight models may demonstrate better instruction adherence and long-chain constraint retention. Their potential advantage isn’t “fewer parameters,” but the ability to cover structured code generation, visual calculations, and constrained process orchestration at lower resource costs.

    Third, latency and stability must be factored into selection. Some models may experience reasoning stuttering, output interruptions, or prolonged periods without a final answer, which affects user experience and cost. Lightweight does not automatically mean fast; first-token latency, output rate, end-to-end completion rate, failure retry costs, and manual intervention costs must be evaluated together.

    It must be emphasized that this is a small-sample observation and cannot be directly generalized into a universal ranking. Model outputs are stochastic; temperature, context, toolchains, and runtime environments all affect results. The correct approach is to incorporate such tasks into a fixed evaluation set, sample multiple times, and record the process, rather than drawing conclusions from a single demo.

    6. Reproducible Evaluation Pipeline

    It is recommended to organize complex geometry code generation evaluation into the following pipeline:

    1. Task templating: Retain constraints like geometric construction, animation, and export, but randomize point names, canvas ratios, side lengths, and auxiliary line transparency to reduce the payoff of hardcoding;
    2. Plan before coding: Require the model to output the construction plan or operation sequence before generating frontend code;
    3. Structured tracking: Mandate the output of step logs for automated checking;
    4. Static checks: Inspect scripts for obvious anomalies, infinite loops, dangerous executions, or uncontrollable external dependencies;
    5. Sandbox rendering: Open the page in an isolated environment, take screenshots, and record console errors;
    6. Geometric assertions: Parse the final point set to verify the regular heptagon and auxiliary relationships;
    7. Animation verification: Extract frames to check if key steps appear, if the GIF exists, and if the final state is consistent;
    8. Process review: Manually or programmatically check for ungrounded extra steps;
    9. Stability retesting: Sample multiple rounds to track success rate, time consumption, stall rate, and fix counts;
    10. Fix loop: Upon verification failure, feed specific failure reasons back to the model, asking it to fix only the unmet items instead of rewriting the whole thing.

    7. Engineering Implementation Suggestions

    To apply such capabilities to real-world business, it is recommended to adopt a “phased generation + automated acceptance + fallback strategy.”

    • Use lightweight models for clear-cut construction code, component generation, parameterized graphics, and formatted output;
    • Use higher-capability models for requirement clarification, error handling, complex refactoring, and multi-option comparisons;
    • Specify deliverables in the prompt: construction step logs, static page, animation logic, and GIF export instructions;
    • Automate geometric validation, screenshot comparison, export checks, and timeout monitoring as quality gates;
    • Implement heartbeat and timeout strategies for “thinking without outputting,” supporting partial result saving, breakpoint resumption, or degraded generation;
    • Run generated pages in a sandbox to avoid directly loading untrusted code;
    • Record latency, retry counts, and manual fix time for each call, selecting models based on Total Cost of Ownership (TCO) rather than unit price.

    8. Practical Checklist

    Finally, here is a reusable checklist:

    • Did you require the model to output structured construction steps?
    • Did you score “correct result” and “correct process” separately?
    • Did you check necessary auxiliary lines and reject extra steps that alter the method?
    • Did you validate the side lengths, circumscribed circle, and angular spacing of the regular heptagon?
    • Did you check animation keyframes, trace necessity, and the final GIF state?
    • Did you record first-token latency, end-to-end time, stalls, and failure rates?
    • Did you run the generated code in an isolated environment?
    • Did you perform multiple rounds of sampling to avoid misjudging based on a single result?
    • Did you set up failure retry and lightweight/high-capacity model switching strategies?

    9. Conclusion

    Complex geometry code generation is not just a “drawing test.” It is a comprehensive task compressing long-chain reasoning, mathematical constraints, frontend implementation, animation states, and file export. It effectively exposes “result-oriented” opportunistic behaviors: the page might be pretty, the shape might be correct, but the construction path may not meet the requirements. For lightweight flagship models, the advantage worth noting is not just whether a single generation succeeds, but whether it can maintain process fidelity, output auditability, and failure recoverability under explicit constraints. Advancing evaluation from screenshot comparison to process auditing brings model capability assessment closer to real-world engineering delivery.