Evaluating Lightweight Models on Complex Geometry Tasks: A Reproducible Method and Checklist

    This article evaluates lightweight models on a complex geometry interaction task rather than simple question answering, short text generation, or simple code completion. Using the construction of a regular heptagon with a neusis ruler as a probe, it defines reproducible evaluation dimensions, pass criteria, experimental setup, failure modes, and a reusable checklist. The focus is whether a model can follow constraints, maintain geometric dependencies, complete omitted steps, and produce verifiable output. It also discusses limitations and engineering compensation.

    1. Evaluation Background: What Should Be Measured Is Procedural Compliance, Not Just Whether the Drawing Looks Correct

    The role of lightweight models is growing in high-frequency invocation scenarios. They offer low inference cost, fast response, and easy integration into business workflows, giving them engineering value in real-time interaction, edge computing, batch content generation, and front-end-assisted development. However, whether a model is “lightweight” does not mean it is sufficient. This cannot be judged by basic question answering, short-text generation, or simple code completion. What truly distinguishes the capability ceiling is whether the model can handle tasks involving multiple constraints, multi-step reasoning, state dependencies, and visual output.

    This article selects a complex geometry interaction task as the evaluation sample. The concern is not merely whether a page can be generated, but whether the model:

    • Strictly follows the original constraints;
    • Maintains complex geometric dependencies;
    • Keeps consistency across long step sequences;
    • Completes implicit steps and omitted steps.

    Compared with “generating a webpage,” this kind of task is closer to a real development scenario and better defines the applicability boundary of lightweight models in engineering practice.

    2. Task Design: Why “Constructing a Regular Heptagon with a Neusis Ruler” Is an Effective Probe

    The core task is to dynamically demonstrate, in a webpage, the construction process of a regular heptagon based on a neusis ruler.

    First, the concept needs to be clarified to avoid misinterpretation as ordinary geometric drawing. A neusis, also known as a marked straightedge, is a straightedge with two fixed marks. During construction, the body of the ruler must be placed against a specified point, and the two marks must fall on two given objects, such as lines or circles. This differs fundamentally from compass-and-straightedge construction. A regular n-gon constructible by an unmarked straightedge and compass requires n to be the product of a power of 2 and distinct Fermat primes. The number 7 does not satisfy this condition, so a regular heptagon must be constructed with stronger tools such as a neusis ruler or angle trisection.

    This determines the nature of the task. If a model tries to “guess” the result, it can only rely on approximate coordinates or preset graphics. It cannot genuinely complete the construction within the specified tool chain. The task itself therefore provides a natural filter for process adherence.

    The prompt can be decomposed into the following key constraints:

    1. A square exists on the page;
    2. Construct the perpendicular bisector of the square’s bottom edge;
    3. Draw a circle centered at the bottom-right vertex with a radius equal to the diagonal;
    4. Use the neusis ruler to complete the subsequent construction;
    5. The neusis ruler stays tightly against the bottom-left vertex of the square;
    6. The first marked line of the neusis ruler moves on a circle;
    7. The construction condition is satisfied when the second marked line crosses the perpendicular bisector of the bottom edge;
    8. Complete the omitted conventional compass-and-straightedge steps;
    9. During compass movement, keep only necessary trails and do not produce redundant visual traces;
    10. Support dynamic demonstration and export as an animated image.

    This task covers five dimensions: mathematical logic, interaction constraints, geometric state, visual presentation, and output format. It can effectively distinguish “result simulation ability” from “process reasoning ability.” The former achieves visual similarity through fixed coordinates, extra auxiliary lines, approximation algorithms, or embedded preset graphics. The latter derives steps according to constraints and implements verifiable geometric relationships in code.

    3. Evaluation Dimensions and Pass Criteria

    Dimension Core checks Pass criteria Weight Common failure examples
    Instruction coverage Compare all 10 constraints one by one ≥9/10 constraints can be located in code or rendering results 20% Missing “keep only necessary trails” or “export animated image”
    Geometric correctness Vertices, perpendicular bisector, circle center, radius Parseable in code, and runtime numerical validation error < 1e-6 25% Wrong circle center using top-left or bottom-left; radius using side length
    Step legality Whether undeclared steps are added Number of auxiliary objects equals the necessary minimum; no unauthorized circles, points, or transformations 20% Introducing extra auxiliary circles to “make” intersection points
    State consistency Linkage among points, lines, circles, ruler marks, trails, and frames Contact conditions do not fail after ruler movement; no self-contradictory states 15% Contact judgment hard-coded; animation detaches midway
    Visualization quality Structure, recognizability, animation, trails, export Page runs, and exported file can be opened by a standard player 10% Trails become blurred; export throws errors
    Output stability and latency TTFT, total elapsed time, interruption rate No truncation, no long period with zero output, final deliverable is complete 10% Stalling after segmented output

    Overall pass line: weighted total score ≥75, and the single-item scores for “geometric correctness” and “step legality” must be ≥80% as veto items. If any item is below 60, the model is considered unusable for this type of task, preventing a high overall score from masking key defects.

    4. Experimental Setup: Four Registration Sheets for Reproducibility

    Registration item Required content
    Model and environment Model identifier, version, parameter scale, quantization method; rendering engine and version, browser, runtime device, API concurrency, and temperature settings
    Prompt Full text retained without simplification or reduced difficulty; explicit prohibition statements must be included, such as not forcing a result with nonexistent extra steps
    Acceptance checklist Item-by-item checklist as shown in Section 9, with pass criteria and weights for each item
    Human review rules Two reviewers work independently; disagreements are resolved by a third reviewer; focus on step legality and whether the result deviates from the original intent

    Automation and human review should be divided clearly. Automated checks can parse key variables, circle parameters, perpendicular bisector logic, and trail-control functions in generated code to improve efficiency. Human review cannot be omitted because it is used to judge step legality, visual reasonableness, and intent consistency. When comparing different models, it is essential to distinguish “model capability differences” from “runtime environment differences”; otherwise, environment limitations may be misjudged as model defects.

    5. Experimental Group Results: Good Performance in This Sample

    Experimental registration table, with example values; actual configuration must be used during reproduction:

    • Model identifier: M-light-a
    • Version: v2025.09-preview
    • Parameter scale: 8B, with 2B activated parameters
    • Deployment: 4×A10, using a vLLM inference service
    • Temperature: 0.2
    • Rendering environment: Chromium 126 + Canvas 2D
    • Sample size: n=5, with the same prompt invoked 5 times
    • Timing definitions: TTFT, meaning first-token latency; total elapsed time, meaning from sending the request to the artifact being usable; interruption rate, meaning the proportion of cases that fail to produce a runnable file
    • Acceptance criteria: see Section 3

    In this sample, the model performed well. It did not stop at generating a static graphic but attempted to fully reproduce the construction process:

    • Constraint decomposition: It broke the task into six stages: basic graphics → circle center and radius → ruler contact and movement → completion of compass-and-straightedge steps → trail control → animated export. Coverage reached the pass line.
    • State maintenance: The ruler position, contact between the first mark and the circle, and the relationship between the second mark and the perpendicular bisector remained linked. Contact judgment was computed in real time rather than hard-coded.
    • Implicit knowledge completion: It completed conventional steps that were not listed one by one, such as the method for constructing the perpendicular bisector and determining key intersection points, showing the ability to invoke domain knowledge.
    • Engineering implementation: It generated a playable animation and export functionality, turning reasoning results into a runnable artifact.

    The conclusion is limited as follows: under the prompt and experimental group configuration in this article, the lightweight model completed this case. This does not represent its general performance on similar tasks.

    6. Control Group and Failure Modes: Correct Results Do Not Guarantee Correct Process

    Control model definition: A model with a higher capability positioning, generally considered stronger in complex reasoning and code generation, using the same prompt, environment, and acceptance criteria. Judgment basis: the checklist in Section 3, code review records, and runtime logs.

    It should be noted that if control experiment data is not publicly disclosed, the following failure modes should be expressed as risks rather than established conclusions. These tasks have the risk of the following failure modes, especially when the prompt does not specify prohibitions.

    1. Directly drawing a regular heptagon with fixed coordinates rather than deriving it from the construction process;
    2. Adding auxiliary steps that do not exist in the prompt in order to forcibly satisfy the visual result;
    3. Simplifying the neusis condition into an ordinary straight line or slider while ignoring the constraints between ruler marks and the circle;
    4. Sacrificing geometric step legality for animation effects;
    5. Keeping only the final graphic while omitting intermediate construction trails;
    6. Failing to implement “keep only necessary trails” and drawing many redundant traces.

    How failure evidence should be retained: generated code snippets, including coordinate constants and contact-judgment functions; checklist results; and runtime log timestamps. Such evidence is more convincing than screenshots. It is especially important for business stakeholders because code generation, workflow orchestration, and structured output all depend on the model’s intermediate process. Once task conditions change, the stability of a model that “guesses the result” will decrease significantly.

    7. Core Capabilities Identified

    • Long-prompt understanding: Maintaining multiple constraints simultaneously within a limited context and meeting the coverage target.
    • Multi-step reasoning: Clarifying the dependency order of objects and knowing what to do first and what to do later.
    • Implicit knowledge completion: Compass-and-straightedge construction, neusis rulers, and regular heptagons are not common front-end requirements, so the model must invoke domain knowledge without one-by-one explanations.
    • Front-end engineering implementation: Converting mathematical logic into runnable code, including drawing, animation, events, state updates, and export.
    • Task stability: Staying on the main line during long-chain tasks, avoiding repeated generation, and not switching solutions midway.

    8. Limitations and Engineering Compensation

    Output speed and stability are the primary shortcoming. Evaluation needs to record three metrics: TTFT, total elapsed time, and interruption rate. These can be supplemented by tokens/s and the human-review pass rate. Complex geometry tasks often come with a long reasoning process, and excessively high TTFT will directly harm the real-time interaction experience.

    Compensation measures include:

    • Real-time interaction scenarios: Split subtasks, use asynchronous calls with progress feedback, render stage results first, and set reasonable timeouts and degradation strategies;
    • Cost and latency: Cache results, pre-generate outputs, route simple requests to lightweight models, and escalate complex requests to higher-capability models;
    • Precision: Geometry tasks involve coordinates, distances, and angles; high-precision scenarios require numerical validation modules and automated tests;
    • Front-end performance: Animations, trails, and multi-element rendering require performance optimization, browser compatibility checks, and resource-usage evaluation;
    • Observability: Logs, monitoring alerts, and call-chain tracing are needed to locate bottlenecks.

    9. Reusable Evaluation Method: From a Single Case to a System

    1. Task grading design

    Level Content Validation target
    Basic Draw simple graphics Front-end basic capability
    Combination Combine and lay out multiple elements Coordinates and layout
    Constraint Add mathematical relationships and geometric conditions Reasoning ability
    Interaction Add animation, events, and state changes Engineering implementation
    Extension Export, compatibility, performance, error recovery Comprehensive capability

    2. Checklist, including pass criteria and weights

    Check item Pass criteria Weight
    Square exists Four vertex coordinates are parseable; side lengths are equal and adjacent sides are perpendicular 6%
    Bottom-edge perpendicular bisector Passes through the midpoint of the bottom edge and is perpendicular to it; parseable in code 8%
    Circle center equals bottom-right corner of square Runtime assertion center == corner passes 10%
    Radius equals diagonal length Runtime assertion ` r - diag < 1e-6` passes 10%
    Neusis structure exists Contains an object with two marked points and a tangency constraint 8%
    Ruler touches bottom-left vertex Anchor distance remains 0; frame-level validation 6%
    First marked line lies on the circle Each frame satisfies ` p1-center - r < 1e-6` 8%
    Second marked line crosses the perpendicular bisector Each frame satisfies ` p2.x - midX < 1e-6` 8%
    Omitted steps are completed Conventional compass-and-straightedge steps are complete, with no logical gap 10%
    Only necessary trails are kept No redundant items in the trail set; confirmed by human review plus counting 10%
    Animation playback is supported Continuous playback without detachment or frame jumps 8%
    Animated export is supported Exported file can be opened by a standard player 6%
    No irrelevant auxiliary steps Number of auxiliary objects equals the necessary minimum 2%
    Total 100%

    3. Introduce automated validation

    1. // Assumed screen coordinate system, with the y-axis pointing downward.
    2. // A: top-left, B: top-right, C: bottom-right, D: bottom-left
    3. const square = {
    4. A: { x: 0, y: 0 }, B: { x: 1, y: 0 },
    5. C: { x: 1, y: 1 }, D: { x: 0, y: 1 }
    6. };
    7. const EPS = 1e-6;
    8. const circleCenter = square.C; // bottom-right corner
    9. const circleRadius = Math.hypot(
    10. square.C.x - square.A.x, square.C.y - square.A.y // diagonal
    11. );
    12. const midX = (square.A.x + square.B.x) / 2; // perpendicular bisector of the bottom edge
    13. function validateGeometry(state) {
    14. const checks = {
    15. hasSquare: state.hasSquare === true,
    16. hasBisector: Math.abs(state.bisector.x - midX) < EPS && state.bisector.vertical === true,
    17. centerIsCorner: dist(state.circle.center, circleCenter) < EPS,
    18. radiusIsDiagonal: Math.abs(state.circle.r - circleRadius) < EPS,
    19. rulerTouchD: dist(state.ruler.anchor, square.D) < EPS,
    20. rulerMark1OnCircle: state.frames.every(f => Math.abs(dist(f.mark1, circleCenter) - circleRadius) < EPS),
    21. rulerMark2OnBisector: state.frames.every(f => Math.abs(f.mark2.x - midX) < EPS),
    22. requiredTrails: state.requiredTrails.every(item => item.necessary === true),
    23. canExportAnimation: typeof state.exportGif === 'function'
    24. };
    25. const passed = Object.values(checks).filter(Boolean).length;
    26. return { checks, score: (passed / Object.keys(checks).length) * 100 };
    27. }

    This set of validations turns “looks complete” into “logically complete.”

    4. Key points for human review

    Human review should focus on whether the original steps are followed; whether extra steps are used to force a result; whether the animation is reasonable; whether trail control is natural; whether the code structure is clear; and whether there are performance or compatibility issues.

    10. Engineering Practice Recommendations

    Scenarios suitable for lightweight models:

    • Content generation with clear rules and decomposable steps;
    • Tasks that are cost-sensitive but require a certain level of reasoning;
    • Front-end-assisted generation, code explanation, and document organization;
    • Rapid iteration that can accept human review;
    • Businesses that can use rule engines or automated tests for validation.

    Scenarios not recommended for direct reliance:

    • High-precision mathematical computation;
    • Strong compliance review;
    • Critical production decisions;
    • Automated execution without human review;
    • Scenarios with extremely low latency requirements that cannot be made asynchronous.

    Data, privacy, copyright, and security review, which is often an overlooked prerequisite for deployment:

    • When code, logs, or user data are involved, data residency and de-identification requirements must be evaluated;
    • Generated code may contain snippets subject to licensing constraints, so license scanning and copyright review are required;
    • Front-end artifacts must undergo XSS/injection and dependency-source review;
    • Before external delivery, confirm that there are no hard-coded keys or internal information leaks.

    Compliance risk should not be borne by the model alone.

    Use prompts to improve stability. The prompt should explicitly state the following:

    • List all input conditions;
    • Explicitly prohibit extra steps;
    • Require step-by-step output of construction logic and key geometric relationship explanations;
    • Require generation of runnable code;
    • Require built-in self-checks;
    • Require a failure degradation strategy.
    1. Before generating output, check the following:
    2. 1. Whether the construction steps specified by the task are used strictly;
    3. 2. Whether nonexistent auxiliary conditions are added;
    4. 3. Whether trail-control requirements are missed;
    5. 4. Whether animation export is implemented;
    6. 5. If there are uncertain steps, state the reason and provide a verifiable alternative.

    11. Conclusion

    This evaluation of a complex geometry interaction task shows that lightweight models are not only suitable for simple question answering or low-complexity generation. In tasks with multiple constraints, multiple steps, state maintenance, and visualization requirements, under this case and the given prompt, the lightweight model in the experimental group demonstrated a certain degree of process adherence. Its value is not only cost control but also constraint decomposition, geometric relationship maintenance, and engineering implementation completeness.

    At the same time, the evaluation reveals shortcomings in output speed, long-reasoning stability, and real-time interaction experience. Whether a lightweight model is usable should not be judged only by whether it can complete a demonstration once. It should be judged by verifiability, maintainability, and controllability in real business scenarios. Only after expanding sample size, adding control groups and automated validation, and repeating validation across multiple prompts and environments can these conclusions be used for formal selection.

    A more reasonable path is: lightweight models + scientific task grading + automated acceptance + prompt engineering + necessary engineering compensation. Only when the process is correct, results are verifiable, and risks are controllable can lightweight models truly move from “demonstrable” to “production-ready.”