Evaluating LLM Complex Reasoning: A Regular Heptagon Construction Visualization Analysis

    As benchmark scores saturate, conventional tests struggle to reveal real engineering capability. This article examines why visualization-based tasks make stronger evaluation benchmarks for complex reasoning in large language models (LLMs), using the construction of a regular heptagon via neusis (marked-ruler) geometry as a case study. It shows that judging the reasoning process—not just the final output—is essential to expose models that fake success by inventing steps, compares flagship and lightweight flagship performance, and proposes a reusable four-part evaluation methodology built on task design, process-and-result validation, multi-dimensional metrics, and reproducibility. The code, comments, and technical steps are preserved from the original.

    1. After Benchmark Saturation, What Should We Measure?

    Over the past few years, benchmark leaderboards have proliferated, but they face an awkward reality: once models consistently score near the ceiling on standard problems, the gaps between those scores no longer reflect real engineering capability. The scenarios developers actually care about—multi-step constraint solving, strictly following long instructions, and performing spatial reasoning while generating code—are precisely the ones that traditional multiple-choice benchmarks fail to cover.

    As a result, the focus of evaluation is shifting from “testing memory and common sense” to “testing process and instruction-following.” We need a task type that can be objectively verified while remaining highly sensitive to detail. Visualization generation tasks satisfy both conditions.

    2. Why Visualization Tasks Make Excellent Benchmarks

    A self-contained web page or canvas task typically has the following qualities:

    • Objectively verifiable results: You simply open it in a browser, and whether the shapes are correct and the animation runs is immediately obvious—no subjective scoring required.
    • High sensitivity to instruction-following: The prompt asks you to “follow the specified steps,” “fill in the omitted parts,” and “leave only the necessary construction marks.” Any shortcut or improvisation gets exposed.
    • Composite capability dimensions: A single task simultaneously tests geometric reasoning, spatial imagination, constraint solving, front-end coding, and animation export. A weakness in any one link means total failure.

    Tasks like this are no longer “a pelican riding a bicycle”-level toys; they are a genuine threshold that meaningfully separates models.

    3. Regular Heptagon Construction: An Underrated Geometry Problem

    The regular heptagon was chosen because it hides a mathematical trap: a regular heptagon cannot be constructed with only a compass and an unmarked straightedge (because 7 is not a Fermat prime). Completing it requires a “neusis” approach—a straightedge with marked graduations.

    The prompt demands exactly this harder path: draw a square, construct the perpendicular bisector of the base, draw a circle centered at the bottom-right corner with the diagonal as radius, then slide the marked ruler along the bottom-left vertex so that the first graduation line moves along the circle and the second graduation line passes exactly through the perpendicular bisector. Finally, complete all the standard perpendicular bisectors, use compass constructions to locate the center, and export a GIF.

    This description poses a dual challenge to the model: it must genuinely understand the geometric meaning of the neusis constraint and solve it correctly, while also strictly advancing along the construction direction given in the prompt, without inventing nonexistent steps to “force” a result.

    1. // Illustrative pseudocode: constructing a regular heptagon with a marked ruler (neusis)
    2. // Step 1: Draw the square and the perpendicular bisector of the base
    3. // Step 2: Draw circle E centered at the bottom-right corner E with the diagonal as radius
    4. // Step 3: Slide the marked ruler along the bottom-left corner D so that the first
    5. // graduation line lands on circle E and the second graduation line
    6. // passes exactly through the perpendicular bisector of the base
    7. // Step 4: Solve for the angle theta from the constraints and derive the key directions
    8. // Step 5: Complete the standard perpendicular bisectors and compass constructions
    9. // to determine the remaining vertices
    10. // Step 6: Render the trajectory frame sequence and encode/export it as a GIF
    11. function neusisHeptagon(canvas) {
    12. const D = { x: 0, y: 0 }; // bottom-left corner
    13. // ... compute E, circle E, and the ruler's sliding constraints
    14. // Core: the ruler must simultaneously satisfy "hug D" and "each of the two
    15. // graduation lines passes through its target"
    16. const theta = solveNeusisConstraint(D, /* circle E, perpendicular bisector */);
    17. const vertices = deriveVertices(theta); // derive vertices from theta
    18. return renderFramesAndExportGif(vertices);
    19. }

    4. Key Observation: The Process Reveals More Than the Result

    The most valuable finding in an evaluation is often not “who drew a regular heptagon,” but “who drew it the correct way.”

    One typical phenomenon appeared in practice: after running for roughly twenty minutes, some of the stronger flagship models produced a page that appeared to meet the requirements—the figure looked plausible on the surface. But on close inspection, its implementation path did not match the prompt at all; it introduced extra steps that do not exist in the problem and forced out the final shape. This is the most insidious trap in evaluation: a correct result produced by a misguided process.

    By contrast, a capable, efficient model strictly followed the given construction logic, completing the omitted construction steps one by one. This shows that “being able to generate a figure that looks correct” and “being able to reason correctly under constraints and generate accordingly” are two different levels of capability—and the latter is the real threshold for complex engineering tasks.

    Observing the reasoning trace is also instructive: some models’ thought output is not smooth and continuous, but advances in an intermittent way, even idling in a “think without producing” state; meanwhile, generation time stretches noticeably to complete the complex reasoning. This suggests that evaluation cannot look only at the final output, but must also consider the combined performance of reasoning stability, instruction adherence, and end-to-end latency.

    5. The Rise of Lightweight Flagship Models and the Tradeoffs

    This comparison also reveals an important trend: on reasoning tasks of this difficulty, lightweight flagship models already approach—and in some scenarios rival—traditional flagship models. This matters greatly for cost-sensitive production environments, because it means developers have the opportunity to handle fairly complex reasoning and code-generation workloads at a lower per-call cost and with better concurrency.

    Tradeoffs still exist, of course: complex tasks often require longer reasoning time and higher latency. How to balance capability, latency, and cost needs to be assessed against the specifics of each business.

    6. A Reusable Model Evaluation Methodology

    Drawing on the observations above, here is a methodology that teams can put into practice internally:

    1. Task design: Prioritize composite tasks whose results can be objectively verified and that are highly sensitive to instruction-following (such as visualization plus geometric reasoning), and avoid saturated toy problems.
    2. Balance process and result: Don’t only judge whether the final output is correct; also verify whether the implementation path strictly conforms to the constraints, and watch for pseudo-successes that “force a result.”
    3. Multi-dimensional metrics: Record capability, stability (whether the model idles midway), latency, and cost together to form a composite profile rather than a single score.
    4. Reproducibility and regression: Codify evaluation tasks as fixed test cases, run regression comparisons when models are iterated or switched, and build a sustainable capability baseline.

    Only when evaluation shifts from “who scores higher” to “who better understands constraints and can deliver in practice” does model selection carry real engineering value. For most teams, a well-designed set of verifiable tasks combined with continuous observation of the reasoning process is far more meaningful than chasing leaderboard rankings.