Evaluating Complex Reasoning in LLMs: The Neusis Construction of a Regular Heptagon as a Benchmark

    Toy-level benchmarks can no longer tell capable LLMs apart, and they often hide logic errors and instruction drift. This article argues that benchmarks built on tasks that look simple but are hard to implement, such as geometric construction tasks that demand a long, verifiable chain of reasoning, expose the real gaps between models. Using an HTML animation of a neusis construction of a regular heptagon as a case study, it shows which reasoning and coding skills the task puts to the test. It also outlines the dimensions to score and the patterns observed in practice, such as "passing" that is not actually correct and lightweight models that approach flagship-level rigor. Finally, it gives guidance for building an evaluation system centered on failure modes.}

    1. Why “Toy-Level” Benchmarks Are Losing Their Edge

    Early LLM evaluations leaned on a few classic small tasks: writing a sorting routine, explaining a concept, or generating an SVG of “a pelican riding a bicycle.” These tests were short, semantically clear, and had correct answers that were easy to spot. They worked well for a while.

    As model capabilities have improved overall, however, these tasks can no longer tell models apart. When nearly every model passes them easily, the tests lose their power to discriminate. You can no longer tell whether model A is genuinely stronger or just happened to be luckier. There is a bigger problem, too. Many outputs that “look like they run” actually hide logical flaws or drift from the instructions, and toy-level tests are too forgiving to catch them.

    A valuable benchmark therefore needs tasks that look easy but are in fact hard. The instructions are clear, but a correct implementation takes several steps of rigorous reasoning. Any step that goes off track corrupts the result. Only tasks like these can reveal how far apart a model’s real ceiling and floor are.

    2. What Makes a Hard Task Reveal Real Model Capability

    An ideal benchmark task meets three conditions at once:

    1. The instructions can be judged objectively. The input is a natural-language description, and a human can quickly check whether the output matches it exactly.
    2. The reasoning chain is long. The model has to go through the whole pipeline from understanding and modeling, to breaking the problem down, to implementing the solution. A mistake at any stage will surface.
    3. There is one correct answer or strong constraints. The model cannot improvise its way to a pass.

    Geometric construction fits these conditions naturally, especially when a neusis construction (also called a marked-ruler construction) is combined with standard straightedge-and-compass steps. The math behind this comes from a well-known result. A regular heptagon cannot be drawn with only a compass and an unmarked straightedge. A regular n-gon is constructible with those tools if and only if n is a power of 2 or a product of a power of 2 with distinct Fermat primes, and 7 meets neither case. The construction therefore has to use a marked straightedge with a fixed length between two marks. The ruler is slid to a position the compass and straightedge cannot reach, with each mark landing on a given locus.

    Tasks like this are demanding because a model must do more than understand the mathematics. It has to translate abstract construction steps into runnable animation code, while following the constraints to the letter.

    3. Case Study: An HTML Animation of the Neusis Construction of a Regular Heptagon

    The evaluation task is stated as the following natural-language instruction (paraphrased):

    Build an HTML page that performs a neusis construction of a regular heptagon. Start from a square. Construct the perpendicular bisector of the base, then draw circle E centered at the bottom-right vertex with the diagonal as its radius. Keep the marked ruler flush against the bottom-left vertex D and slide it so that its first mark moves along circle E until its second mark passes through the perpendicular bisector of the base. Then complete the remaining standard straightedge-and-compass steps to locate the center. Fill in all the steps the description leaves out. Compass constructions should retain their necessary motion trails. Finally, add an option to export the animation as a GIF.

    This task looks simple but is hard because it tests four capabilities at once:

    • Mathematical correctness. Sliding the ruler amounts to solving a constraint: the first mark must land on the circle and the second mark must land on the perpendicular bisector. The model has to understand the principle behind a neusis construction, not just replay steps mechanically.
    • Mapping geometry onto code. Abstract geometric relations have to become coordinate calculations and path drawing in SVG or Canvas.
    • Instruction adherence. The instructions explicitly say to keep only the necessary trail marks, which demands close attention to detailed constraints.
    • Engineering completeness. GIF export adds frame-by-frame rendering and encoding.

    The sketch below outlines the constraint solver a model needs to come up with. It makes the core logic easier to follow.

    1. // Sketch: a neusis constraint solver framework (not a product implementation)
    2. // Goal: first mark p1 lies on circle E; second mark p2, extended along the ruler,
    3. // lies on the perpendicular bisector of the base
    4. function solveNeusis(D, circleE, midline) {
    5. let lo = 0, hi = 2 * Math.PI;
    6. for (let i = 0; i < 60; i++) {
    7. const mid = (lo + hi) / 2;
    8. const p1 = pointOnCircle(circleE, mid); // First mark on the circle
    9. const p2 = extendAlongRuler(p1, D); // Extend along the ruler held against D
    10. // Check which side of the perpendicular bisector the second mark falls on, then bisect toward the constraint
    11. if (signedDistanceToLine(p2, midline) > 0) hi = mid;
    12. else lo = mid;
    13. }
    14. return finalizeRulerPosition(lo);
    15. }

    The key point is this: the model needs to realize that the neusis step cannot be replaced by a standard compass-and-straightedge move, and to build the constraint solver above itself. It cannot quietly skip the step or fake a result that only looks right.

    4. Core Evaluation Dimensions

    Using this case study as a starting point, judging whether a model truly understands should cover at least the following dimensions:

    • Instruction adherence. Does the model follow the instructions exactly, or add steps that were never described in order to reach an answer? This is where things most often go wrong. Even capable models may take shortcuts around the prescribed steps just to end up with the right final figure.
    • Reasoning-chain completeness. Are the intermediate steps self-consistent? Can the model correctly carry out unusual operations such as the neusis sliding step?
    • Code quality and runnability. Does the generated animation actually run? Are the coordinate calculations correct and the trails drawn as required?
    • Response characteristics. Does the reasoning flow continuously, or does the model spend long stretches reasoning without producing any output? What are the overall latency and throughput like?
    • Adherence to boundary constraints. Can the model follow easily overlooked details, such as keeping only the necessary trail marks?

    5. Patterns Observed in Testing

    Hands-on tests surface several recurring patterns. Together, they form the criteria for judging results.

    1. “Passing” does not mean “correct.” After spending more than ten minutes on the task, some models produced a page with a regular heptagon that seemed to meet the requirements. A closer look, though, showed that the construction steps did not match the instructions at all. The model added steps that were never described and forced out the final figure. Results like this, which look complete but actually drift from the brief, are exactly what hard tasks are good at exposing. They reveal instruction-following weaknesses.
    2. Lightweight models can come close to flagship level. Some lightweight, low-cost models showed reasoning rigor in these tasks that was close to, or even matched, that of much larger flagship models. Their overall performance does not feel like it comes from a small model. This suggests that parameter count is no longer the only factor that determines capability. Training for reasoning and data quality matter just as much.
    3. Speed is an acceptable trade-off. Lightweight models most often struggle with slow output and reasoning that feels choppy and keeps stalling. In complex reasoning scenarios where capability comes first, however, this latency is usually a price worth paying.
    4. “Thinking without outputting” is a warning sign. Some models stay in the reasoning phase for a long time without producing any visible output. They may eventually return a correct answer, but they make the interaction worse and harder to monitor. The engineering side therefore needs timeouts and progress feedback.

    6. Building a Reliable Evaluation System

    Based on the analysis above, developers can build an evaluation system around the following practices:

    • Build a “deep-water” task set. Choose tasks that are clearly specified but hard to implement, such as complex geometric constructions, multi-constraint optimization, or long-chain mathematical proofs. These tasks make it easier to tell models apart.
    • Define checkable criteria. For each task, decide in advance what the correct approach looks like and which deviations come up most often. Then check both the process and the final result, using human reviewers, scripts, or both, instead of judging only the output.
    • Score across multiple dimensions. Record correctness, instruction adherence, latency, and stability separately, then combine them into a capability profile for each model.
    • Pay attention to failure modes. More important than whether a model gets the right answer is where it goes wrong. Does the reasoning chain break? Does it drift from the instructions? Does it overlook an engineering detail? These failure patterns usually carry more information than success cases.
    • Set up supporting engineering. If batch evaluations run through an AI coding tool or a cloud platform, general-purpose capabilities such as logging, monitoring and alerting, and containerized scheduling can help with task orchestration and result archiving. That makes evaluations more reproducible and easier to run at scale.

    7. Conclusion

    Testing a model with a hard task such as the neusis construction of a regular heptagon is really a test of one thing. Can the model turn clear natural-language instructions into correct multi-step reasoning and runnable code? Toy-level tests can no longer do this job. Complex reasoning tasks are built to look simple but be hard, which lets them cover instruction adherence, mathematical reasoning, code quality, and response characteristics all at once.

    For developers, an evaluation system built around failure modes and scored across multiple dimensions offers more lasting value than chasing a single leaderboard score. The real gaps between model capabilities are usually hidden in these deceptively simple deep-water tasks.