Evaluating Complex Reasoning in Large Language Models with Geometric Construction Tasks
Once large language models perform similarly on routine Q&A, summarization, and simple code completion, these low-bar tasks can no longer reliably separate models by their true capabilities. This article explains why complex geometric construction and animation tasks make strong evaluation probes: their mathematical irreducibility (for example, a regular heptagon cannot be constructed with compass and straightedge alone), their simultaneous demand for math reasoning, geometric construction, instruction following, and code generation, and their sensitivity to subtle constraints. It outlines how to design evaluation prompts, a multi-dimensional scoring framework, common failure modes to watch for, and practical recommendations such as control groups, mixed task sets, and combined human-plus-automated verification.
Introduction: Why Basic Evaluations Are Falling Short
When large language models start performing similarly on routine Q&A, summarization, and simple code completion, these low-bar use cases can no longer effectively distinguish the real capability differences between models. When choosing a model in practice, developers increasingly need a stress test that can genuinely “stall” a model—a composite task that looks simple and logically clear but demands mathematical reasoning, geometric construction, instruction following, and code generation all at once. Complex geometric construction and animation generation are exactly this kind of ideal evaluation probe.
Why Complex Construction Tasks Make High-Quality Evaluation Cases
The “Mathematical Irreducibility” of the Task
Take a regular heptagon as an example: it is not an arbitrary drawing exercise. From a pure mathematics standpoint, a regular heptagon cannot be constructed with compass and straightedge alone—its vertex coordinates involve the 7th roots of unity, and because 7 is not a Fermat prime, they cannot be expressed through a finite number of additions, subtractions, multiplications, divisions, and square roots. This means the model must step outside the conventional framework of compass-and-straightedge construction and rely on stronger tools, such as the marked ruler (also called neusis construction, a ruler with graduations), to complete it.
This “mathematical irreducibility” is precisely where the evaluation value lies: it requires the model not only to recite construction steps, but to understand the geometric constraints behind them, thereby avoiding superficial “copy-the-cat-and-draw-the-tiger” imitation.
Animation Generation as a Comprehensive Capability Amplifier
When the final deliverable is required to be an interactive HTML animation, several capabilities are engaged at once: geometric computation, state management, DOM or Canvas drawing, animation timing control, and precise adherence to detail constraints such as “leave construction traces only where necessary.” If any one link is weak, the final result will deviate from the requirements.
Key Points for Designing Evaluation Tasks
A high-quality evaluation prompt should include the following elements:
- A clear target deliverable (e.g., “output runnable HTML and support exporting to GIF”);
- Clear and verifiable step-by-step instructions (the geometric objects, centers, radii, and intersection relationships of each step);
- Clear visualization constraints (e.g., “construction traces appear only where necessary”);
- A clear closing action (completing omitted steps, adding an export feature).
An illustrative constraint snippet (described in natural language, to be turned into runnable code by the model):
Draw an HTML page that produces an animation of the neusis (marked-ruler) construction of a regular heptagon:1. Take a square and construct the perpendicular bisector of its base edge;2. With the bottom-right vertex E as center and the diagonal as radius, draw circle E;3. Keep the marked ruler flush against the bottom-left vertex D, moving the first graduation along circle Ewhile the second graduation passes through the perpendicular bisector of the base;4. Complete the remaining standard construction steps to locate the center and each vertex;5. In the animation, construction traces appear only where necessary;6. Add an "Export as GIF" feature.
A Multi-Dimensional Evaluation Framework
Evaluation should not look only at “can it draw the figure in the end,” but should be broken down into several independently observable dimensions:
Instruction Following: Whether the model’s implementation strictly matches the method described in the prompt, rather than “fudging” a result through another path. This is the key to distinguishing “genuine understanding” from “a lucky guess.”
Mathematical correctness: Whether the generated vertices satisfy the angle and side-length relationships of a regular heptagon, and whether intersections fall at their theoretical positions. Coordinates can be numerically validated with a script:
// Sample: numerical validation of the model-generated regular heptagon verticesconst N = 7, R = 1; // circumradiusconst expected = Array.from({ length: N }, (_, i) => {const angle = (2 * Math.PI * i) / N;return { x: R * Math.cos(angle), y: R * Math.sin(angle) };});function maxError(points) {return Math.max(...points.map((p, i) =>Math.hypot(p.x - expected[i].x, p.y - expected[i].y)));}
Stability of the Reasoning Process: Observe whether the thinking process is coherent. Some models exhibit “intermittent reasoning” or “thinking without outputting”—spending a long time computing without producing a result.
Output Usability: Whether the code runs on the first try, whether the animation is smooth, and whether the export feature actually works.
Latency and Cost: The time taken to generate a complete, complex deliverable. Weak spots often show up as “slow and unusable,” while strong ones manage to be both “complete and relatively efficient.”
Analysis of Common Failure Modes
When actually testing these tasks, be alert to several typical failures:
- Looks right, but isn’t (form without substance): The final image may seem to meet the “regular heptagon” requirement, yet the implementation method has nothing to do with the prompt. These “seemingly correct” results are the most deceptive and must be reviewed at the implementation level.
- Thinking without outputting: The model gets stuck in lengthy computation during reasoning, even producing no output for a long time, so the result arrives late or is missing.
- Missed constraints: Ignoring minor but explicit constraints such as “traces only where necessary” or “export as GIF.”
- Sluggish pace: Even when the result is correct, a lengthy generation process significantly hurts engineering usability and cost.
Practical Recommendations for Evaluation
To make the evaluation more rigorous and reproducible, consider the following:
- Set up control groups: Evaluate multiple models of different capability tiers at the same time, amplifying differences through side-by-side comparison. A single model often looks “unremarkable” on its own; only with a control group can you tell high from low.
- Combine multiple tasks: Beyond geometric construction, pair in high-difficulty prompts from different domains to avoid the randomness of a single task.
- Combine human and automated verification: Use scripts to automatically check numerical correctness, and have humans review instruction following and visualization reasonableness.
- Focus on the “process,” not just the “result”: Record the reasoning process, elapsed time, and failure points to build a traceable evaluation record.
Conclusion
Complex construction and animation generation tasks are becoming a high-quality yardstick for measuring the true level of large language models, because of their combined demands on mathematical reasoning, geometric construction, code ability, and instruction following. Rather than relying on basic use cases that increasingly “can’t tell the difference,” it is better to build an evaluation system that includes control groups, multi-dimensional scoring, and dual verification—one that both helps teams make more accurate model-selection decisions and, in turn, deepens understanding of the boundaries of model capability.
