Evaluating LLM Geometry Reasoning for Regular Heptagon Neusis: Model Selection Criteria
This article presents a process-first evaluation method for LLM geometry reasoning on regular heptagon construction with a neusis ruler. Instead of judging whether a screenshot merely resembles a heptagon, it checks whether the model can connect the pivot, circle, straight line, marked-point distance, and vertex closure into a verifiable construction chain. The deliverable includes reproducible coordinates, the definition of the d parameter, failure modes, process correctness, an evaluation checklist, and a model-selection decision table that evaluators can reuse directly.
Evaluating LLM Geometry Reasoning for Regular Heptagon Neusis: Model Selection Criteria
This article concludes that evaluating regular heptagon construction with a neusis ruler should not focus only on whether a screenshot “looks like a heptagon.” Instead, the evaluation should determine whether a model can connect the pivot, circle, straight line, marked-point distance, and vertex closure into a reproducible and reviewable construction chain.
The deliverables of this article are reproducible coordinates, the definition of the d parameter, common failure modes, process correctness, a reusable checklist, and a model-selection decision table. These are intended to help evaluators reuse the method directly.
1. What the Evaluation Task Is: Inputs and Expected Constraints
As tasks that ask a model to “generate an HTML web page” become increasingly saturated, longer-tail questions are emerging in complex geometry evaluation. These include whether a regular heptagon is constructible with straightedge and compass, neusis construction, and how to select models for large-model geometry reasoning.
This article does not focus on whether an output “looks similar” to a heptagon. Instead, it provides a process-based evaluation method with reproducible results. The evaluation input can be parameterized as follows.
Take a square ABCD with side length s = 2, where:
- A(0,0)
- B(2,0)
- C(2,2)
- D(0,2)
The perpendicular bisector of the bottom side AB is line L: x = 1.
Use the bottom-right corner B(2,0) as the center of circle O. Let the radius be equal to the diagonal of the square, so:
- r = 2√2
- Equation of circle O: (x - 2)^2 + y^2 = 8
One end of the neusis ruler is hinged at the bottom-left corner A(0,0). It may only rotate around A and may not translate.
Marked point P slides along circle O. Marked point Q lies on the perpendicular bisector L. The body of the ruler satisfies the condition that A, P, and Q are collinear.
The expected output is not a static image. It is an HTML page that can step by step show:
- the perpendicular bisector;
- circle O;
- the sliding neusis ruler;
- the positions of the marked points;
- key intersections;
- the vertices of the regular heptagon;
- the center of the heptagon;
- connections between the center and vertices.
If any ruler-and-compass steps are omitted, they should be completed. Compass traces should retain only necessary evidence. The page should also support GIF export, and the animation should be playable without missing key steps.
2. How the Neusis Device Maps to the Vertices of a Regular Heptagon
To separately check whether the “device is valid” and whether the “heptagon is correct,” this article defines the neusis device as a parameterized angle locker.
It does not require the model to replicate one unique historical construction. However, it does require the model to explain how P, Q, A, and B in the device map to the vertices of the regular heptagon, and to verify the mapping using coordinates or formulas.
The evaluation adopts the following reference construction chain.
Step 1: Determine the first side
Let the first vertex of the regular heptagon be:
- V0 = A(0,0)
Let the second vertex be:
- V1 = B(2,0)
Thus, the first side with length 2 is determined by the bottom side AB of the square.
Step 2: Determine the angular reference
Let the reference angle be:
- θ = π/7 ≈ 25.714°
The angle between the neusis line AQ and the positive x-axis is θ. Since Q lies on L: x = 1, Q can be written as:
- Q = (1, tan θ)
Let:
- m = tan θ
Step 3: Find the intersection point P between circle O and ray AP
The line equation is:
- y = mx
Substitute this into circle O:
(x - 2)^2 + (mx)^2 = 8
After simplification:
(1 + m^2)x^2 - 4x - 4 = 0
For the positive intersection branch used in this reference construction:
x_P = 2(1 + sqrt(2 + m^2)) / (1 + m^2)
y_P = m x_P
Step 4: Define the marked distance d between the two marked points
Because P and Q lie on the same ray passing through A, and because in the reference branch P lies outside Q, we have:
d = |PQ| = (x_P - 1) * sqrt(1 + m^2)
After simplification:
d(θ) = [1 + 2 * sqrt(2 + tan^2 θ) - tan^2 θ] / sqrt(1 + tan^2 θ)
When θ = π/7, the reference value is:
- d_ref ≈ 3.384
This value can be used as the default evaluation criterion when the problem statement does not provide d. If the problem statement explicitly gives d, the provided d should be used first, and the evaluator should check whether |PQ| = d.
Step 5: Generate the regular heptagon from θ
The exterior angle and central angle of a regular heptagon are:
- Δ = 2π/7 = 2θ
Starting with the edge vector from V0 to V1, each edge is rotated by Δ.
The vertex coordinates are:
Vk = (2 * Σ{j=0}^{k-1} cos(jΔ), 2 * Σ_{j=0}^{k-1} sin(jΔ)), for k = 0, 1, …, 7
In this formulation:
- V0 = (0,0)
- V1 = (2,0)
- V2 through V6 are the remaining vertices
- V7 should return to V0
If a downward-mirrored construction is used, it should still satisfy the same side length, angle, and closure conditions. The direction used should also be noted in the evaluation record.
Equivalent center coordinates can be written as:
- Center C = (1, cot(π/7))
- Circumradius R = csc(π/7)
A model may generate vertices by rotating around C by 2π/7, or by accumulating edge vectors. The evaluation does not force one implementation over another, but the result must be verifiable from A, B, θ, or d, and must produce seven vertices and seven edges.
3. Why a Regular Heptagon Requires Neusis
According to the Gauss–Wantzel theorem, a regular n-gon is constructible with straightedge and compass if and only if n is the product of a power of 2 and some distinct Fermat primes.
The known Fermat primes are:
- 3
- 5
- 17
- 257
- 65537
The number 7 is neither a power of 2 nor a Fermat prime. Therefore, a regular heptagon is not constructible with straightedge and compass alone.
The value of the neusis method is that it allows sliding contact. With the pivot fixed and the marked points constrained, the ruler can be continuously adjusted to find a position that satisfies the required geometric conditions.
If a model merely draws a heptagon with angles close to 360°/7, or bypasses the sliding constraint with approximate formulas, it has not completed the real difficulty of the task.
Therefore, evaluation should prioritize checking whether:
- P and Q are genuinely constrained by circle O and line L;
- the vertices derived from d or θ close correctly.
It is also necessary to add one point: straightedge-and-compass impossibility does not mean that every arbitrary neusis construction is valid. This evaluation requires the construction to be verifiable by coordinates. A response should not pass merely because it “appears to use a neusis ruler.”
4. Where Models Most Often Fail in the Evaluation
In complex geometry tasks, failure often does not occur in the final graphic. It occurs in intermediate constraints and process management.
The following table lists common failure types, detection signals, and severity.
| Failure type | Detection signal | Severity |
|---|---|---|
| Self-created steps | The output contains auxiliary lines, extra marks, extra loops, or extra objects that were not required by the prompt and whose necessity cannot be explained | High |
| Angle fudging | The angle is close to some fixed value, but it is not derived from the neusis constraint or from d | Blocking |
| Animation step skipping | Key intersections or line segments appear instantly without a visible construction process | High |
| Trace management failure | Compass traces are excessively retained, or necessary traces are missing, making review impossible | Medium |
| Export failure | The GIF is not generated, the file is corrupted, key steps are missing, or the file cannot be played | Blocking |
| Think-only without output | Reasoning content exists, but there is no runnable HTML, SVG, Canvas, or code output | Blocking |
Among these, “angle fudging” and “export failure” should be prioritized as blocking failures.
The former causes the evaluation target to drift away from the intended construction problem. The latter makes it impossible to verify engineering usability.
5. Think-Only Without Output: A Field Template, Not a Conclusion
In complex tasks, some models may reason for a long time but fail to produce runnable code.
The following is a synthetic data field template. It is used only to explain the structure of evaluation fields. It does not represent real measurement data and does not point to any specific vendor.
Production model selection should be based on real self-test data.
| Model | Version | Runs | Temperature | Completion rate | Process correctness | Key constraint check | Notes |
|---|---|---|---|---|---|---|---|
| Synthetic example A | Mock field | 3 | 0.2 | 2/3 | 67% | Closure error exceeds tolerance | Only demonstrates field structure |
| Synthetic example B | Mock field | 3 | 0.2 | 3/3 | 90% | Pivot, marked-point placement, and GIF pass | Only demonstrates field structure |
Process correctness is defined as the number of steps that pass step-by-step review divided by the total number of steps.
For example, if there are 10 total steps and 8 steps pass, the process correctness is 80%.
It is recommended to run the same prompt multiple times and record:
- first-token latency;
- total elapsed time;
- completion rate;
- process correctness;
- key-constraint pass rate.
6. A Reusable Evaluation Checklist
Evaluators can map each requirement in the prompt to code, animation behavior, or page behavior.
The following checklist items can be scripted or reviewed semi-automatically.
| Check item | Script approach or judgment method | Pass criterion | ||
|---|---|---|---|---|
| Runtime environment | Check browser compatibility, SVG or Canvas implementation, scaling behavior, and error logs. For GIF output, check size, frame rate, and frame order | Runs in mainstream browsers without critical errors; GIF is playable and key steps are not missing | ||
| Pivot constraint | Parse SVG or Canvas coordinates. Check that A(0,0) is fixed and that the ruler body can only rotate around A | The pivot does not drift, and the sliding process conforms to the neusis constraint | ||
| Two marked-point placements | Calculate whether P satisfies (x - 2)^2 + y^2 = 8 and whether Q.x = 1. If d is given, calculate the error in | PQ | Error is no more than 1e-3, and the placements satisfy the constraints | |
| Circle center and radius | Check whether the center is B(2,0) and whether the squared radius is 8 | The center and radius settings are consistent | ||
| Arcs and intersections | Check whether key arcs and intersections are derived by solving line-circle equations. Calculate relevant distances or angles | No intersections or shortcut lines appear without derivation | ||
| Regular heptagon closure | Check the seven vertices, seven edges, side-length consistency, central angle approximately equal to 360°/7, and closure error | Side length is 2 ± 1e-3; central angle error is less than 0.01°; closure error is no more than 1e-4 | ||
| Instruction-by-instruction check | Map each prompt requirement to code, animation, or page behavior | Every instruction is locatable and verifiable | ||
| GIF export | Actually click the export function. Check file size, frame count, frame order, and playability | GIF is playable and key steps are complete. It is recommended to use no fewer than 24 frames, with a frame rate between 8 and 15 |
For the regular heptagon closure check, three points deserve special attention.
First, the seven vertices from V0 to V6 should be unique and non-duplicated.
Second, the adjacent angle between vertices viewed from the center C should be 360°/7.
Third, V7 should return to V0. If there is an obvious opening or overlap, the construction should be judged incomplete.
7. How to Choose a Model: Balancing Capability and Speed
Complex reasoning tasks are often slower, but they may achieve higher process correctness.
Model selection should start by defining the scenario:
- for real-time interaction, focus on first-token latency, total time, and stability;
- for offline batch processing, focus on process correctness and completeness;
- for production high-risk scenarios, focus on whether key constraints have zero tolerance.
The thresholds below are example thresholds, not universal standards. In actual production, they should be calibrated according to business SLA, input length, concurrency, model version, and data distribution.
First-token latency should be measured using P95 or P99 statistics rather than a single best-case value.
| Scenario | Priority metrics | Example threshold | Description |
|---|---|---|---|
| Real-time interaction | First-token latency, total time, stability | First-token latency no more than 2 seconds; total time does not noticeably block interaction; completion rate no less than 90%; key-constraint process correctness no less than 80% | If not satisfied, consider switching to offline generation or adding timeout policies |
| Offline batch processing | Process correctness, code runnability | Process correctness no less than 90%; regular heptagon closure check passed; longer processing time is tolerable | Suitable for complex animations, multi-step geometry, and code generation |
| Production high-risk | Key constraints, reviewability | 100% pass on key check items; manual review when necessary | Speed cannot replace correctness |
If a lightweight model is close to or better than larger models on key constraints and process correctness, it can become a more cost-effective choice. However, this is only valid if hard constraints are not compromised, including the pivot, marked-point placements, circle, arcs, intersections, closure, and GIF export.
8. Example of Vendor and Platform Capabilities
Vendor and platform capabilities can be used as conditions for executing the evaluation, but they should not be presented as rankings or promises.
For example, the Baidu AI Cloud Qianfan model platform may provide capabilities related to model invocation, model services, and evaluation support. It can be used in scenarios such as unified evaluation and multi-model comparison.
This article uses it only as a capability example. It does not constitute a model-selection commitment, performance guarantee, or ranking conclusion. Specific capabilities and availability should be based on official documentation and actual self-testing.
9. Summary
Regular heptagon construction with a neusis ruler appears to be a geometry problem. In practice, it is a comprehensive benchmark of mathematical understanding, multi-step instruction following, geometric precision, and visualization engineering capability.
Evaluators should insist on a process-first approach, resist superficial similarity, and use control groups. Parameters, coordinates, intersections, the definition of d, frame order, and export results should all be included in the review.
The real comparison of model strength is not whether a model can draw a figure that looks like a heptagon. It is whether the model can satisfy reproducible constraints, present a complete construction process, and clearly explain failures when they occur.
