Lightweight Flagship LLM Benchmark: Judging Process Quality and Model Selection with a Geometry Construction Task
Choosing a lightweight flagship model by parameters, pricing, or vendor marketing alone is unreliable. This article benchmarks a lightweight flagship model against a high-end flagship model on a geometry construction animation task: building a regular heptagon with a two-mark ruler (neusis) rendered as a runnable HTML animation. The test focuses not on whether the heptagon looks correct, but on whether the model follows the specified construction steps under complex constraints. You will learn why process review matters as much as result review, what a static-scan checklist can and cannot verify, how to design tiered model evaluations, and why a "lightweight" model can still carry high reasoning cost and slow output in deep, multi-step tasks. Includes the revised prompt, a comparison setup, a Python static-check example, and a common-error table.
1. Introduction: How to Choose a Lightweight Flagship Model — It’s Not Just About “Looking Right”
When choosing a lightweight flagship model, many people start with parameters, pricing, or vendor marketing. But in real tasks, whether a model is worth adopting often does not come down to a single metric. It comes down to whether the model can complete a task following the specified steps and deliver reliably under complex constraints.
This is especially true for code generation, geometry animation, and engineering automation. Here, a model must not only “produce a result” but also “complete the task through the specified process.” If a model bypasses the requirements and generates a visually similar result using a simpler approach, it may look acceptable on the surface — yet it has lost process controllability.
This article therefore uses a single geometry construction animation task to observe how lightweight flagship models perform. The focus of the test is not “did it draw a regular heptagon,” but “did it complete the specified construction as required by the prompt.” For readers asking how to choose a lightweight flagship model, this hands-on test helps you build a more reliable judgment method: don’t look only at the final result — examine the process, the cost, and the stability.
2. Test Task Analysis: Why Use a Geometry Construction Animation to Evaluate a Model?
2.1 The Revised Prompt
To avoid naming ambiguity, this test first standardizes vertex naming: the square is ABCD, with C as the bottom-right vertex and D as the bottom-left vertex; the base edge is CD; the diagonal is AC. The revised test prompt is as follows:
Build an HTML web page with an animation that demonstrates constructing a regular heptagon using a “two-mark ruler” (neusis).
- Take square ABCD, with bottom edge CD, and construct the perpendicular bisector of CD;
- With the bottom-right vertex C as the center and diagonal AC as the radius, draw circle C;
- Using the “two-mark ruler,” keep the ruler’s body flush against the bottom-left vertex D, and move the first graduation mark along circle C until the second graduation mark passes through the perpendicular bisector of CD;
- Then perform standard compass-and-straightedge construction to determine elements such as the center of the regular heptagon;
- Complete all omitted compass-and-straightedge steps. When using the compass, leave the point’s moving trace, but keep only the necessary traces;
- Add a feature to export the animation as an animated image (GIF).
Note that the original prompt’s “first graduation mark” lacks fully reproducible parameters. To avoid treating a vague setup as a strict engineering specification, this article treats this part as an “illustration of the intended approach.” If used for a formal evaluation, additional parameters should be supplied: the square’s side length, the coordinate system, the spacing between the two marks on the ruler, how the ruler body stays flush, and which intersection point of the graduation mark and the circle to select. Otherwise, even a model that completes an approximate construction cannot simply be judged as producing a single correct answer.
2.2 Where the Difficulty Lies: Four Challenges Stacked Together
At first glance this looks like a simple geometry construction, but it actually contains four stacked challenges:
- Mathematical barrier: A regular heptagon cannot be easily constructed with an unmarked compass and straightedge alone; it requires a marked-ruler (neusis) construction.
- Step constraints: The model must execute the specified steps and cannot switch to another algorithm for convenience.
- Timing and visualization: The result must be presented as an animation that runs in a browser.
- Detail specification: The animation must retain necessary traces — neither over-marking nor showing no process at all.
The most vulnerable point is the step constraint. A regular heptagon’s vertices can be generated quickly in code using trigonometric functions, so a model can easily “take a shortcut.” Once it bypasses the specified construction, the final result may look correct, but the process no longer matches what the task requires.
2.3 Why It Can Distinguish Model Capability
Common HTML-generation tasks are losing discriminative power, because many of them can be covered by templates. A geometry animation task is hard to solve with a canned approach: it requires a model to simultaneously demonstrate mathematical reasoning, instruction following, code implementation, and detail control.
This type of task has three characteristics:
- Hard to memorize: The problem rarely appears directly in common training data, so a model cannot rely on memory alone.
- Hard to cut corners: The specified construction must genuinely exist in the code; otherwise, it is exposed during manual review or runtime checks.
- Verifiable results: Whether the figure is correct and whether key steps appear can be verified through manual review, static scanning, and browser execution.
So this task is not merely testing “can it draw a regular heptagon,” but whether the model can reliably execute a complete workflow under complex constraints.
3. Hands-On Observations: Differences Between the Test Group and the Control Group
3.1 Test Setup
To keep things as fair as possible, the test group and control group used the same revised prompt and adopted broadly similar acceptance criteria. Note that this was a single-sample observation: it discusses capability tendencies, not a precise ranking.
| Item | Test group: lightweight flagship model | Control group: high-end flagship model |
|---|---|---|
| Model tier | Lightweight flagship positioning | High-end flagship positioning |
| Name and version | Model A; exact version per official documentation. Anonymized here to avoid evaluation bias and for confidentiality | Model B; exact version per official documentation. Anonymized here to avoid evaluation bias and for confidentiality |
| Interaction mode | Runs in a command-line AI coding agent, directly modifying and running project files | Generates a complete web page in a visual chat interface |
| Running tool | CLI AI coding agent | GUI chat tool |
| Runtime | About 34 minutes, including thinking, generation, and delivery phases | About 20 minutes |
| Output form | HTML page, geometry animation, GIF export feature | HTML page, geometry animation |
| Verification method | Manual review of the construction, observing runtime results | Manual review of the construction, observing runtime results |
| Limitations | Single sample, CLI environment; sample not reproducible, conclusions not generalizable | Single sample, GUI environment; sample not reproducible, conclusions not generalizable |
The test recorded three dimensions at the same time:
- Process: Whether the reasoning process was complete and whether any output anomalies appeared.
- Result: Whether the construction was completed as required and whether the page ran correctly.
- Cost: Time spent and resource consumption.
Note in particular that no multiple retries were performed, so failure rates and retry counts were not fully tallied. For a future formal selection process, additional metrics are recommended: time to first token, total time, failure or retry count, and whether manual review is needed.
3.2 Test Group: Strictly Follows the Steps, About 34 Minutes End to End
The test group’s output process showed a distinct “chunk-by-chunk” pattern rather than the usual smooth streaming output. There were phases that only thought without producing output, and the total time from request to final delivery was about 34 minutes.
In terms of results, this time bought relatively strong prompt adherence. After manual review, the test group’s output broadly completed the following flow:
- First use compass and straightedge to construct the square and the perpendicular bisector;
- Then, with the bottom-right vertex C as the center and diagonal AC as the radius, draw the circle;
- Attempt to implement the core neusis constraint: the ruler body stays flush against the specified vertex, the first graduation mark lands on the specified circle, and the second graduation mark passes through the perpendicular bisector;
- On this basis, complete the remaining steps such as determining the center;
- Complete the animation per the “leave traces only where necessary” requirement, and add the GIF export feature.
Of course, since the original prompt lacked some neusis parameters, this evaluation focuses on whether the model attempted to follow the setup’s structure, rather than requiring it to produce a single mathematical answer. The test group’s output showed no clear sign of “drawing a regular heptagon that merely looks the same using a simpler method.”
3.3 Control Group: The Result Looks Fine, but the Process Needs Scrutiny
After about 20 minutes of generation, the control group produced a page that looked acceptable, and a regular heptagon appeared in the animation. But upon closer inspection of the implementation, its construction deviated from the direction described in the prompt and did not match the required steps.
Manual review conclusion: the control group’s output showed no verifiable implementation of circle C, the perpendicular bisector of CD, or the two-mark neusis constraint. Because these key steps were not adequately presented, this article does not judge it as strictly satisfying the prompt’s requirements, but rather states that “the result looks acceptable, but the process was not verified to meet the requirements.”
This task is really testing the process. Looking only at the final frame, “it drew it” seems to pass; once you check the construction, the gap becomes apparent. This also shows that in evaluation, both “process review” and “result review” are indispensable.
Core finding: Looking only at the finished output, the two groups differ little; after checking the construction, the gap is significant.
3.4 Takeaways from the Comparison: A Correct Result Does Not Equal a Correct Process
Placing the two groups side by side yields an important conclusion:
- By the “final output” standard, the two are close.
- By the “implementation process” standard, the gap is significant.
In real engineering, the process determines the controllability and maintainability of the result. If a model autonomously swaps algorithms, it can introduce hard-to-anticipate boundary conditions, numerical errors, and compatibility problems. Evaluating a model by staring only at the final output means giving up the most valuable information.
4. Advantages Observed in This Sample: How Did the Lightweight Flagship Model Perform?
All the advantages below come from this single-sample, CLI-environment observation. Note that the test group ran in a CLI environment while the control group ran in a GUI environment, so the two are not a strict same-condition comparison. These advantages may therefore partly stem from the CLI agent form, and cannot be fully attributed to the model itself.
4.1 Strict Instruction Following: Constraints Over Convenience
In complex multi-step tasks, the most common mistake a model makes is not “can’t do it” but “didn’t do it as required.” In this experiment, the model under test fully executed the specified construction and did not deviate because “there was an easier way.” This “constraints over convenience” behavior pattern is an important prerequisite for an agent to take on engineering tasks.
4.2 Complex Geometry Reasoning: It Can Crack the Hard Problems
The regular heptagon is a typical non-trivial construction that combines compass-and-straightedge work with a marked-ruler construction, clearly harder than common shapes such as triangles, squares, and regular pentagons. The test group could derive and attempt this construction, indicating good performance in spatial geometry and multi-step mathematical reasoning.
4.3 Global Consistency and Detail Control
“Filling in the omitted steps” requires the model to stay logically consistent over a long context. The test group showed no obvious contradictions between steps, duplicated point definitions, or omissions, and handled the animation per the “keep only necessary traces” requirement. This shows it can continuously maintain the overall structure over a longer task.
4.4 Engineering Delivery: Not Just Computing, but Shipping
The final output is a page that runs directly in the browser, including the animation and export feature. In other words, the model not only tried to compute correctly but also completed coding, interaction, and visualization packaging. In this sample, this delivery capability gave it the potential to approach a high-end model on the specified task.
5. Shortcomings and Costs That Must Be Acknowledged
5.1 Output Speed: The Most Notable Shortcoming in This Test
In this single test, the test group took about 34 minutes from request to delivery, versus about 20 minutes for the control group. Besides the slower output speed, no other obvious defects were observed in this sample. But this conclusion applies only to this sample, the CLI environment, a single task, and the ~34-minute total-time measure; it cannot be generalized.
Why would a model marketed on being lightweight and high-throughput show a speed disadvantage on a complex task? A few reasons are worth analyzing:
- Overhead from the reasoning mechanism: Facing a difficult task, the model performs extensive internal reasoning, and this time is not reflected as visible output.
- Deep reasoning conflicts with low latency: If a model invests more compute in thinking, output speed yields to thinking depth.
- Impact on interaction experience: The thinking process produces choppy, chunk-by-chunk output with a long wait for the first token, so users may mistakenly think the task has frozen.
5.2 Reasoning Cost: A Lightweight Positioning Does Not Mean Low Reasoning Overhead
A lightweight model can still incur high reasoning cost on complex tasks. The reason is that the reasoning process in complex tasks brings substantial compute, and a single request’s compute consumption can be far higher than an ordinary Q&A.
So when selecting a model, don’t look only at the word “lightweight” — also pay attention to the actual cost measures:
- Time to first token;
- Total time;
- Failure or retry count;
- Whether manual review is needed;
- Stability under high concurrency.
This article does not cite cluster sizes or deployment details that have not been officially confirmed. If a vendor publicly provides related service information, it should be based on official documentation and actual load-test results.
5.3 Positioning: Better Suited to Deep Tasks Than Quick Q&A
Based on these observations, this type of model is currently better suited to the following scenarios:
- Complex code generation and refactoring;
- Multi-step automation workflows;
- Analysis tasks that require deep reasoning.
For scenarios sensitive to time to first token — such as real-time conversation and high-frequency simple Q&A — you should still evaluate carefully against actual response performance.
6. Evaluation Insights: How to Build a Trustworthy Model Evaluation?
6.1 Design Tasks in Tiers by Capability Dimension
It is recommended to divide evaluation tasks into four levels, raising the difficulty step by step:
- Basic instruction following: Simple multi-step tasks, checking whether the model completes them in order.
- Mathematical and spatial reasoning: Construction tasks with a mathematical barrier, checking whether the model can derive the correct construction.
- Engineering implementation: Whether it can produce runnable, maintainable code and interactions.
- Long-chain consistency: Whether it stays consistent across a long context and multiple steps.
6.2 Balance Result Verification with Process Review
Automated verification can cover part of the result correctness. For example, you can check whether the code contains expressions related to the regular heptagon’s central angle, whether the neusis constraint is present, and whether an export entry point is provided. Here is a static-scan example:
import redef static_check_heptagon_angle_expr(generated_code: str) -> dict:patterns = [r"360\s*/\s*7",r"2\s*\*\s*(?:Math\.PI|pi|PI)\s*/\s*7",r"51\.428571?",r"51\.43"]hits = []for p in patterns:m = re.search(p, generated_code, re.I)if m:hits.append(m.group(0))target_angle = round(360 / 7, 4)return {"angle_expr_hits": hits,"has_heptagon_angle_expr": bool(hits),"target_angle_deg": target_angle,"note": "This function only performs a static scan and cannot replace runtime verification; confirm the construction via manual review or sandbox execution."}
This method only detects whether relevant angle expressions appear in the code; it cannot on its own prove that the model actually completed the specified construction. Whether the angle is truly used in vertex computation and whether the neusis constraint is embodied still require manual review or execution in an isolated environment.
Common errors can be organized into the table below:
| Error type | Symptom | Check method | Pass/fail criterion |
|---|---|---|---|
| Direct trigonometric vertex generation | Uses Math.cos and Math.sin to compute 7 vertices directly, with no specified construction | Check whether direct vertex-generation logic exists in the code | If circle C, the perpendicular bisector, and the neusis constraint are not reflected, fail |
| Keyword false hit | Words like “circle” and “neusis” appear in the code, but there is no real geometric constraint | Search whether the keywords are actually invoked | If the keywords appear but serve no function, fail |
| Uncontrolled animation traces | Leaves many irrelevant traces, or no necessary traces at all | Check the animation-drawing and trace-recording logic | If it does not follow “keep only necessary traces,” revise |
| Fake export entry | A button exists but cannot generate an animated image | Check the export function, file-writing, and download logic | If it cannot actually export, fail |
| Missing step completion | Only draws the final result without showing the intermediate compass-and-straightedge operations | Check whether intermediate points, circles, and lines appear | If key steps are missing, do not consider it complete |
Security note: Code generated by the model should be run in an isolated environment or sandbox; exporting an animated image involves local file writing and downloading, so review the code logic first.
6.3 Run Multiple Times and Watch for Stability
A single result is subject to chance. It is recommended to run the same prompt multiple times and tally the success rate, average time, and process-compliance rate. If you must compare the CLI and GUI forms, clearly mark the environment differences and do not simply attribute the difference to model capability itself.
6.4 Validate in Real Business Scenarios
Beyond general benchmarks, you should also include your own business tasks in the evaluation — for example, domain-document Q&A, codebase maintenance, and data-analysis report generation. Only then can evaluation conclusions truly serve technology selection.
7. FAQ: How to Choose a Lightweight Flagship Model?
How do you distinguish a lightweight flagship model from an ordinary flagship?
Don’t rely on parameters or marketing alone. A more reliable method is to test them on the same task: whether the model follows steps, whether it can complete complex reasoning, whether it can deliver a runnable output, and whether it can control latency and cost.
How do you choose a lightweight flagship model?
Start by clarifying the task type. If it involves deep coding tasks, multi-step automation, or complex analysis, focus on the lightweight flagship model’s instruction following and reasoning depth; if it involves real-time, high-frequency Q&A, focus on testing time to first token and concurrency stability.
What does process review check?
Mainly four points: whether it strictly follows the specified construction steps; whether it fills in the omitted steps; whether the animation traces are controlled; and whether the export feature is genuinely usable.
Why can a lightweight model also be slow?
Because complex tasks trigger more internal reasoning. A lightweight positioning does not equal low latency — especially in scenarios that require deep thinking and long-chain generation, speed may yield to reasoning depth.
8. Conclusion: Understanding Capability Boundaries Helps You Use Models Well
This hands-on test can be summarized in three sentences:
- Capability: In this single sample, the lightweight flagship model showed good instruction following, geometry reasoning, global consistency, and engineering delivery on a geometry animation task that strictly follows a specified construction.
- Cost: Slower output speed was the most notable shortcoming in this test, and deep thinking also brings higher reasoning cost.
- Method: To judge a model’s strength, don’t look only at whether the “result looks similar” — look more at whether the “process is correct.”
For developers, whether to adopt a lightweight flagship model does not hinge on whether it looks close to a high-end model, but on whether it can deliver reliably on the target task. Understanding a model’s capability boundaries and choosing based on task requirements, cost measures, and verification methods is the more sound technical decision. How to choose a lightweight flagship model? The answer is not to look only at parameters, nor only at the final page, but at whether the task can verify the process, whether it can control cost, and whether it can deliver reliably.
