Evaluating LLM Frontend Generation and Multi-Round Search Capabilities: A Practical Guide

    This article argues that evaluating generative models is shifting from reading a text answer to accepting a working artifact. It proposes a five-layer scoring rubric for frontend generation tasks, shows how to make physics and geometry checkable through assertions, how to enforce rendering consistency with a single source of truth, how to catch intent drift between chain-of-thought claims and the actual output, and how to guarantee multi-round search through orchestration-layer budgets rather than model self-reporting. It closes with a shared root-cause analysis and an implementation checklist for building reliable generate–verify–repair pipelines.

    1. Shift the goal from “sounds right” to “actually delivers”

    Over the past year, testing LLMs with tasks like “build an interactive 3D mini-scene” or “train an agent to solve a maze” has become a low-cost, high-information-density stress test. Its value isn’t whether the result looks good, but that these tasks share three traits: the artifact runs, the process is observable, and the result is judgeable. A pure Q&A task only reveals whether the model “says the right thing,” whereas a scene that actually runs exposes weaknesses in code correctness, physical common sense, spatial alignment, and instruction following all at once.

    First, correct the evaluation mindset: visual polish — “the lighting and particles look the part” — is now close to a passing baseline. Almost any general-purpose model with coding ability can produce basic lighting and materials. If you treat this as a primary scoring item, the evaluation becomes distorted. What truly separates capability is three kinds of hard failure:

    • Physics and common-sense errors: geometry that doesn’t hold, constraints that contradict each other;
    • Inconsistent multi-layer rendering: coordinate misalignment or size overflow between the agent layer and the terrain layer;
    • Uncontrolled process and facts: claiming N searches but running only 1, or inventing configuration fields that don’t exist.

    The rest of this article follows a “layered evaluation → typical cases → root cause → engineering fixes” structure.

    2. Break subjective impressions into a measurable five-layer scale

    For frontend-artifact tasks, use a fixed layered score where each layer can be judged automatically, so reviewers aren’t each telling a different story.

    Level What to check How to check Suggested weight
    L1 Runnable Builds successfully, no fatal console errors Build pipeline + smoke tests 20%
    L2 Instruction alignment Each requirement item is met (including goals stated in the chain of thought) Manual/model checklist review 20%
    L3 Physics & geometry Whether collider size, spacing, initial velocity, and conservation relationships hold Assertions + numerical simulation 25%
    L4 Interactive & playable Input response, state machine completeness, ability to reach the goal Automated replay of action sequences 20%
    L5 Engineering quality Code structure, resource usage, frame rate, maintainability Static checks + performance metrics 15%

    The key point is that L3 must be quantified. Take “can a bowling ball clear all pins in one throw” as an example: this needs no expert review; two assertions settle it:

    1. # Turn "common sense" into executable constraints
    2. assert pin_pitch <= BALL_DIAMETER, "Pin spacing exceeds ball diameter; no chain reaction is possible"
    3. assert rack_depth <= BALL_RADIUS * 2, "Rack depth too large; a single trajectory cannot traverse the full row"
    4. # Monte Carlo check: is the offset enough to reach the outermost pins?
    5. hits = simulate_strike(trials=20000, aim_noise=SPREAD_SIGMA)
    6. assert hits.mean() > 0.95, f"Single-ball clear rate only {hits.mean():.2f}; scene design is invalid"

    Models lose points here not because they can’t write collision detection, but because they treat “looks like bowling” as the completion condition and never run a single numerical check. This kind of failure is fully catchable in engineering.

    3. Multi-layer rendering: every visual element must come from the same data

    The most frequent failure in maze-type tasks is “looks right, but isn’t playable”: the terrain mesh is correct, yet the agent runs off the boundary; or the action layer and terrain layer each compute their own offset, so the landing point doesn’t match the logical cell; in extreme cases the agent’s rendered size is far larger than a cell, visually taking up half the map.

    The root cause is singular: multiple sources of truth. The fix principle is equally singular — make the grid data the single source of truth, and derive the terrain layer, agent layer, and Q-value heatmap layer all from the same coordinate function.

    1. CELL = 40 # Cell edge length: shared by all three layers, never hardcode per layer
    2. ORIGIN = (0, 0) # Single origin
    3. def cell_center(r, c):
    4. return (ORIGIN[0] + c * CELL + CELL / 2,
    5. ORIGIN[1] + r * CELL + CELL / 2)
    6. def render(grid, agent_state, q_table):
    7. for r, row in enumerate(grid):
    8. for c, cell in enumerate(row):
    9. x, y = cell_center(r, c)
    10. draw_tile(x, y, cell) # Terrain layer
    11. draw_heat(x, y, q_table[(r, c)]) # Value layer, same coordinate system
    12. x, y = cell_center(*agent_state["pos"]) # Agent goes through the same function
    13. draw_agent(x, y, size=CELL * 0.6) # Size never exceeds a cell

    With three acceptance assertions to match:

    1. assert agent_size <= CELL, "Agent render size out of bounds"
    2. assert all(is_walkable(grid[r][c]) for r, c in path), "Path crosses a wall"
    3. assert within_bounds(agent_state["pos"], grid), "State out of bounds: coordinates and grid not from the same source"

    Worth noting: overlaying the Q-value heatmap on the ground and watching the reward curve converge in later training are the two most valuable observation designs for this kind of task — they turn “did the agent actually learn anything” from a subjective impression into readable curves. In engineering, write convergence into the acceptance criteria — for example, “the moving-average return exceeds a threshold within K episodes, and the variance across three random seeds is below an upper bound” — rather than judging only by the final animated GIF.

    4. Intent drift in the chain of thought: a stated goal ≠ the delivered result

    Another easily overlooked failure: the model clearly says during reasoning that it will present a certain visual style or structure, but the final artifact doesn’t implement it at all. This may be goal-forgetting in a long context, or hallucinatory self-description during the reasoning stage.

    The countermeasure is turning declarations into a checklist. Force the model to output a requirements-item table before generation, then verify each item automatically afterward; a miss deducts points and feeds back for repair:

    1. {
    2. "requirements": [
    3. {"id": "R1", "desc": "Use a solid-color gradient background", "check": "style.bg.type == 'gradient'"},
    4. {"id": "R2", "desc": "Agent size does not exceed a cell", "check": "agent.size <= CELL"},
    5. {"id": "R3", "desc": "Training reward curve converges", "check": "reward_ma.mean() > TH"}
    6. ]
    7. }

    The side effect is positive: once requirements are made explicit, the model’s omission rate drops noticeably, and review gains a unified basis.

    5. Multi-round search: rounds must be guaranteed by the orchestration layer, not self-reported by the model

    This is the highest-business-risk of the three failure types, and it contains two independent problems.

    Problem 1: Uncontrollable round count. Requiring a forced 4 rounds of search, but only 1 actually runs, then repeatedly reusing first-round evidence in the text while claiming to have searched 3 times. Some implementations fire a dozen calls concurrently within a single round, with neither rounds nor call counts bound by any budget. The root cause is handing “rounds” to the model’s self-planning: the model cannot reliably count how many times it called a tool; its self-description is generated text, not runtime state.

    The correct approach is to treat rounds as a hard constraint of the orchestration layer, and to judge whether a round is effective by deduplicating queries:

    1. BUDGET = {"max_rounds": 4, "max_calls_per_round": 3}
    2. used_queries = set()
    3. for i in range(BUDGET["max_rounds"]):
    4. q = planner.next_query(used=used_queries) # Must return a query that hasn't been used
    5. if q in used_queries:
    6. break # Reusing an old query: this round doesn't count
    7. used_queries.add(q)
    8. for evidence in retriever.search(q, top_k=BUDGET["max_calls_per_round"]):
    9. memory.append(evidence) # Trust actual returns
    10. if enough(memory):
    11. break
    12. assert len(used_queries) == BUDGET["max_rounds"], "Round count is enforced by the orchestration layer, not self-reported"

    Also require the model to submit only a “search plan” each time rather than narrating directly, then feed the search results back before the next round — this significantly reduces “pretending to have searched.”

    Problem 2: Key-field hallucination. More seriously, the output invents configuration fields out of thin air — for example, adding a dual-chamber air suspension that doesn’t exist to a car. This isn’t a wording blemish; it’s a core-field error that directly affects purchasing decisions, and its severity should be rated highest and tracked separately from ordinary typos.

    The governance approach is constrained output plus a whitelist: a structured schema restricts the field set, a hit on the authoritative spec cites the source, and a miss explicitly marks “unknown,” forbidding completion from language priors.

    1. {
    2. "type": "object",
    3. "additionalProperties": false,
    4. "properties": {
    5. "model": {"type": "string"},
    6. "air_suspension": {
    7. "enum": ["single_chamber", "dual_chamber", "not_available", "unknown"]
    8. }
    9. },
    10. "required": ["model", "air_suspension"],
    11. "x-nullable-fields": ["air_suspension"]
    12. }

    With two hard rules: first, missing values must fall to unknown with a note “no supporting evidence found in search results”; second, if any field can’t be bound to a specific evidence snippet, the whole conclusion is downgraded to “unverified” and must not be stated affirmatively. In practice, layering a post-check that compares enum values against the authoritative list can bring the interception rate for this kind of error to an acceptable level.

    6. Root cause: three failure types share one affliction

    Abstracting the cases above, they aren’t independent:

    1. Generation ends the job; there’s no closed-loop verification. Code is delivered as soon as it’s written, lacking automated checks for compilation, assertions, unit tests, and headless render replay — so common-sense errors go uncaught.
    2. The process is unobservable. How many times tools were called, what the queries were, where evidence came from — all of it settles in the model’s self-description, and the orchestration layer keeps no trustworthy record.
    3. Output is unconstrained. Fields, enums, and rounds all rely on natural-language agreements, lacking strong constraints from schemas and whitelists.
    4. Self-assessment bias. The model’s statements about “what it did” are generated results and shouldn’t be used as a basis for evaluation or audit.

    7. Implementation checklist

    • Build a generate–verify–repair loop, feeding only structured failure reports back to the model so it doesn’t rewrite by guessing:
    1. def build_artifact(prompt, max_iter=3):
    2. code = generate(prompt, requirement_list=extract_requirements(prompt))
    3. report = None
    4. for _ in range(max_iter):
    5. ok, report = run_checks(code) # build → assertions → unit tests → headless render screenshot
    6. if ok:
    7. return code, report
    8. code = repair(code, report)
    9. return code, report
    • Move constraints upfront: define the requirements-item table, schema whitelist, parameter placeholders, and unknown-value policy before generation.
    • Internalize physical constraints as assertions: translate “looks reasonable” into computable conditions such as spacing, size, conservation, and convergence thresholds.
    • Single-source rendering: define coordinates, scale, and origin each in only one place, eliminating per-layer offset computation.
    • Tool-call budget management: round limits, query deduplication, evidence anchors, and per-citation binding — all enforced by the orchestration layer.
    • Make process and artifact observable: structurally retain each round’s queries, each failed assertion, artifact versions, render screenshots, and key metrics — use object storage for artifacts and screenshots, a logging service for structured events, and monitoring/alerting with thresholds on failure and convergence rates; use a message queue to smooth out spikes in asynchronous search tasks. These capabilities are common categories on mainstream cloud platforms; combine them as needed.
    • Track by severity tiers: count core-field errors, invalid scenes, and visual deviations separately, so an average score doesn’t mask high-severity problems.

    8. Conclusion

    Evaluating generative models is evolving from “reading an answer” to “accepting an artifact.” When a task can be run, asserted, and replayed, evaluation turns from an argument about aesthetics into engineering measurement: physical common sense is judged by numerical simulation, rendering consistency by a single source of truth plus out-of-bounds assertions, and search quality by round budgets plus evidence binding. Do these three things solidly, and no matter which model or version you switch to, the team gets stable, reproducible, comparable conclusions — no longer relying on an unquantifiable judgment like “this one feels a bit better than the others.”