LLM Agents: Controlling Overthinking, Structuring Review Depth, and Governing Budgets

    Evaluating an LLM agent on accuracy or cost alone hides the engineering problems that actually shape the real experience. This article presents a general, reusable method for the cost control and quality assurance of LLM agents, abstracted from a real evaluation scenario with product-specific details removed. It covers a six-dimension model evaluation framework, the causes and remedies for overthinking and redundant tool calls, a layered (L1/L2/L3) review process with a structured output template that prevents reviews from getting lost in details, budget tiering and task orchestration for long-running jobs, and template-based style governance backed by a human feedback loop. Practical entry points include standardized prompt templates with built-in budget constraints, lightweight logging and monitoring, and a weekly review of the most expensive and lowest-quality tasks.

    LLM Agents: Controlling Overthinking, Structuring Review Depth, and Governing Budgets

    Introduction: Single-point evaluation hides the real problem

    In practice, developers tend to judge LLM agents on two axes: one asks “is the answer accurate enough?” and the other asks “is it worth the cost?” These two perspectives look contradictory, but they actually map to different cost structures. Capability decides whether a task can be completed; process and scheduling decide how much budget and time the same task consumes. Evaluating only the first axis makes it easy to overlook the hidden costs introduced by the second.

    Starting from a typical evaluation scenario, this article abstracts a general method for the cost control and quality assurance of LLM agents. The original discussion revolved around the experience of using a new version of a third-party assistant; this article strips away its product attributes and keeps only the universal engineering problems.

    Two phenomena in the original discussion are especially worth noting:

    • The same model performed outstandingly when writing lecture notes for mathematical proofs, yet underperformed on paper and project reviews, taking a long time because it over-checked details.
    • A long task launched at bedtime hit two rolling quota limits in a row during the early morning hours and was ultimately interrupted.

    This shows that a single “model strength” measure is not enough to explain the real experience. A more accurate framing is this: whether model capability, task type, prompt design, and scheduling strategy are well matched is what jointly determines the final result and cost. The following sections analyze these problems one by one and offer actionable remedies.

    1. Getting “strength” right: a multi-dimensional model evaluation framework

    Evaluating a model or an agent should cover at least the following six dimensions. When the consumer is an organization and data sharing or compliance requirements are involved, data handling and compliance capability should also be included.

    Dimension Key question Typical validation method
    Instruction following Does it strictly obey format, length, and scope constraints? Define clear output templates and prohibited items, then measure the violation rate
    Tool-calling decisions Does it search when it should and hold back when it shouldn’t? Record retrieval counts, cited sources, and their correlation with answer accuracy
    Long-conversation stability After many turns, does it drop constraints or drift off topic? Insert conflicting constraints in the later part of a conversation and check whether they are still honored
    Reasoning ability Accuracy and self-checking ability on complex derivations Use question sets with known answers, comparing “answer directly” against “step-by-step self-check”
    Long-form generation Structure, terminology consistency, and citability Have humans score for clarity, redundancy, and terminology uniformity
    Cost and efficiency Budget and time cost per unit of work Track budget consumption, duration, and rerun rate per task

    1. Capability does not equal suitability for every task

    One user shared an experience of generating math lecture notes: they gave a model a proposition that had no ready-made proof in any public reference library and asked it to complete the proof and lay it out as an English handout. The quality of the resulting handout was very high, yet the prompt was only a single sentence. This shows that a model with strong general capability can often produce high-quality output on a well-scoped generation task at very low interaction cost.

    The same phenomenon plays out in reverse on review tasks. Some users report that their accumulated fact-checking, paper-review, and code-review workflows worked well on earlier model versions, but the new version tends to “dig deeper and deeper,” taking a long time before it can give an overall opinion. This is a classic mismatch between task type and model behavior, not a simple matter of the model getting “stronger” or “weaker.”

    2. Math capability tiers: distinguishing “can do” from “can do independently”

    In mathematical reasoning tasks, model capability can be roughly divided into tiers:

    • The first tier can independently explore open problems that have no ready-made answer.
    • The second tier is suited to producing standardized proofs and organizing solutions into handouts when a solution already exists.
    • The third tier needs substantial human intervention for step completion and error localization.

    The three tiers cannot substitute for one another, and the key question is whether a known solution path exists for the task.

    In addition, the length of a reasoning chain is not the same as the quality of reasoning. Some users divide models into two categories: those with “concise chains of thought and a style closer to human writing” and those with “the longest, most complex chains.” The former are not necessarily weaker, while models with overly long chains are more prone to “overthinking,” which carries two risks: first, burning budget on low-value branch verification; second, the deeper the reasoning goes, the more easily it slips up on a critical step. Therefore, when evaluating reasoning ability, one should examine both accuracy and “the reasoning budget consumed per unit of correct output.”

    3. Version iteration needs regression validation

    A new version may bring noticeable changes in capability and style, and may even regress on certain tasks. Typical changes include:

    • More aggressive tool calling, so the model searches on its own even in tasks that need no external information.
    • Output drifting toward a template-like style, with more decorative and vague phrasing.

    For this reason, a model upgrade should not “just switch to the new one”; it should follow a regression-validation process:

    1. Distill frequently used tasks into a fixed evaluation set.
    2. Record each version’s performance on accuracy, budget consumption, and style compliance.
    3. Automate the evaluation process and fold it into the pre-release checklist.

    If resources allow, it is better to bind different model versions to different task types rather than switching the whole system uniformly.

    2. Overthinking: causes and governance of ineffective tool calls

    1. How it shows up

    “Searching even when you didn’t ask it to” is a typical manifestation of an agent overthinking. It can be broken down into several types:

    • Redundant retrieval: searching online to verify common-knowledge questions.
    • Repeated retrieval: querying the same information over and over with different keywords.
    • Unbounded self-checking: the task is already done, yet the model keeps generating additional verification plans.
    • Detail drowning: during a review, the model gets lost in minutiae and cannot reach an overall judgment.

    The direct cost of these behaviors is higher budget consumption and longer task duration; the indirect cost is longer user wait times and useful information buried in process output.

    2. Cause analysis

    From an engineering perspective, there are four main causes:

    • Incomplete goal: the user says only “what to do,” not “what counts as done.” The model can only lower its error risk by raising its level of caution.
    • Reward bias: during training and alignment, “giving evidence” and “checking more” are usually treated as positive behaviors, so the model tends toward a “the more I check, the better” disposition.
    • Fuzzy task boundaries: open-ended research tasks inherently lack a stopping condition, and without external constraints the process expands without end.
    • No budget awareness: the model itself is not aware of the user’s quota constraints unless that information is explicitly written into the context.

    3. Governance strategies

    (1) Define completion and stopping conditions. At the start of a task, provide acceptance criteria and a budget ceiling, and allow the model to return its current best result when the budget runs out.

    (2) Tiered routing. Build task-classification rules so that tasks of different complexity take different tiers:

    • Lightweight tasks (format conversion, information extraction, summarization): use a low-cost lightweight model with deep reasoning off.
    • Medium tasks (routine Q&A, code snippet generation): use a balanced configuration.
    • Heavy tasks (complex proofs, architecture reviews, code reviews): use a long-chain reasoning configuration with budget controls.

    (3) Explicit switches for tool calls. Specify the retrieval policy in the prompt or system configuration. For example, allow retrieval only when “local knowledge is insufficient or the information may be outdated,” and cap the number of retrievals per task.

    (4) Budget awareness. Write budget constraints directly into the task description so the model has a “sense of cost”:

    1. Task: Review the technical proposal in the attachment
    2. Budget constraints:
    3. - Total length no more than 800 words
    4. - At most 2 external retrievals, used only to verify data explicitly marked "to be verified"
    5. - Give the overall conclusion first, then list the key issues
    6. Stopping condition: End after completing the three items above; do not add further verification

    (5) Observability of the process. Use logging and monitoring alerts to record the following data for quantitatively evaluating improvements:

    • Budget consumption and retrieval counts per task.
    • Execution time and error rate of each phase.
    • Task completion rate and human rework rate.

    3. Layered review: keeping reviews out of the weeds

    1. The essence of the problem

    The goal structure of review tasks is fundamentally different from that of generation tasks. Generation pursues “complete content”; review pursues “ranked risks.” The two demand very different cost allocations:

    • Generation tasks should concentrate cost on content quality.
    • Review tasks should concentrate cost on “finding high-severity problems.”

    If a model spends most of its budget on details unrelated to the overall conclusion, it falls into the “detail-verification” trap, which is especially visible in fact-checking, paper reviews, and code reviews. Code review also includes identifying low-quality AI-generated code (what the industry often calls “AI slop”): such code is structurally complete on the surface but often has empty comments, repetitive boilerplate, and similar issues, so it needs an overall judgment rather than line-by-line scrutiny.

    2. A three-layer review process

    Break the review into three layers and converge layer by layer:

    Layer Goal Main approach Suggested budget share
    L1 Structural screening Quickly judge whether there are blocking problems Check completeness, consistency, and obvious factual errors ~20%
    L2 Key verification Verify the arguments and data with the greatest impact on the conclusion Sample citations, data definitions, and key derivation steps ~50%
    L3 Overall assessment Give an overall judgment and improvement priorities Summarize by severity and form actionable recommendations ~30%

    Each layer uses a different model tier: L1 uses a lightweight model for fast filtering, while L2 and L3 use long-chain reasoning models for deeper handling.

    3. A structured output template

    Review results must include an “overall conclusion” and a “severity grading.” You can apply the following prompt template directly:

    1. You are a senior technical reviewer. Output your review conclusion in the following format:
    2. 1. [Overall conclusion] Pass / Conditional pass / Fail (one sentence stating the core reason)
    3. 2. [Blocking issues] Ordered from highest to lowest severity, each containing:
    4. - Location: paragraph or section number
    5. - Description: the specific manifestation of the issue
    6. - Basis: the facts or reasoning chain that support the judgment
    7. - Recommendation: an actionable direction for the fix
    8. 3. [Important issues] Same format as above
    9. 4. [Suggestions] Non-mandatory items
    10. Requirements:
    11. - The overall conclusion must come first and must not be buried after the details
    12. - Do not enumerate every minor issue; prioritize reporting issues that affect the overall conclusion
    13. - If the overall quality is good, state clearly that "no blocking issues were found"; do not pad the list with irrelevant details

    The core of this template is “conclusion first, details later.” Once the output structure is fixed, the model must complete the overall judgment first and then organize its evidence around that judgment, which prevents it from being led around by details.

    4. Segmented review of long documents

    For long papers or large codebases, throwing the entire budget at them at once is often uneconomical. A “segment in parallel, decide in aggregate” strategy works better:

    1. # Pseudocode: segmented review with aggregation
    2. def review_long_document(doc, budget_ratio=(0.2, 0.5, 0.3)):
    3. # 1. Split the document into independently processable fragments by section or module
    4. sections = split_by_structure(doc)
    5. # 2. L1: light model scans in parallel to quickly flag suspected high-risk fragments
    6. l1_results = [light_model.scan(s) for s in sections]
    7. # 3. L2: deep-verify only high-risk fragments to avoid checking everything
    8. high_risk = [s for s, r in zip(sections, l1_results) if r.is_high_risk]
    9. l2_results = [deep_model.verify(s) for s in high_risk]
    10. # 4. L3: aggregate all results into an overall conclusion with priorities
    11. report = deep_model.synthesize(
    12. overall_summary=l1_results,
    13. risk_findings=l2_results,
    14. template=REVIEW_TEMPLATE,
    15. )
    16. return report

    This has two benefits:

    • Most fragments only need lightweight handling, so the budget is concentrated on the parts that genuinely need deep review.
    • A single aggregation step produces the overall conclusion, avoiding fragmented opinions.

    5. Prompt techniques for review tasks

    • Declare the review scope and focus up front: for example, “This review focuses on architectural feasibility and data consistency, not writing style.”
    • Separate “verified” from “inferred”: require the model to explicitly mark which are verified facts and which are inferences. This reduces vague phrasing.
    • Build in an anti-hallucination mechanism: when evidence is insufficient, require the model to answer “insufficient evidence, cannot judge” rather than forcing a conclusion.

    4. Budget governance: orchestration and allocation for long tasks

    1. Where the problem comes from

    Long tasks are the main source of runaway budgets, for the following reasons:

    • Uncontrollable task duration: complex reasoning tasks can run for tens of minutes.
    • Stacked limits: common limiting patterns include a context cap per request, cumulative quotas within rolling time windows (for example, one window every 5 hours, another every 7 days), and concurrency limits. When multiple windows stack, users can easily trigger several limits at once without realizing it.
    • Unattended nighttime runs: a user starts a task before bed and cannot adjust the strategy when limits are hit in the early morning, so the task can only be interrupted passively.

    2. Budget tiering and task orchestration

    (1) Budget tiering. Divide the total budget into three parts and set alert thresholds:

    • Exploration budget (~30%): for analysis, retrieval, and trial and error.
    • Execution budget (~50%): for formal generation and verification.
    • Contingency budget (~20%): for unexpected problems, such as regeneration.

    (2) Task splitting and checkpoints. Break a long task into several independently completable stages, and persist the intermediate results of each stage (via object storage or local files). If a limit is hit, the task can resume from the checkpoint rather than starting over.

    (3) Phased scheduling. Split a long task into multiple short phases that avoid peak periods of the rolling window. For example, break a 30-minute task into six 5-minute subtasks scheduled through a message queue or a scheduled job. Each subtask consumes budget independently and will not hit the window cap all at once.

    (4) Cost monitoring and alerts. Monitoring can be granular down to a single task’s budget consumption. Set alert thresholds (for example, trigger a reminder when a single task consumes more than 80% of its budget) so you can step in and adjust in time.

    3. Phased budget allocation for math proof tasks

    Take “generating lecture notes for a mathematical proof” as an example. You can adopt the following budget allocation and scheduling strategy:

    Phase Main work Suggested budget share Scheduling
    Understand and plan Parse the proposition’s conditions, break down the proof approach 10% Synchronous
    Main proof Generate the derivation 40% Interruptible, save intermediate results
    Self-check and verification Check the logic chain, verify key lemmas 30% Segmented, avoid peak hours
    Formatting and polishing Organize formatting, unify terminology 20% Lightweight model

    The key is to separate the “self-check and verification” phase from the “main proof” phase. The main proof is a long-chain task that easily exhausts the budget; self-checking can be done in segments, verifying only one lemma or one reasoning step at a time, which keeps the budget consumption of each run controllable.

    4. Cross-session and cross-model budget collaboration

    A complex task need not rely on a single model. You can adopt a “primary-plus-auxiliary” collaboration model:

    • The primary model handles core reasoning.
    • The auxiliary model handles peripheral work such as parallel verification, format conversion, and terminology proofreading.

    Coordinating the two through a unified task queue and message bus lets you complete more thorough verification without adding to the primary model’s load. At the same time, all call records are aggregated into a unified logging service, making it easier to later analyze the budget distribution of each phase and continuously optimize the scheduling strategy.

    5. Style governance: from “usable” to “good”

    1. Causes of style differences

    Different versions within the same model family can differ noticeably in output style. In short conversations, earlier versions tend to lead with conclusions and stay concise; in long conversations, they may pile on metaphors and coin new terms. A new version may swing to the opposite extreme: a more “plain water” style that likes to list evidence but adds more hedging.

    The root of style differences lies in differences in training-data distribution and alignment objectives. Different versions introduce different data during instruction fine-tuning and preference optimization, which shifts the output style. This is not a “capability regression” but a “style drift.”

    2. Templating style constraints

    For scenarios that require strict style (such as academic papers, technical documentation, and formal reports), you can add a style-constraint section to the prompt:

    1. Writing style requirements:
    2. - Use concise, objective phrasing; avoid literary rhetoric
    3. - Lead with conclusions: give the judgment first, then the evidence
    4. - Use terminology consistently, and define a term the first time it appears
    5. - Avoid vague phrasing; if you need to express uncertainty, state the reason and degree clearly
    6. - Use lists and tables where appropriate to improve readability

    Such constraints are not complicated, yet they can significantly improve the consistency and readability of the output. For scenarios that need more of a “human touch” (such as popular-science articles aimed at a general audience), you can relax the constraints appropriately and allow more rhetoric.

    3. Human review and a feedback loop

    Style problems ultimately need human oversight. Build a “generate–review–feedback” loop:

    • Reviewers score against a unified style rubric (for example, conciseness, objectivity, terminology consistency).
    • The specific issues behind low scores are fed back into the prompt template, forming an iteration.
    • For frequently recurring style problems, you can distill a “banned-phrasing list” and declare it directly in the prompt.

    Through multiple rounds of iteration, the model’s output style gradually converges toward the level the team expects.

    6. Summary and practical recommendations

    Around the cost control and quality assurance of LLM agents, this article lays out a systematic method from six angles:

    1. Multi-dimensional evaluation: assess a model across instruction following, tool calling, long-conversation stability, reasoning ability, long-form generation, and cost efficiency, avoiding the bias of a single dimension.
    2. Task tiering: choose different model tiers by task complexity, so lightweight tasks do not “use a sledgehammer to crack a nut.”
    3. Overthinking governance: reduce ineffective retrieval and self-checking through clear completion definitions, budget constraints, and tool-calling switches.
    4. Layered review: break the review into three layers, L1/L2/L3, paired with a structured output template to avoid getting lost in details.
    5. Budget orchestration: make long tasks controllable, recoverable, and optimizable through budget tiering, task splitting, checkpoints, and monitoring alerts.
    6. Style governance: keep output style stable and controllable through templated constraints and a human feedback loop.

    In practice, start with three small entry points:

    • Write standardized prompt templates for your most frequently used task types, including budget constraints and output-format requirements.
    • Set up lightweight logging and monitoring to record each task’s budget consumption and completion status.
    • Hold a weekly review to identify the tasks with the highest budget consumption and the worst results, and optimize them specifically.

    The value of an LLM agent does not lie in “solving every problem in one shot,” but in “reliably producing usable results at a controllable cost.” Only by getting cost control and quality assurance right can an agent truly become a productivity tool rather than a “quota burner.”