Why New Foundation Models Underperform in AI Agents: From Base Capability to Production-Ready Systems

    A newly released foundation model can excel in chat, fluent writing, and hard reasoning yet stumble the moment it is wired into an agent system—declaring a task "done" after one turn, quitting after a single tool error, or drifting from the original goal after dozens of interactions. This article explains why: an agent's real-world performance is roughly the product of base capability, post-training alignment, and engineering quality, and a new release usually ships with a strong base but immature interaction training. It walks through the industry pattern of multi-generation iteration, the systematic failure modes agents exhibit, and nine engineering measures—layered evaluation, externalized state, narrow tool design, deterministic orchestration, independent validation, sandboxing, full-chain observability, failure clustering, and model routing—that let teams contain model uncertainty and turn new-model adoption from a risky migration into a measurable, gradual rollout.

    1. Restating the Problem: Where the Experience Gap Comes From

    After a new model version ships, many developers feel a letdown. The model is clear and logical in chat, writes fluently, and can solve hard reasoning problems—yet the moment it is wired into an agent system, things change:

    • Some models announce “done” after a single turn.
    • Some give up after hitting one tool error.
    • Some lose the original goal after dozens of interactions.

    The common verdict is a flat “this model is hard to use.” A more precise way to judge agent behavior is to break it down:

    1. Agent performance ≈ base capability × post-training alignment × engineering quality

    These three factors are multiplicative: if any one of them nears a bottleneck, overall performance drops sharply. Base capability determines what the model “knows and can reason about”; post-training alignment determines whether it “can actually act inside a loop”; engineering quality determines whether it “has a stable, executable environment to act in.” The three are independent—a strength in one cannot offset a weakness in another.

    Taking the common industry pattern of model version iteration as a starting point, this article analyzes why new releases often underperform in agent scenarios and lays out a systematic set of engineering remedies.

    2. How Agent Capability Is Built: Iteration, Not Instant Mastery

    Reviewing the evolution of the major model families reveals a similar pattern: new versions in the same family rarely arrive fully formed. They improve across multiple generations. The table below summarizes the differences across three generations in schematic form:

    Dimension First-gen new architecture (base v1) General capability enhanced Agent-optimized
    Main improvement Knowledge and reasoning ceiling under a new architecture and data mix Instruction following, long context, complex reasoning, multimodality Multi-turn tool calling, long-horizon planning, error recovery
    Training focus Mainly pretraining Continued pretraining + general SFT + preference alignment Interaction trajectory data and environment-feedback reinforcement on top of general alignment
    Chat and writing experience May already be strong Clearly improved Improved further, but usually not the main target
    Agent scenario performance Volatile; hallucinations and mid-task abandonment are common Gradually improving Marked gains in stability and completion rate
    Suited tasks Single-turn Q&A, content generation, assisted analysis Multi-turn dialogue, long-document processing Multi-tool collaboration, long-pipeline automation, long-running tasks

    The key row is “training focus.” For the later generations from each vendor, the training emphasis gradually shifts from “making the model smarter” to “making the model more reliable in interaction.”

    This shift has a clear technical necessity:

    • The bottleneck is transfer. Base capability solves single-step reasoning problems—“how do I solve this problem,” “how should this code be written.” An agent faces sequence decisions spanning tens or even hundreds of steps: each step chooses an action, interprets an observation, and revises the plan. Excelling at single-step tasks does not automatically confer this ability.
    • The data shape differs. General training corpora are mostly static text, whereas agent training requires large amounts of “task–action–observation–result” trajectories. This data can only be accumulated through extensive trial in real or simulated environments and is hard to backfill quickly.
    • The reward signal differs. General alignment optimizes whether a response is appropriate; agent training cares whether the task is actually completed. The latter usually relies on verifiable judgments—whether unit tests pass, whether business rules are satisfied, whether the final artifact is usable.

    A telling comparison: one model family was only widely recognized as stable and usable on agent tasks after two to three generations of iteration; another followed the same path from a base version to an enhanced version to an interaction-optimized version; and yet another made agent interaction its primary optimization target from the earliest releases. What they share is “enough iterations”; they differ only in starting point and path.

    3. The First Release After a Long Pause: Why It Inevitably Stumbles

    When a new-architecture model appears for the first time after a long wait, it is typically in a “strong base, unfinished interaction” stage. This is exactly when it gets wired directly into agent systems—and when the sense of letdown is strongest. There are three reasons.

    First, the initial release usually prioritizes general alignment over interaction alignment. The first release carries the task of establishing a baseline and validating the architecture. Resources go first to instruction following, safety, and basic reasoning. Building tool-call training data, setting up sandbox environments, collecting trajectories, and designing rewards all come only after the base is stable.

    Second, interaction failure modes are systematic, not occasional. In practice, these models tend to “act up” on moderately complex agent tasks, as shown below:

    Failure mode Typical symptom Root cause Engineering countermeasure
    Hallucinated tool calls Calls nonexistent tools, fabricates arguments or return values Unfamiliar with the true distribution of the tool protocol and environment Strict schema validation; server rejects illegal calls and returns structured errors
    Premature completion Claims “done” while the artifact is missing or wrong Lacks a training signal for task-completion judgment Require completion evidence; auto-verify at the end
    Fragile abandonment Terminates or asks for help after one error Lacks training on error recovery and retry strategies Inject retry and reroute strategies at the orchestration layer
    Instruction drift Deviates from the original goal after many turns Weak long-horizon goal retention and planning Externalize task state; restate the goal at intervals
    Context pollution Misled by early errors and trial-and-error messages Lacks filtering and compression of history Structured state and summary compression
    Confident errors Produces plausible-looking but wrong conclusions Lacks a “verify-first” behavior habit Decide with an independent verifier rather than trusting the model’s self-report

    These modes share one common trait: most are not “can’t” but “was never trained on what to do in this situation.” So short-term prompt tweaks yield limited returns—you need “model-side iteration plus engineering-side hardening” working together.

    Third, the baseline the release is measured against keeps moving. In the agent ecosystem, tool protocols, framework interfaces, and development conventions evolve continuously. A model trained earlier may not cover interaction patterns that appeared later. Judging it by today’s standards naturally produces an impression of “not good enough.”

    4. Systematic Engineering Remedies

    For developers, rather than waiting for the next version, the better move is to build a “model-swappable” engineering system. This hedges against fluctuations in model capability and enables a smooth switch when a stronger version arrives. The nine measures below are ordered by implementation priority.

    4.1 Build Evaluation First, Then Talk About “Usability”

    Personal experience is easily swayed by a single task or a single prompt. To decide whether a model meets your business needs, you need a layered, repeatable evaluation set:

    1. Task layering. Four levels are suggested:
      • L0 single-step tool call: focus on “picking the right tool and filling the right parameters”;
      • L1 multi-step tool collaboration: focus on the handoffs between steps;
      • L2 long-horizon tasks: include branch decisions, error recovery, and goal retention;
      • L3 end-to-end business tasks: judged by whether the final artifact meets the bar.
    2. Sample composition. Each level should cover the happy path, boundary conditions, and adversarial scenarios—for example, missing parameters, tool timeouts, and empty results.
    3. Evaluation metrics. Beyond task success rate, also track average steps, cost per task, human intervention rate, and number of privilege-violating operations—to avoid “succeeds but expensive” or “succeeds but dangerous.”
    4. Versioned execution. Put the evaluation set under version control, and run an automatic regression in CI on every model change.

    4.2 Use Context and State Governance to Make Tasks “Simpler”

    When a model drifts after many turns, the root cause often lies in context management rather than the model itself. Effective practices include:

    • Externalize state. Keep the task goal, completed steps, and key constraints in an external structured object instead of relying only on the conversation history.
    • Sliding window + summary. Retain the system instruction and the most recent few turns, and compress older process information into summaries.
    • Layered memory. Put long-term knowledge in a retrieval store and fetch it on demand; update short-term working state in real time as the task advances.
    1. from dataclasses import dataclass, field
    2. @dataclass
    3. class TaskState:
    4. goal: str # task goal: stays constant across the whole run, unaffected by context changes
    5. constraints: list[str] # hard constraints, e.g. write operations are forbidden
    6. completed: list[dict] = field(default_factory=list) # completed steps and their evidence
    7. pending: list[str] = field(default_factory=list) # pending steps
    8. retries: dict[str, int] = field(default_factory=dict) # failure count per step
    9. def build_context(state: TaskState, recent: list[dict], budget: int = 8000) -> list[dict]:
    10. # 1) always carry the original goal and constraints to prevent instruction drift
    11. # 2) keep only conclusions and evidence summaries for completed steps, filtering noise from failed attempts
    12. # 3) append the most recent turns; when over budget, trim from the earliest history first
    13. ...

    The core principle: anything the model must remember at every step should be externalized as structured state and re-supplied explicitly at each turn.

    4.3 Design Tools to Be “Narrow and Precise”

    A model that can “infer” the correct usage in a unit test may still pick the wrong tool in a real task. The goal of tool design is to shrink the room for the model to err:

    • Keep the count lean. Keep one tool per responsibility; avoid overlapping functionality that makes selection hard.
    • Be specific in descriptions. State applicable and non-applicable scenarios, the meaning of each parameter, and the structure of the return value.
    • Enforce strict schemas. Validate parameter types and enum values with structured schemas, and reject illegal parameters outright.
    • Make errors readable. Return “error code + human-readable cause + fix suggestion” instead of a generic exception.
    1. # Tool parameter definition: fields, types, and value ranges are all explicitly constrained
    2. TOOL_SCHEMA = {
    3. "name": "query_order",
    4. "description": "Look up an order by order ID. Read-only; cannot modify orders.",
    5. "parameters": {
    6. "type": "object",
    7. "properties": {
    8. "order_id": {"type": "string", "pattern": r"^[A-Z]{2}\d{10}$"},
    9. "fields": {"type": "array", "items": {"enum": ["status", "amount", "paid_at"]}}
    10. },
    11. "required": ["order_id"],
    12. "additionalProperties": False # reject fabricated extra parameters
    13. }
    14. }
    15. def call_tool(name: str, args: dict) -> dict:
    16. try:
    17. # validate parameters against TOOL_SCHEMA; reject the call if validation fails
    18. ...
    19. return {"ok": True, "data": result}
    20. except ToolError as e:
    21. # return a structured error so the model can fix it rather than guessing repeatedly
    22. return {"ok": False, "error_code": e.code, "message": str(e), "hint": e.hint}

    4.4 Constrain Free Play with an Orchestration Skeleton

    Building an agent as a “completely free loop” means staking correctness on the model’s decision at every single step. A more robust approach is “skeleton + local intelligence”:

    • The main flow (e.g., “retrieve → analyze → generate → verify”) is controlled by deterministic code;
    • The model handles only the local decisions inside each node;
    • Before entering the next node, check preconditions; if unmet, roll back or abort.

    This sharply compresses the span over which the model must maintain long-horizon consistency, so a single-point mistake no longer escalates directly into an overall failure.

    4.5 Let Validators Judge Right and Wrong

    A model lacks reliable self-awareness of its own output, and self-reporting “done” does not mean it is truly done. Introduce an independent, deterministic verification step:

    • Code tasks: unit tests, static checks, type checks;
    • Data tasks: business-rule validation, consistency comparison;
    • Text tasks: citation-traceability checks, format-compliance checks.

    When verification fails, feed the structured failure information back to the model for repair, forming an “execute–verify–fix” loop, and set a maximum number of rounds to avoid an infinite loop.

    1. def run_agent(model, tools, task, validator, max_rounds: int = 10) -> dict:
    2. state = TaskState(goal=task.goal, constraints=task.constraints)
    3. for _ in range(max_rounds):
    4. action = model.decide(build_context(state, tools.get_recent()))
    5. obs = tools.invoke(action) # invoke tool, return a structured result
    6. state.completed.append({"action": action.name, "result": obs})
    7. verdict = validator.check(task, state, obs) # independent validator decides completion
    8. if verdict.passed:
    9. return {"success": True, "evidence": verdict.evidence, "rounds": ...}
    10. if verdict.needs_fix:
    11. state.pending.append(verdict.feedback) # feed the failure reason back to the model for repair
    12. continue
    13. # single-step failure that cannot be fixed: the orchestration layer decides retry or reroute, not the model giving up
    14. ...
    15. return {"success": False, "reason": "max_rounds_exceeded", "state": state}

    4.6 Sandboxing, Least Privilege, and Rollback

    The more capable the agent, the higher the cost of a misoperation. The execution environment should meet these requirements:

    • Isolation. Run in a container or dedicated runtime, restricting outbound network access and the file-system scope;
    • Least privilege. Grant only the read/write permissions required to complete the task;
    • Resource limits. Set caps on timeout, memory, and output size to prevent runaway loops from taking down the service;
    • Operation auditing. Introduce approval or human confirmation for high-risk write and delete operations;
    • Rollback. Support fast recovery for key changes through versioned configuration, snapshots, or a branching mechanism.

    4.7 Full-Chain Observability

    Agent problems are “long-chain problems”; without link data you can only locate issues by guesswork. Record the following for every execution:

    • A unique task and step identifier to stitch the full chain together;
    • The model input (truncated prompt and tool list), output and arguments, and tool returns;
    • Time spent and token consumption per step;
    • The final verdict and a classification of the failure reason.

    On the cloud, these capabilities are usually assembled from general building blocks such as logging services, object storage, and monitoring and alerting: logs are aggregated for search, raw traces are archived to object storage for replay, and alerts fire automatically when key metrics deviate. For example, general-purpose components such as the logging and object storage services offered by Baidu AI Cloud can serve as a common implementation of this stack. No model—however “correct” or “stronger”—exempts you from this step; it determines how fast your team can improve.

    4.8 Failure Clustering and Regression Replay

    Collected failure traces should not be “reviewed once and forgotten.” Periodically cluster them by failure mode (e.g., parameter hallucination, timeout abandonment, repetitive loops) to produce an actionable improvement list:

    • If high-frequency failures concentrate in one category, prioritize fixing the tool description or the orchestration strategy;
    • If a class of tasks consistently cannot be completed, consider introducing finer-grained tools or clearer validation rules;
    • Fixed failure samples should be solidified into regression cases to prevent later version changes from reintroducing regressions.

    4.9 Model Routing by Tier

    Different tasks demand very different levels of capability. Simple, low-risk, read-only tasks can go to a low-cost model; complex, write, or high-value tasks are routed to a stronger model. Routing rules can be based on task type, risk level, and step count:

    1. def route(task, cheap_model, strong_model):
    2. # complex tasks: write operations, cross-system collaboration, or multi-step reasoning
    3. if task.risk_level == "high" or task.estimated_steps > 8:
    4. return strong_model
    5. # simple tasks: read-only, clear parameters, few steps
    6. if task.risk_level == "low" and task.tool_set.is_read_only:
    7. return cheap_model
    8. return strong_model

    When a stronger model becomes available, routing only needs to adjust the weights—no business logic changes are required.

    5. On “Cheap” and “Targeted Optimization”

    When discussing models, a common misjudgment is: one model costs a fraction of another, so it seems not worth taking seriously. Price reflects the cost per token, not the whole picture of capability. An end-to-end comparison should look at the “cost per successful task”:

    1. cost per task = single-step call cost × average steps × average attempts + cost of failed retries and human intervention

    For example, a general large model may have stronger single-step reasoning and reach an answer in one step on simple tasks; a cheaper model, lacking interaction training, may need many exploratory steps on average and have a higher failure rate. In that case, the cost per task of the former can actually be lower. Conversely, a model specifically trained for agent interaction at a modest unit price, if it converges reliably on your target task types, may be just as cost-effective overall.

    So “cheap” is no reason for dismissal, and “optimized for interaction” does not mean stronger in every scenario. The only reliable basis for judgment is the success rate and cost per task measured on your own business evaluation set.

    At the same time, two points deserve an objective view:

    • The base ceiling sets the upper limit. If the new base is first-tier in parameters, training data, and training compute, its capability ceiling is usually high. What limits performance is mainly interaction training and the engineering environment, not the underlying capability.
    • Interaction optimization takes time. From base release to a stable, usable agent, several rounds of iteration are typically required. Throughout that process, engineering-side remedies are usually more practical than waiting.

    6. Conclusion: Turn “Is the Model Good” into “Is the System Controllable”

    When a new model version underperforms in agent scenarios, most of the time it is not that the model “got dumber,” but that its stage means the capability focus has not yet aligned with interaction needs. For developers, three actionable principles apply:

    1. Evaluate first. Replace personal impressions with a self-built, layered evaluation set, so every model change has a comparable baseline;
    2. Fall back on engineering. Use externalized state, narrow tools, deterministic orchestration, independent validation, sandboxing, and observability to keep the model’s uncertainty within controllable bounds;
    3. Use in tiers. Route models by task risk and complexity—spend strong capability where it matters, and let low-cost models handle high-frequency simple tasks.

    Once this system is in place, a new model release stops being “a risky migration” and becomes “an assessable, gradual upgrade.”