Agent Engineering: A Practical Guide to Building Reliable Agents on Pretrained Foundation Models

1. The Underestimated Starting Point: Capability Baselines Are Rising Fast

One judgment circulates widely in the industry: the level shown by today’s best models looks more like a floor than a ceiling. If the last time you seriously evaluated model capability was a few months ago, your technical judgment has probably gone stale. Capability is iterating faster than most teams update their mental models and their evaluation sets.

For engineering teams, this has three direct implications:

• Do not treat “current failure samples” as a permanent capability boundary. Tasks previously judged as “the model cannot do this” (long chains of tool calls, multi-file code modification, navigation in open-ended environments) may have become routine with a base model upgrade. Meanwhile, manual fallback logic designed on top of those old conclusions turns into a burden on performance and cost.
• Do not try to patch capability gaps with prompt tricks. A large body of “prompt engineering” work is tightly bound to a specific model version. The moment the base model is upgraded, it either stops working or starts conflicting with new behavior. The sustainable approach is to make the stable parts explicit (business constraints, tool contracts, acceptance criteria) and leave the parts that will change (reasoning and generation) to the model.
• Do not equate “a strong model” with “a good business outcome.” Sitting between the two are the evaluation set, tool quality, state management, and permission boundaries. Without this middle layer, a capability upgrade never converts into business value.

There is also a structural shift: the agent research direction is “returning” to pretrained bases. Early agents often relied on reinforcement learning or rule search to learn from scratch, which was expensive and generalized poorly. Now an agent can start out standing on a general pretrained foundation model.

2. What the Pretrained Base Actually Gives an Agent

If you break the priors of a general model into engineering terms, they come down to roughly four reusable assets:

1. World knowledge priors. A basic understanding of domain concepts, industry conventions, and common document structures, which removes a lot of cold-start rule configuration.
2. Procedural skills. Writing code, reading structured API documentation, emitting structured data in an agreed format, and translating between natural language and formal descriptions. As a result, “using a tool” no longer requires training a separate policy for every single interface.
3. Text and multimodal understanding. Logs, reports, contracts, screenshots, and the way UI elements are organized have all been seen repeatedly during training, so extraction and consolidation start from a fairly high baseline.
4. Task decomposition tendency. When facing complex instructions, models tend to break the work into sub-goals on their own, which gives the “plan — execute — verify” orchestration loop a usable starting point.

An analogy: every new agent has already finished reading the general textbooks before it is “born.” What engineering actually has to supply is the part of its environment that the textbooks do not cover.

Where Inherited Intelligence Stops Is Exactly What Engineering Fills In

Capability gap	Typical symptom	Engineering response
Missing up-to-date knowledge	Cites deprecated APIs or outdated business rules	Retrieval augmentation, real-time tool queries, versioned knowledge base
Missing private context	Knows nothing about internal terminology or processes	Retrieval plus a memory layer plus structured business rule injection
No state, no side effects	Cannot actually change any external system	Tool layer and execution environment (sandbox, approval, audit)
Hallucination and overconfidence	Invents field names, invents query results	Structured validation, deterministic code fallbacks, traceable citations
Goal drift on long tasks	Drifts away from the original requirement after many turns	Plan snapshots, progress ledger, context compression and re-anchoring

From this, one core design principle follows: the model is responsible for “judgment and generation”; deterministic code is responsible for “computation, state transitions, and permission control.” Anything that can be expressed precisely in code (summing amounts, evaluating date ranges, permission checks, writing data to storage) should not be left to the model’s free interpretation.

3. Reference Architecture: Five Layers

A sustainable, evolvable agent system usually divides into five layers:

• Interaction layer: task entry points, progress visibility, human intervention and confirmation points.
• Orchestration layer: loop control, plan management, budget control, termination conditions, failure retry and fallback.
• Capability layer: the tool set and execution environment (code execution sandbox, data access, external system calls).
• Memory layer: session context, task state ledger, long-term knowledge and accumulated experience.
• Observability and governance layer: end-to-end logs, trace replay, evaluation and regression, cost and security auditing.

The key design constraint: the orchestration layer and the capability layer must be decoupled. Adding, removing, or modifying tools should not force a rewrite of orchestration logic, and changes to orchestration strategy should not affect the permission model of the tools.

4. The Orchestration Loop: A Minimal Working Skeleton

The following is illustrative pseudocode. It shows structure, not any specific API.

MAX_STEPS = 12          # Step cap, prevents infinite loops
MAX_TOKENS = 200_000    # Global budget, used for cost control
async def run_task(goal: str, ctx: Context) -> Result:
    plan = await planner(goal, ctx)        # Generate a structured plan: sub-goals + acceptance criteria
    ledger = TaskLedger(plan)              # Task ledger, persisted to state storage
    for step in range(MAX_STEPS):
        if ctx.tokens_used > MAX_TOKENS:
            return Result(status="budget_exceeded", ledger=ledger)
        observation = ctx.recent_observations()      # Trimmed to fit the context budget
        action = await policy(goal, ledger, observation)
        if action.type == "finish":
            verdict = await verifier(goal, ledger)   # Independent verifier, avoids self-grading
            if verdict.ok:
                return Result(status="ok", ledger=ledger)
            ledger.append_feedback(verdict.reasons)  # Continue with the failure reasons attached
        elif action.type == "tool_call":
            result = await tools.invoke(                 # Permission check + idempotency + timeout
                action.name, action.args,
                requires_approval=is_sensitive(action),
            )
            ledger.record(action, result)                # Persist the full trace
        ctx.compress(keep=ledger.summary())              # Compress stale context
    return Result(status="step_exceeded", ledger=ledger)

Four details in this skeleton are easy to overlook but decide whether it works:

• Separate planning from execution. Freeze the plan into a structured object first (including acceptance criteria), then check against it repeatedly during execution. This substantially suppresses goal drift.
• An independent verifier. Letting the “generator” grade itself is just relaxing the standard. Verification should land on executable checks wherever possible (unit tests pass, schema validation passes, required fields are non-empty, citations are traceable).
• Budget as a control plane. Steps, tokens, elapsed time, and call counts all need to be first-class citizens, not merely monitoring metrics.
• Persist the full trace. Without a replayable trace, you cannot attribute failures, and you cannot accumulate an evaluation set.

5. Context Engineering: Managing a Limited Window as a Budget

Context is the number one engineering variable that determines agent performance. In practice it is usually layered by priority:

1. System constraints (non-overridable): safety boundaries, tool usage rules, output format requirements.
2. Task and plan: the goal, acceptance criteria, current sub-goal, completed and remaining items.
3. Retrieved fragments: injected by relevance and freshness, and always carrying a source identifier so they can be traced.
4. Recent trajectory: only tool calls and result summaries relevant to the current sub-goal.
5. Compressed history summary: stale details compressed into conclusions, keeping the rationale behind decisions and the reasons for failures.

Three practical recommendations:

• Give context a “quota” instead of “filling it up.” Reserve headroom for output and tool results, so a long task does not get truncated mid-way and end up with a broken structure.
• Do not let large results enter the context. Large files, large tables, and long logs should be persisted as addressable temporary artifacts; only the path, a summary, and a query interface should go to the model.
• Watch for context contamination. A single wrong tool output that keeps getting cited will propagate through every subsequent reasoning step. When a wrong result is discovered, explicitly mark it as invalid rather than expecting the model to forget it on its own.

6. Tool Design: Contract Quality Sets the Agent’s Ceiling

The same base model, equipped with poorly written tools, performs far below expectations. A tool contract should be fixed at six elements:

• Naming: verb plus object, semantically unique, avoid having near-synonym tools coexist.
• Semantic description: spell out “when to use it and when not to use it,” not just “what it can do.”
• Parameter schema: types, required flags, value ranges, enums, and defaults all declared explicitly.
• Return structure: stable fields with brief semantic descriptions; avoid returning “one big blob of text.”
• Error semantics: distinguish “retryable” (timeout, rate limiting) from “non-retryable” (insufficient permissions, invalid arguments), and make the error message include a suggested next step.
• Permissions and idempotency: declare required permissions, whether the operation is idempotent, and whether it can be rolled back.

An example of a structured error response:

{
  "ok": false,
  "code": "RATE_LIMITED",
  "retryable": true,
  "retry_after_ms": 1500,
  "message": "Query rate exceeded, please retry in 1.5 seconds",
  "hint": "Narrow the time range to within 7 days, or switch to a paginated query"
}

Paired with this is operation tiering and human confirmation. At minimum, three tiers are recommended:

• Read-only operations: queries, retrieval, statistics. Can run directly, with an audit log.
• Reversible write operations: draft creation, configuration changes. Can run and be executed with a rollback point retained.
• Irreversible or high-impact operations: money movement, deletion, outbound sending, production environment changes. Require human confirmation plus a second verification plus full auditing.

For the execution environment, code execution and data access should happen in an isolated sandbox: a separate container, a least-privilege account, an outbound network allowlist, temporary credentials, and credentials revoked when the session ends. Credentials should be centrally managed in a secrets management service and never enter the context.

7. Memory and Experience Accumulation

Memory is not “storing the chat history.” It is three stores with different natures:

• Working memory: the current task’s context and tool results; its lifecycle ends with the task.
• Task state: the plan, the ledger, completed and pending items, failure reasons. It needs to be persisted so work can resume after an interruption.
• Long-term memory: business rules, a glossary, typical solutions, known pitfalls. For retrieval, a mix of vector semantic search + exact keyword matching + structured filtering is recommended, so that pure semantic search does not miss proper nouns and identifiers.

One engineering point is often overlooked: long-term memory must be editable, invalidatable, and traceable. A wrongly written rule will contaminate all subsequent tasks for a long time, so it needs version numbers, a defined scope of effect, and a human review entry point.

8. Multi-Agent Collaboration: Split on Demand, Not by Default

Splitting produces a net positive only when a task has one of the following characteristics: sub-tasks use different tool sets, context requirements differ substantially, work needs to be parallelized, or genuinely independent verification perspectives are needed. Common division patterns include:

• Planner / executor: the planner produces the plan and acceptance criteria; the executor is only responsible for landing a single step.
• Generator / reviewer: the reviewer checks the result against an independent context and standard, avoiding self-endorsement.
• Explorer / summarizer: several solution branches are explored in parallel, and the summarizer converges them.

What to watch out for: multi-agent is not “stronger intelligence.” What it brings is greater context loss, higher latency and cost, and more complex failure attribution. A multi-agent setup without clear interfaces and a convergence mechanism is usually less stable than a single agent.

9. Evaluation: Turning “Capability” into Regressable Numbers

This is the piece most often missing in agent projects, and the one that most affects long-term iteration speed. At least four kinds of assets are recommended:

1. Offline task set: real business tasks, desensitized and frozen into cases with input, expected result, and acceptance criteria.
2. Trajectory-level metrics: task success rate, step distribution, tool call correctness rate, invalid retry rate, context overflow rate.
3. Process checks: whether the plan deviated, whether unnecessary high-impact operations were triggered, whether permissions were exceeded.
4. Regression gate: before a base model upgrade, a prompt change, or a tool change, run the offline set first.

An illustrative evaluation script:

async def evaluate(cases, agent):
    report = []
    for case in cases:
        result = await agent.run(case.input)
        report.append({
            "id": case.id,
            "success": case.verify(result),          # Prefer executable checks
            "steps": result.step_count,
            "tokens": result.tokens_used,
            "tool_errors": result.retryable_errors,
            "violations": result.policy_violations,  # Permission violations, high-risk operations, etc.
        })
    return summarize(report)  # Group by task type, rather than only looking at the overall mean

Two lessons from practice: first, a verifier beats human judgment. Anything that can be written as an assertion should not depend on a subjective score; if model-assisted scoring is unavoidable, fix the scoring rubric, calibrate it with humans, and state its limitations explicitly. Second, evaluation must be looked at per task type. An overall average hides structural regressions in a specific category.

10. Cloud Deployment: Using Generic Cloud Capabilities to Fill the Engineering Foundation

A large share of an agent system’s engineering complexity can be absorbed by generic cloud capabilities instead of being built in-house:

• Model serving and gateway: unified access, rate limiting, quota, routing, failure retries; makes switching smoothly during a base model upgrade easier.
• Container platform: hosts the execution sandbox and orchestration services, scales elastically on demand, controls idle-time cost.
• Object storage: holds traces, large result files, evaluation datasets, and model artifacts for replay and analysis.
• Message queue: decouples long-running tasks from interactive requests, supports task queuing, retries, and peak shaving.
• Logging service and monitoring/alerting: trace queries, metric dashboards, anomaly and cost threshold alerts.
• Secrets management service: centralized credential hosting and rotation, keeping credentials out of code and out of the context.
• Vector retrieval capability: supports semantic recall for long-term memory and the knowledge base.
• Batch compute capability: offline evaluation, data synthesis, and bulk preprocessing.

On cost and performance, several generic practices are fairly reliable: route models by task complexity (simple tasks to a lightweight model, complex reasoning to a stronger model); use semantic caching to reuse high-frequency Q&A results; use batch APIs for non-real-time evaluation and preprocessing; use a small model for initial screening and a large model for the final verdict. On Baidu AI Cloud, these capabilities are typically delivered as combinations of general-purpose cloud product categories (container platform, object storage, message queue, logging service, monitoring and alerting, secrets management, and so on). Service availability and exact forms vary by region, so refer to the official documentation for details.

11. A Gradual Path to Autonomy

Going for a “fully autonomous agent” in one shot is high risk. A staged progression is safer, with a clear acceptance bar at each level:

• L0 Assistance: the model gives suggestions, the human executes. The value is lowering the barrier to use.
• L1 Suggestive automation: the model produces directly adoptable artifacts (code, SQL, configuration), a human reviews them, and then they are executed.
• L2 Controlled autonomy: read-only or reversible operations run automatically; high-impact operations require confirmation.
• L3 Bounded autonomy: end-to-end automated execution within a clearly defined business domain, with full auditing and automatic rollback.
• L4 Open-domain autonomy: cross-domain autonomous planning, which requires very strong evaluation coverage and governance mechanisms and is usually suitable only for internal experiments.

Recommendation: only raise the autonomy level once the current level has stably met the bar on the offline evaluation set, and online incidents are attributable and rollback-able.

12. Common Anti-Patterns

• Unbounded autonomy: no step cap, no budget, no termination condition; cost scales out of control with task complexity.
• Tool sprawl: dozens of semantically overlapping tools, with a markedly higher rate of the model picking the wrong one. Better to merge them into a few highly cohesive tools.
• Stuffed context: cramming everything into the window, which dilutes key information and makes long tasks fail partway through.
• Self-acceptance: generation and verification share the same context and the same standard, which amounts to no verification at all.
• Demo-only focus: no offline evaluation set and no regression gate, so when the base model is upgraded you cannot tell whether things got better or worse.
• Excessive permissions: opening up production write access and outbound network access all at once for convenience, escalating an engineering problem into a security problem.
• Ungovernable memory: long-term memory that is write-only and never revised, so a wrong rule stays in effect for a long time and is hard to locate.

Conclusion

The real engineering question at this stage is not “which model to pick,” but how to build a layer of stable, observable, evaluable, and safely convergent agent infrastructure on top of a fast-upgrading base model. Base model capability will keep rising, and probably faster than expected. What will not go out of date no matter how the base changes are the quality of your tool contracts, the discipline of your context budgeting, the completeness of your evaluation system, and the strictness of your permissions and auditing.

Get those four things solid, and your system benefits automatically with every base model upgrade. Get them wrong, and even the best available base model only buys you a prototype with a nice demo that cannot be delivered to production.