AI Agents on Top of Pretrained Large Models: Core Architecture, Key Engineering Practices, and Cloud Infrastructure

3 hr ago0 views

This article explains why AI agents have made a strong comeback and what it takes to build them. It covers how fast large-model capabilities are advancing, why the current frontier should be treated as a floor rather than a ceiling, and how pretrained LLMs give agents a knowledge head start that earlier rule-based and reinforcement-learning approaches lacked. It then breaks down the core components of an agent (planning, memory, tool use, and action with feedback), with a ReAct-style Python example. Next, it discusses the engineering challenges of running agents in production, from context management and memory consistency to tool-call reliability, observability, and evaluation, along with the cloud infrastructure that supports them. Readers will gain a clearer picture of where base model capabilities end, where engineering work begins, and what it takes to build reliable agents.

The Capability Horizon Is Rising Fast

Large-model technology is evolving faster than most people expect. Capabilities once considered industry benchmarks can now be table stakes within months. When expectations are reset every quarter, any technical judgment based on a snapshot can quickly become outdated.

One point bears repeating: the frontier model capabilities you use today are more of a floor than a ceiling. They represent the baseline you can reliably rely on right now, not the limit of what the technology can achieve. This is key to understanding where upper-layer applications such as agents are headed.

Why Agents Are Back

Agents are not a new concept. Early approaches relied mainly on symbolic reasoning and rule systems, or trained reinforcement learning models from scratch. They all shared the same weaknesses: knowledge was costly to acquire, generalization was limited, and agents failed quickly once a task fell outside preset boundaries.

What sets the current wave of agents apart is where they start. Rather than building knowledge and reasoning capabilities from zero, agents now stand on the shoulders of pretrained large models. Base models have already learned language, common sense, and even complex reasoning patterns from vast amounts of training data. An agent that layers planning, memory, and tool use on top can deliver far better task performance than previous generations of agents.

Pretrained LLMs: An Agent’s Innate Intelligence

You can think of a pretrained large model as the source of an agent’s “innate intelligence.” During pretraining, the model has already “read” a huge amount of world knowledge. This prior knowledge means it doesn’t need to explore from scratch when facing a new task. Instead, it can quickly draw on existing cognitive patterns to reason and make decisions.

Consider an analogy: if an agent that has to learn strategies in a complex, adversarial environment starts with many proven strategic paradigms already stored in its “memory,” it will make far better decisions from the outset than one starting from a blank slate. Pretraining gives agents this kind of knowledge advantage: they start from a high baseline. This also explains why today’s agents can quickly reach a usable level at tasks such as coding, exploring unfamiliar environments, and handling complex interactions. Their foundational intelligence comes from the base model.

Core Agent Architecture

Setting specific products aside, most mainstream agent systems follow a similar loop: perceive, plan, act, and receive feedback. Their core components can be broken down as follows:

Planning: responsible for task decomposition and decision-making. It breaks high-level goals into executable sub-steps and is what sets agents apart from ordinary conversational models.
Memory: consists of short-term contextual memory and long-term knowledge memory. Short-term memory relies on the context window, while long-term memory often relies on external vector retrieval.
Tool use: the bridge between an agent and the external world. Structured calls to tools such as search, databases, and code execution environments let an agent go beyond the inherent limits of a single model.
Action & feedback: carries out actions and observes the results, then feeds those results into the next round of decision-making for continuous improvement.

A widely used pattern is to alternate between reasoning and acting (often called the ReAct approach). The model first reasons about the current state, then decides on an action. After observing the result, it moves on to the next round of reasoning. The code below illustrates this pattern:

def agent_loop(goal, tools, model, max_steps=10):
    history = [f"Task goal: {goal}"]
    for _ in range(max_steps):
        # 1. Have the base model reason and decide based on the current history
        decision = model.generate(prompt=history, tools=tools)
        if decision.is_final:
            return decision.answer
        # 2. Execute the selected tool action
        observation = tools[decision.tool](decision.arguments)
        # 3. Write the action and observation back into the history for the next round
        history.append(f"Thought: {decision.thought}")
        history.append(f"Action: {decision.tool}({decision.arguments})")
        history.append(f"Observation: {observation}")
    return "Max steps reached; returning the best result so far"

This code is a generic example of how an agent runs. It is not tied to the APIs of any particular platform.

Key Engineering Considerations for Production

Moving a working agent into production usually comes down to engineering details rather than the model itself:

Context management: context windows are limited, so long-running tasks need a sound strategy for compressing and retrieving information. Otherwise, important details get buried or early information is lost.
Memory consistency: long-term memory must handle writes, updates, and deduplication. Without these, incorrect knowledge builds up and pollutes later decisions.
Tool-call reliability: external tools may return unstable results, so you need to handle exceptions, timeouts, and retries.
Observability: an agent’s decision chain can be long, so log the reasoning, actions, and observations at every step. This makes it possible to trace and troubleshoot problems.
Evaluation: no single metric can capture how an agent performs. Build an evaluation set that reflects real-world scenarios and covers task success rate, step efficiency, safety, and other dimensions.

Cloud Infrastructure for Production Agents

Taking an agent from prototype to large-scale service typically takes a complete set of cloud infrastructure. Below are only common industry capability categories:

Object storage: stores training data, model files, and runtime logs
Message queue: decouples asynchronous tasks and supports scheduling of long-running jobs
Container platform: provides elastic scaling and isolated runtime environments
Log service with monitoring and alerting: supports observability across the inference pipeline

Baidu AI Cloud, for example, offers object storage, messaging services, containers, and log monitoring. These common capabilities can serve as the underlying foundation for productionizing agents, so developers can focus on business logic instead of building and maintaining infrastructure.

Conclusion

The shift from pretrained large models to agents is fundamentally a leap in the form of capability. The model supplies the “innate intelligence,” while the agent adds planning, memory, and action. Together, they turn static knowledge into executable behavior. Understanding this relationship explains why building agents today is not reinventing the wheel. Instead, it extends the value of a mature foundation. For developers, the core path to reliable agents lies in understanding the boundaries of base model capabilities and pairing them with solid engineering practices.