Introduction: The “Last Mile” of Putting Agents into Production

Over the past two years, the capability boundaries of large language models have kept expanding: longer context windows, stronger code generation and tool calling, faster inference. Models that could only chat are gradually evolving into agents that can actually get work done. Yet when it comes to running an agent in a real production environment, the hard part is usually not the model itself, but the runtime framework around it—commonly referred to in the industry as the harness. It handles tool calling, state management, sandboxed execution, failure recovery, task orchestration, and a long list of other unglamorous but essential jobs.

A new trend is taking shape: platform providers are packaging this harness directly as a managed cloud service, known as hosted agents. Enterprises no longer need to build and maintain their own agent runtime frameworks; instead, they can consume a long-running agent execution environment on demand, just like any other managed cloud service. This article examines this emerging form from four angles: technical composition, core capabilities, business logic, and adoption scenarios.

What Are Hosted Agents? Turning the Harness into a Cloud Service

An Agent = Model + Runtime Framework

A working agent is typically made up of two parts:

• The model: understands goals, reasons and makes decisions, and generates actions;
• The runtime framework (harness): turns the model’s “thoughts” into “actions,” covering tool registration and invocation, context and memory management, sandboxed execution of code and commands, loop control for multi-step tasks, retries on failure, and result validation.

However powerful a model is, without a harness it is just a Q&A interface. However complete a harness is, without a strong model driving it, it can only execute rule-based workflows. The two are inseparable.

Defining the Hosted Agent

A hosted agent is a product form in which the platform provider wraps the agent runtime framework as a cloud service for enterprises and developers. Its core characteristics include:

• No self-building: enterprises do not need to develop, deploy, or maintain the runtime framework themselves;
• Managed operation: the platform provides a 24/7 execution environment and takes care of process startup, health checks, failure recovery, and resource scheduling;
• On-demand configuration: enterprises define an agent’s goals, available tools, permission boundaries, and resource quotas through declarative configuration, then go live quickly.

A useful analogy: a model API is like an engine, while a hosted agent is like a complete car with a chauffeur service—what enterprises receive is a digital worker they can assign tasks to right away, not a pile of parts.

Why Hosting? The Four Costs of Building Your Own Runtime Framework

Before the hosted model emerged, enterprises that wanted to run agents in production generally had to build the entire framework themselves—and in practice this exposes four categories of high cost.

High Engineering Complexity

A production-grade agent runtime framework must at minimum handle: authentication and timeout control for tool calls, isolation of concurrent requests, interruption and resumption of streaming output, retries and idempotency for failed model calls, and file system and network isolation for execution sandboxes. Each of these engineering details is manageable on its own, but together their complexity can easily consume months of engineering effort from a small team.

Reliability of Long-Running Tasks Is Hard to Guarantee

Typical agent tasks are long-running by nature: a deep research task may involve dozens of tool calls and run for hours; an automated operations inspection may repeat on a daily cycle. Long-running execution means all kinds of exceptions can occur—process crashes, context overflow, flaky external tools, mid-task human intervention. A self-built framework has to implement state persistence, checkpointing, and resumability from scratch, which is among the hardest things to get right in distributed systems.

Heavy Operational Burden

24/7 availability, monitoring and alerting, log tracing, version upgrades, security patches—in a self-built setup, all of these operational responsibilities fall on the enterprise itself. For most business teams, maintaining an agent runtime framework is not a core competency.

Resource Utilization Is Hard to Optimize

Agent workloads fluctuate naturally: tasks demand burst compute when triggered, while resources sit nearly idle in between. Self-built clusters either run with long-term redundancy and waste capacity, or get overloaded at peaks and miss task deadlines. Hosted services, backed by the elastic scheduling of cloud platforms, naturally fit this load profile.

Core Technical Elements of a Hosted Agent Service

A mature hosted agent service is typically built around four categories of capability.

Managed Runtime Environment

The platform provisions an isolated execution sandbox for each agent in the cloud, with its own file system, code execution environment, and controlled network access. The sandbox enforces permission isolation and resource quotas, and combined with lifecycle management—automatic startup, health checks, and failover—it keeps agents reliably online over the long term.

Long-Running Task Execution

• State persistence and checkpointing: task state is externalized to durable storage, so a crash at any moment resumes from the most recent checkpoint instead of starting over;
• Asynchronous task model: submitting a task immediately returns a task handle; execution happens in the background, and callers retrieve progress and results through polling or callbacks;
• Context management: for very long tasks, techniques such as context compression and layered memory direct the limited context window toward the most critical information.

Multi-Agent Orchestration

Complex tasks often exceed what a single agent can handle, so hosted platforms typically provide multi-agent orchestration:

• Role division: a planner decomposes tasks, executors carry out specific operations, and a reviewer validates results;
• Communication mechanisms: agents collaborate through message passing, shared memory, and task queues, preventing one agent’s context from polluting another’s;
• Result aggregation: subtask results are merged, conflicts resolved, and a final report generated.

Observability and Governance

Agents are systems that act autonomously, which makes governance even more important than observability for traditional services:

• End-to-end tracing: every model call, tool call, and intermediate artifact can be recorded and replayed for debugging and auditing;
• Permissions and approvals: tool calls follow least-privilege authorization, and high-risk operations support human approval or dual review;
• Cost metering: transparent metering by task or token makes each agent’s return on investment accountable.

The Business Logic: From Selling Inference APIs to Selling Agents

The rise of hosted agents reflects more than a change in technical form—it signals a shift in the delivery unit of the large model industry.

Model APIs deliver stateless, short-lived inference capability: what enterprises get is an atomic service where each call returns a piece of text. This delivery model has a low integration barrier but limited value density and slides easily into homogeneous competition.

Hosted agents deliver a digital worker that gets things done: enterprises are no longer buying tokens but business outcomes—“inspection reports delivered on schedule” or “customer issues resolved the same day.” When the delivery unit shifts from capability to outcome, three things change:

• For enterprises: one-time fixed R&D costs become operating costs that scale with usage, significantly lowering the barrier to experimentation;
• For platforms: billing moves from per-token to per-task or subscription models, raising both deal size and customer stickiness;
• For the industry: this mirrors the historical pattern of cloud computing—infrastructure keeps moving up the stack, from IaaS and PaaS to Agent-as-a-Service.

It is worth noting that this path only works if underlying model capabilities keep improving. Recent advances in long context, code generation, and tool calling, along with inference variants tuned for different latency requirements, are what make long-running, multi-step, reliable agent execution possible in the first place. Hosted agents are, at their core, a way of converting model capability gains into productivity that enterprises can consume directly, through engineering and packaging.

Two Typical Use Cases and How to Land Them

Internal Systems: Monitoring and Ops Response

This is the lowest-risk, fastest-payoff entry point. The typical approach connects an agent to internal systems to handle two kinds of work:

• Recurring tasks: scheduled inspections of logs and metrics, summarizing anomalies, and generating on-call reports;
• Event-driven tasks: when an alert arrives, automatically executing the playbook—checking the status of related services, pulling error logs, attempting predefined recovery actions, and recording the whole process as a ticket.

Implementation points:

• Least-privilege tools: grant the agent only read-only queries and low-risk operations; route write operations through approval;
• Integrate with existing systems: connect to the enterprise’s existing monitoring and alerting, ticketing, and messaging systems rather than building parallel ones;
• Human in the loop: keep human confirmation in the early stages and gradually expand the automation radius as trust builds.

Customer-Facing Products: In-App Assistants

The second scenario embeds agent capabilities into products offered to external users, turning them into end-user-facing features—for example, an in-app research assistant (information aggregation, data analysis, report drafting) or a customer service assistant (multi-turn Q&A, issue triage, ticket routing).

Implementation points:

• Latency control: conversational scenarios are sensitive to response time, so consider tiered “dual-channel” handling—instant Q&A runs through a fast inference channel, while deep reports run through an asynchronous long-task channel;
• Traceable results: outputs involving data and conclusions should carry source citations to reduce the business risk of hallucination;
• Content safety and compliance: outputs shown to external users must pass content safety filtering, with audit logs retained.

The following illustrative pseudocode shows a typical pattern for enterprises adopting hosted agents (conceptual illustration only, not a real interface):

# Illustrative code: define and trigger a hosted agent task (pseudocode)
agent = hosted_agent.create(
    name="ops-inspector",
    model="platform-provided-model",       # Model service provided by the platform
    tools=["log_query", "metric_query", "runbook_executor"],
    permissions={"write": "require_approval"},  # Write operations require human approval
    checkpoint="enabled"                   # Enable checkpoint-based recovery
)
task = agent.submit(
    goal="Inspect production services and report anomalies with remediation suggestions",
    schedule="0 * * * *",                  # Run once every hour
    timeout="30m"
)
report = task.wait()                       # Asynchronous task: poll or use callbacks

# Illustrative code: task decomposition for multi-agent collaboration (pseudocode)
planner = hosted_agent.get("planner")
worker  = hosted_agent.get("executor")
shared  = hosted_agent.shared_memory("research-room")
for subtask in planner.decompose(user_goal):
    result = worker.run(subtask, context=shared)
    shared.write(subtask.id, result)
final = planner.summarize(shared.read_all())

Evaluation and Adoption Recommendations

For technical teams planning to adopt hosted agents, evaluate candidates along these dimensions:

• Isolation and security: the strength of sandbox isolation, the granularity of tool-call authorization, and whether data usage boundaries are clearly defined;
• Long-running reliability: whether checkpointing and resumability are supported, the maximum task duration, and how robust the retry strategy is;
• Orchestration flexibility: whether multi-agent collaboration supports custom roles and communication topologies;
• Observability: whether tracing covers every single call, and whether logs can be exported into the enterprise’s own systems;
• Billing transparency: whether the metering model is clear and whether costs can be attributed per task.

For rollout, a sensible principle is “governance first, inside out”: pilot in low-risk internal scenarios, get tool integration, approval workflows, and audit trails working end to end, then gradually extend to scenarios facing external users. Major cloud providers—including Baidu AI Cloud—are currently building out model services and agent development products; teams can compare options against their own tech stack and compliance requirements and choose the integration approach that best fits their business.

Conclusion

Hosted agents mark a key step for the large model industry as it moves from supplying capabilities to delivering outcomes: platforms take the tedious runtime framework into the cloud, freeing enterprises to focus on the business itself. For technical teams, the real question is no longer “should we use agents?” but “which parts of our workflow should be delegated to hosted agents, and which must remain under human control or self-built systems?” Drawing that boundary clearly is what allows teams to capture the efficiency gains of this infrastructure shift while maintaining security and compliance.