Hosted Agent Services: A New Paradigm for Enterprise Agent Deployment

The Industry Turning Point: From Model API Calls to Purchasing Agents

Over the past two years, the dominant way for enterprises to adopt large language models has been through inference APIs: send a prompt, receive a result. This approach solved the problem of “having the capability” but not the problem of “getting the job done.” As agent technology matures, the commercial focus of model and cloud services is undergoing a critical shift—from “selling model inference APIs” to “selling agents.” What customers purchase is no longer a single inference call but a service unit capable of continuously completing tasks.

The underlying value logic is straightforward: a model API sells a single capability, while an agent service delivers sustained outcomes. To reliably deliver sustained outcomes, a conversational model alone is insufficient. What is needed is an engineering framework that makes the model “run, run for a long time, and run reliably.” In the industry, this framework is commonly referred to as the Harness (runtime framework). Turning the Harness from an enterprise’s self-built burden into a cloud-delivered service is precisely the question that hosted agents aim to answer.

What Is a Hosted Agent: Turning the Harness into a Cloud Service

An agent that works reliably in a production environment depends on at least six categories of engineering components, beyond the model itself:

1. Task Planning and Orchestration: Decomposing user goals into executable steps and sequencing them based on dependencies.
2. Tool Invocation and Integration: Safe, controlled access to enterprise internal and external systems, data sources, and APIs.
3. State and Memory Management: Maintaining task context, intermediate results, and domain knowledge across turns and sessions.
4. Exception Handling and Recovery: Retry on failure, resume from breakpoints, graceful degradation on timeout, and prevention of unauthorized operations.
5. Observability: Logs, trace chains, metrics, and alerts that make every agent action auditable and replayable.
6. Resource Isolation and Permission Control: Multi-tenant, multi-task non-interference, with guardrails for sensitive operations.

The cost of building this Harness in-house is extremely high. The core idea behind hosted agents is to deliver the entire framework as a cloud service. Enterprises only need to configure the agent’s goals, tools, and policies as needed. Running, scaling, monitoring, and recovery are all handled by the platform side. The enterprise gains a 7×24 managed environment without having to build and maintain the runtime framework itself.

The differences between self-built and hosted solutions can be summarized as follows:

• Self-Built Approach: Maximum control and deep customization, but requires self-developed orchestration engines, state persistence, and observability systems. The team must include backend engineers, SREs, and algorithm engineers, and going from project initiation to production often takes quarters.
• Hosted Approach: Out-of-the-box and pay-as-you-go, exchanging “customization freedom within the platform’s capability boundary” for extremely low startup costs. For most enterprises looking to quickly validate business value, hosting is the lowest-cost path for experimentation.

Two Technical Touchstones: Long-Running Tasks and Multi-Agent Collaboration

1. Long-Running Tasks: From One-Shot Q&A to Continuous Operations

Early agent demonstrations mostly involved short tasks completable within minutes. However, real enterprise value often lies in long-running tasks. A complete online incident investigation may require hours of log retrieval and hypothesis validation. A deep industry research report may take days of data collection, cross-verification, and iterative revision. Long-running tasks impose three hard requirements on the runtime:

• State Persistence: After a process restart or platform maintenance, the task should resume from the breakpoint rather than start from scratch.
• Layered Memory: Short-term memory carries the full context of the current task, while long-term memory accumulates domain knowledge and operational preferences that can be reused across tasks.
• Budget and Circuit Breaking: Resource consumption such as tokens, time, and call counts must be controllable. In abnormal situations, the system should degrade safely or escalate to human intervention.

This is precisely the hardest part of self-built solutions. It is essentially a combination of a distributed workflow engine and a memory system, far more complex than “a model with a loop.”

2. Multi-Agent Collaboration: From Solo Operation to Coordinated Division of Labor

When task complexity exceeds the capability boundary of a single agent, multi-agent collaboration becomes necessary. Common orchestration patterns in the industry include:

• Orchestrator-Worker: A coordinator handles task decomposition and distribution, multiple workers operate in parallel, and the coordinator aggregates and validates results.
• Pipeline: Multiple agents work in sequence, such as “data retrieval → analysis → report writing → quality review.”
• Debate/Review: Multiple agents independently solve the same problem and cross-examine each other, using an arbitration mechanism to improve reliability.

For collaboration to be truly efficient, the platform must provide infrastructure for task distribution, message communication, shared context (the “blackboard” mechanism), result arbitration, and conflict resolution. These are high-barrier, repetitive constructions. Platform-hosted delivery is clearly more economical than having every enterprise rebuild these wheels.

Two Typical Deployment Scenarios: Internal Digital Employees and External Product Assistants

Based on industry practice, enterprises can configure agents on demand and quickly build two types of business applications.

Scenario One: Internal Business—Monitoring and Operations Response

Agents integrated into internal enterprise systems can handle business monitoring and operations response tasks. A typical operations response agent workflow is as follows:

1. Continuous Monitoring: Subscribe to core metrics (service error rate, API latency, resource levels, etc.) and key event streams.
2. Automated Root Cause Analysis: Upon anomaly detection, automatically aggregate relevant logs, recent deployment records, and configuration changes to form initial hypotheses.
3. Tiered Handling: Automatically execute low-risk operations such as scaling or service restarts according to playbooks. For high-risk operations such as traffic switching or data correction, generate recommendations and escalate for human approval.
4. Post-Mortem and Knowledge Accumulation: Archive the handling process and conclusions, enriching a searchable and reusable repository of operational experience.

A declarative configuration for such an agent might look like the following (illustrative; refer to the selected platform’s documentation for exact fields):

agent:
  name: ops-response-agent
  mode: hosted                # Runtime managed by the platform
  availability: 7x24
  tools:
    - metrics_query           # Metrics query (read-only)
    - log_search              # Log search (read-only)
    - ticket_system           # Ticket system (read/write)
    - notification            # On-call notification
  memory:
    persistence: enabled      # Breakpoint recovery for long-running tasks
  policy:
    high_risk_actions: human_approval   # High-risk actions require human approval
    daily_token_budget: 500k            # Daily token budget cap

Scenario Two: For Product Users—In-App Investment Research Assistants and Customer Service Assistants

Another class of applications embeds agents into externally facing products, such as in-app investment research assistants or customer service assistants. These scenarios are characterized by high concurrency, unpredictable request patterns, and direct exposure to end users. Therefore, platform requirements for elastic scaling, content safety, and compliance are more stringent.

Integration is typically lightweight. For example, an asynchronous long-running task can be initiated as follows (illustrative):

POST https://<agent-platform-endpoint>/v1/agents/research/runs
Authorization: Bearer <API_KEY>
Content-Type: application/json
{
  "input": "Summarize the policy changes in the new energy vehicle sector over the last quarter and output a structured summary",
  "mode": "async",                          // Long-running tasks are recommended to run asynchronously
  "callback_url": "https://<your-callback-endpoint>/agent-callback"
}
→ Response example: { "run_id": "run_9f2c", "status": "running" }

The business system then retrieves the execution result via callback or polling. At this point, the developer’s focus shifts from “maintaining the runtime environment” to “defining the assistant’s behavior and boundaries”—setting knowledge sources and retrieval scope, tone of voice, refusal strategies, sensitive topic fallbacks, and rules for transferring to human agents.

Foundation Model Evolution: The “Groundwork” for Agent Experience

The feasibility of hosted agents today is mutually reinforcing with the capability leaps in underlying large models. Current mainstream models exhibit three clear trends:

1. Continuing Growth in Parameter Scale and Long-Horizon Capability. Leading models have entered the trillion-parameter scale, with significantly enhanced performance on long-horizon tasks such as coding, 3D content creation, and financial research. The probability of an agent “getting the plan right on the first attempt” has improved substantially, reducing the cost of trial and error.
2. Million-Token Context Windows Are Becoming Standard. Long context allows an agent to load an entire codebase, lengthy documents, and historical sessions into a single task, improving both planning and retrieval quality. This is especially critical for long-running agents.
3. Parallel Availability of High-Speed Inference Versions. The same model often offers high-speed variants that deliver roughly 2–3× faster output while maintaining equivalent intelligence, suited for latency-sensitive scenarios such as customer service conversations and streaming output. Deep reasoning is handled by the standard version, creating a service matrix that separates speed from depth.

These model-layer advances are inherited by enterprises “transparently” through the hosted platform. When the platform upgrades, the business side benefits simultaneously—another hidden advantage of the hosted model over self-built solutions.

Enterprise Evaluation and Adoption Path

If an enterprise is considering adopting hosted agent services, it is recommended to evaluate along five dimensions:

1. Capability Fit: Does the platform natively support long-running task persistence, multi-agent orchestration, tool ecosystems, and integration with enterprise systems?
2. Hosting Completeness: Are runtime environments, monitoring and alerts, log auditing, and elastic scaling truly “operations-free,” or does the enterprise still need to provide its own compute and on-call staffing?
3. Security and Compliance: Multi-tenant data isolation, tool permission models, audit log retention, and compliance qualifications for data residency.
4. Cost Model: Is billing based on tasks, duration, or tokens? Do long-running tasks have controls such as budget caps and circuit breakers?
5. Portability: Is the agent definition based on open formats and standard protocols, avoiding deep vendor lock-in that would make migration difficult?

A recommended adoption path follows three steps: first, select a high-value internal scenario with clear boundaries (such as operations monitoring or draft report generation) as a pilot. After reliability is validated, embed the agent into external products. Finally, establish an enterprise-level agent asset library and replicate it across more business lines. Currently, major cloud vendors—including Baidu AI Cloud—have launched agent development and hosting products. Enterprises can evaluate options based on their existing cloud technology stack and compliance requirements.

It is also important to emphasize: hosting does not mean hands-off. Enterprises should retain clear human-in-the-loop approval gates, minimal tool permission boundaries, and regular audit mechanisms, especially for high-risk operations and public-facing service scenarios.

Conclusion

The shift from “selling model inference APIs” to “selling agents” is essentially a migration of the industry’s value anchor from “capability” to “outcomes.” Hosted agent services are the engineering vehicle for this migration: they turn the hardest-to-build and hardest-to-maintain Harness layer into a subscribable cloud service, like electricity or water, allowing enterprises to redirect engineering effort from “keeping things running” to “defining the business.”

As trillion-parameter, million-context models continue to evolve and high-speed inference versions proliferate, “agent-as-a-service” is poised to become the default option in enterprise AI infrastructure. For developers and architects, understanding the runtime logic and selection criteria of this new paradigm is one of the most worthwhile technical investments in the coming year or two.