Always-On AI Agents: Architecture Design and Security Boundaries

1. Background and Problem Definition: From One-Off Q&A to Proactive Execution

As generative AI moves from a conversational tool to a workflow execution engine, a new form of agent is taking shape: always-on, proactive, and capable of end-to-end execution. It works differently from the “user asks a question, model gives an answer” pattern. Instead, it is driven by the user’s goals: it continuously tracks information in the background, decides whether action is needed, and proactively delivers results or recommendations to the user.

Several major model vendors have recently made “runs in the background, pushes proactively” a core selling point of their agents, and are already applying it to serve small teams and professionals. Products in this category are typically driven by a flagship reasoning model and run in an isolated environment with its own browsing session. They reuse personalized settings and connected apps from the user’s account, connect to a large number of third-party services through a connector ecosystem, and span mobile, desktop, and web.

From an engineering perspective, this kind of agent has to solve four problems at once:

1. Proactivity vs. controllability: the more the agent acts on its own, the clearer its boundaries must be; a single misjudgment can cause irreversible damage.
2. Personalization vs. privacy: to genuinely help, the agent has to accumulate user data, but the more concentrated that data becomes, the more severe the consequences of a leak or misuse.
3. Long-running tasks vs. resource cost: continuous background operation means continuous inference and API costs, which requires fine-grained budgeting and scheduling.
4. Open capabilities vs. security: the more connectors there are, the more the agent can do, and the larger the attack surface it can be exploited through.

This article first clarifies the paradigm shift, then presents a deployable seven-layer reference architecture. It then breaks down proactive decision-making, permission tiering, memory management, sandbox isolation, reliable execution, and evaluation and monitoring one by one, and closes with a phased rollout roadmap.

1.1 Comparing the Two Paradigms

Dimension	Conversational Assistant	Always-On Agent
Trigger	User-initiated, turn by turn	System-initiated: scheduled, event-driven, or condition-triggered
State lifecycle	Mostly limited to a single session	Task state, user preferences, and historical context retained across sessions
Working mode	Passive responses, one request per answer	Continuously tracks goals, executes across multiple steps and long time spans
Information flow	User asks, model answers	System detects changes first, then decides whether and how to act
Typical output	Text, code, or suggestions	Verifiable deliverables such as briefings, reports, and dashboards
Risk profile	Mainly incorrect content, with a controllable impact	Involves writes to real systems and external communications, with irreversible risk

This table reveals a core rule: every step up in an agent’s autonomy raises the requirements for permission governance, observability, and failure recovery by an order of magnitude. The design focus of an always-on agent is therefore not “what can it do” but “under what conditions it acts, how far it goes, and how to detect and recover when something goes wrong.”

2. Reference Architecture: Seven Layers

An always-on agent can be modeled as seven layers, with three cross-cutting concerns running vertically through the entire stack: identity and credential management, policy and audit, and cost and quota.

+-----------------------------------------------------------------+
|                       (7) Interaction & Delivery                |
|   Chat UI - mobile / desktop / web - approval center - notifications|
+-----------------------------------------------------------------+
|                     (6) Scheduling & Execution                  |
|   Scheduled tasks - event subscriptions - workflow orchestration  |
|   - retries / rate limits / idempotency                         |
+-----------------------------------------------------------------+
|                        (5) Agent Reasoning                      |
|   Task planning - tool selection - proactivity decision           |
|   - context assembly                                            |
+-----------------------------------------------------------------+
|                     (4) Tools & Connectors                      |
|   Connector catalog - unified tool protocol - credential          |
|   management - quota governance                                 |
+-----------------------------------------------------------------+
|                     (3) Memory & Personalization                |
|   User profile - preferences & rules - task state - long-term     |
|   memory store                                                  |
+-----------------------------------------------------------------+
|                        (2) Execution Sandbox                    |
|   Isolated browsing session - isolated container - file workspace |
|   - outbound network control                                    |
+-----------------------------------------------------------------+
|                   (1) Data Ingestion & Perception               |
|   Connected apps - calendar / email / documents - business data   |
|   sources - public web pages                                    |
+-----------------------------------------------------------------+
  Cross-cutting: identity & credentials | policy engine & audit logs |
                 cost control & observability

Layering principle: each layer exposes only stable, minimal interfaces. The reasoning layer never holds credentials directly; it issues “capability requests” to the tools layer. The tools layer executes or rejects each request according to the policy engine’s ruling and writes every call to the audit log. This concentrates the security boundary in just two places, the tools layer and the policy engine, instead of scattering it across model prompts.

3. Breaking Down the Key Capabilities

3.1 Data Ingestion and Perception: What the Agent Can See

An always-on agent’s capability ceiling is largely determined by the range of information it can perceive. Common data sources include:

• Personal work data: calendars, email, to-do lists, and cloud documents and spreadsheets.
• Business data: orders, inventory, customers, financials, and operational metrics.
• Public external data: web pages, public reports, and market news.

Engineering teams should follow three principles:

1. Combine pull and push: periodic full-data pulls are expensive and stale, so event-driven push should be the primary mechanism, with scheduled reconciliation as a fallback.
2. Prioritize incremental sync: process only what has changed in each sync, and use data version numbers or change timestamps to avoid duplicate processing.
3. Tag the source: every piece of data entering the context should carry source, timestamp, and trust-level tags so the basis for the model’s conclusions can be traced later.

3.2 Tools and Connectors: From “Can Connect” to “Safe to Connect”

The connector ecosystem is what sets these agents apart from traditional chatbots, and it is also where risk is most concentrated.

A mature connector governance model includes at least the following elements:

Element	Description	Typical approach
Capability declaration	Specifies which operations a connector supports and whether they read or write	Metadata registration, e.g., “read calendar,” “create event,” “send email”
Authorization model	Obtains access tokens via a standard authorization-code flow	Requests least-privilege scopes per operation and lets users revoke access at any time
Credential storage	Tokens and keys must never enter model context or logs	Centralized management in a secrets manager; the server substitutes credentials at call time
Quota and rate limiting	Prevents high-frequency calls from overloading third-party APIs or generating large bills	Token-bucket rate limiting per user and per connector
Fault isolation	A single connector failure must not bring down the whole task	Circuit breaking, graceful degradation, and queued retries

To let the model call heterogeneous tools in a uniform way, the industry commonly introduces a “tool description protocol.” Each tool declares its name, purpose, parameter schema, and permission level in a structured format, and the runtime parses and registers them centrally. Adding a connector then means adding one declaration, with no change to the reasoning logic. Example tool declaration:

{
  "name": "calendar.create_event",
  "description": "Create an event in the user's calendar",
  "risk_level": "L1",
  "reversible": true,
  "parameters": {
    "type": "object",
    "properties": {
      "title": { "type": "string" },
      "start_time": { "type": "string", "format": "date-time" },
      "end_time": { "type": "string", "format": "date-time" },
      "attendees": { "type": "array", "items": { "type": "string" } }
    },
    "required": ["title", "start_time", "end_time"]
  },
  "policy": {
    "requires_approval": "if_attendee_count_gt_5",
    "max_per_day": 20
  }
}

One point deserves emphasis: description is an “instruction manual” written for the model, and it must never be treated as a security guarantee. A malicious or careless third party can easily declare “this operation has no side effects” to obtain higher privileges. Real constraints must be enforced by the runtime based on the connector’s actual capabilities, following the principle of “deny by default, allow explicitly.”

3.3 Agent Reasoning: Model Tiering and Structured Planning

An always-on agent reasons frequently, so calling the most expensive model at every step drives costs out of control quickly. The common engineering approach is a tiered strategy:

• Lightweight models handle intent classification, data summarization, format conversion, and repetitive checks.
• Flagship models handle only critical steps such as complex planning, cross-tool reasoning, and result verification.

The reasoning process should output a structured plan rather than free-form text, which makes validation and execution easier. Example:

{
  "goal": "Summarize this week's business data and flag anomalies",
  "steps": [
    { "id": 1, "tool": "data_source.query_sales",   "risk": "L0", "read_only": true },
    { "id": 2, "tool": "data_source.query_expense", "risk": "L0", "read_only": true },
    { "id": 3, "tool": "report.render_chart",       "risk": "L0", "read_only": true },
    { "id": 4, "tool": "notify.push_digest",        "risk": "L0", "read_only": true },
    { "id": 5, "tool": "chat.schedule_meeting",     "risk": "L2", "requires_approval": true }
  ],
  "stop_conditions": ["sales decline MoM exceeds threshold", "any data source unavailable"],
  "budget": { "max_steps": 15, "max_tokens": 120000, "max_cost": 0.05 }
}

Before execution, the execution engine runs a plan pre-check:

• When it encounters a step with requires_approval: true, it pauses and requests human approval.
• When it hits a budget limit, it stops early and reports progress.
• When it hits a stop condition, it immediately halts the remaining steps to prevent one data anomaly from triggering a chain of operations.

3.4 Execution Sandbox: Give the Agent Hands Without Letting It Cross the Line

To actually get work done, an agent needs an environment it can operate in. The common approach is to give it a dedicated browser VM or an isolated container with its own session, cookies, and cache that are not shared with the user’s login state. It also gets its own working directory for generating documents, spreadsheets, and reports, and when it accesses the internet, traffic goes through a unified egress proxy with allowlist controls.

Key sandbox design points:

1. Least-privilege identity: when accessing third-party services, the sandbox should preferably use a dedicated service account with minimal permissions rather than reusing the user’s primary account.
2. Egress control: domain allowlisting combined with outbound auditing blocks channels that could exfiltrate data to unknown addresses.
3. Encryption at rest: when artifacts are written to persistent storage such as object storage, enable server-side encryption and scope access by task.
4. Lifecycle management: destroy the session and temporary files when the task ends or times out, so no sensitive data is left behind.
5. Human takeover: for actions only a human can complete, such as login, two-factor verification, or a CAPTCHA, use a “takeover session” approach: the user temporarily takes over, then hands control back to the agent. Credentials are never persisted or logged.

3.5 Memory and Personalization: The More It Is Used, the Better It Knows You

What lets an always-on agent keep serving you is memory that spans sessions. The design should distinguish four types of memory:

Type	Content	Lifecycle	Example
Session memory	Intermediate state of the current task	Per task	Data already queried, drafts already generated
Working memory	Context of the task being processed	Hours	Which metrics the current briefing uses
Long-term memory	User preferences, habits, fixed rules	Persistent	“Send the weekly report at 9:00 a.m. Monday”; “Always round amounts to two decimals”
Knowledge memory	Documents and skills the user has deliberately built up	Persistent	Custom report templates, data-processing scripts

The hard part of memory management is what to write, when to write, and how to correct errors:

• Write filtering: persist only high-value, stable information, such as explicitly stated preferences and recurring task patterns, and avoid storing one-off context long-term.
• Conflict resolution: when a new preference conflicts with an old one, the newer one wins, and a version history is kept for traceability.
• Viewable, editable, deletable: users should be able to view, modify, and delete any memory item, and deletions should propagate to all derived tasks.
• Contamination prevention: external web pages and email content can carry malicious instructions (prompt injection). Memory writes must pass source verification and human confirmation; external data must never be able to “write to the database by itself.”

3.6 Interaction and Delivery: The Right Amount of Proactive Push

Proactive push is the most visible value of these products and the design element most likely to annoy users. A tiered reach principle is recommended:

• L0 silent: routine data updates with no anomalies are shown only when the user opens the app.
• L1 digest: a briefing is pushed at a fixed time daily or weekly, covering key metrics, a summary of changes, and suggested to-dos.
• L2 alert: a notification is pushed when an anomaly appears or a deadline approaches.
• L3 request: when a user decision or authorization is needed, an approval card that can be acted on in one click is pushed.

Teams should also provide push-frequency controls, do-not-disturb hours, and a “minimal interruption” mode so the agent does not become a new source of information noise.

4. Deciding When to Act

Proactivity does not mean “push more often.” It means responding to the user’s needs at the right time, with the right action, at the right scale. Each candidate action can be scored on four dimensions:

• Value: the actual benefit the action delivers, such as avoiding a loss or saving time.
• Urgency: how fast the cost of inaction grows over time.
• Confidence: how confident the model is that its assessment of the current information is sufficient and its conclusion is reliable.
• Risk: how irreversible the action is and how far its potential impact spreads.

Illustrative decision logic:

def decide(action, context):
    # 1. Hard red line: irreversible, high-impact actions go straight to human review
    if action.risk_level == "BLOCKED":
        return Decision(action="escalate", reason="beyond automated execution scope")
    value      = estimate_value(action, context)        # 0~1
    urgency    = estimate_urgency(action, context)      # 0~1
    confidence = estimate_confidence(action, context)   # 0~1
    score = 0.4 * value + 0.3 * urgency + 0.3 * confidence
    if action.reversible and score >= 0.6:
        return Decision(action="execute", notify="digest")     # execute now, report in digest
    if action.reversible and score >= 0.4:
        return Decision(action="execute", notify="immediate")  # execute, but notify immediately
    if not action.reversible and confidence >= 0.8:
        return Decision(action="request_approval", timeout="24h")  # request approval
    return Decision(action="suggest_only")                  # suggest only, do not execute

Three engineering principles sit behind this rule:

1. Reversibility outranks confidence: even when the model is highly confident, irreversible actions should be approved first.
2. Confidence must be explainable: the model should provide the reasoning behind its judgment rather than just a score, otherwise a human reviewer has nothing to go on.
3. Every decision leaves a trace: each decision, along with its rationale, is written to the audit log for post-hoc review and threshold tuning.

5. Permissions and Security: Tiered Authorization and the “Non-Delegable” Red Line

The permission model for a background agent should be organized around actions, not around apps. Under the same calendar permission, “check availability” and “send an invitation to an external person on the user’s behalf” carry completely different risk levels.

5.1 A Four-Tier Permission Model

Tier	Action characteristics	Examples	Authorization requirement
L0 read-only	Query, read, summarize	Query sales data, read calendar	Runs automatically after user authorization
L1 reversible write	Create or modify, and undoable	Create an event, edit a draft document	Runs after authorization, with immediate notification on completion
L2 irreversible write	Send externally, publish, delete	Send an email, publish a report, clear a temp directory	Explicit approval required per action or per rule
L3 restricted operations	Irreversible and high impact	Fund transfers, securities trades, signing legal documents, deleting critical data	No automated execution capability by default; not executed even with authorization

Why must L3 have a “non-delegable” red line? From a software design perspective, these operations share three traits: first, they are irreversible, so mistakes cannot be rolled back; second, their legal and financial consequences spill over, and losses are not limited to the individual account; third, they offer an extremely high payoff for attackers, and if a prompt injection or privilege-escalation instruction exploits one, the outcome far exceeds an ordinary data breach. These operations should therefore be excluded from the automated execution path and completed by the user in person through official channels. This is not a missing capability but part of the product positioning: the value of an agent lies in handling “high-frequency, low-per-incident-risk” work for the user, not in taking major decisions on the user’s behalf.

5.2 Policy as Code

Permission rules should be managed in version-controlled, testable, auditable configuration files, not hardcoded in prompts. Example:

version: 1
defaults:
  mode: deny            # deny by default
  audit: full
policies:
  - name: daily report auto-generation
    tools: ["data_source.query_*", "report.render_*", "notify.push_digest"]
    risk_max: L0
    schedule: "0 8 * * 1-5"
    notify: digest
  - name: meeting scheduling
    tools: ["calendar.*"]
    risk_max: L2
    approval:
      trigger: "external_attendee"   # confirm each time an external person is involved
      channel: mobile
      timeout_hours: 24
  - name: external sending
    tools: ["mail.send", "chat.send_external"]
    risk_max: L2
    approval:
      trigger: always                # every send requires confirmation
      preview: true                  # show full content in the approval card
      rate_limit: "10/day"
  - name: high-risk operations
    tools: ["payment.*", "contract.sign", "account.transfer", "data.purge"]
    policy: hard_block               # hard block; no automated execution path offered

Accompanying governance recommendations:

• Revocable permissions: users can view the connectors and scopes they have authorized at any time and revoke them in one click; related tasks are immediately downgraded or stopped after revocation.
• Per-session confirmation: for L2 actions, even with long-term authorization, show a summary of “what will be done and which objects are affected” before execution.
• Human-in-the-loop: an approval request should include an action summary, expected impact, data sources, and the model’s rationale so the user can make a judgment within seconds.
• Approval timeout handling: on timeout, the default is not to execute, not to approve.

6. Reliable Execution and Cost Governance

Always-on means tasks may run unattended, which makes reliability especially important:

1. Idempotency: each task carries an idempotency key (e.g., “user + task type + date”) so repeated triggers do not produce duplicate results.
2. Deduplication and convergence: the same event is processed only once within a short time window, and notifications on the same topic are aggregated by time window.
3. Retry and backoff: transient failures are retried with exponential backoff; once the retry limit is exceeded, the task goes to a dead-letter queue and triggers an alert.
4. Checkpoints and recovery: long-running tasks save intermediate state at stage boundaries and resume from the checkpoint after a failure instead of starting over.
5. Timeouts and watchdogs: every task gets a maximum execution time and a maximum number of steps to prevent infinite loops.
6. Budget control: set per-user daily and monthly inference budgets and connector call limits; when approaching a threshold, degrade to “alert only, do not execute.”

7. Observability and Evaluation

An agent that executes autonomously must be explainable, reviewable, and improvable. At a minimum, collect the following metrics:

Category	Metrics	Purpose
Effectiveness	Task completion rate, first-pass rate, human rework rate	Measures whether the agent actually solves problems
Proactivity	Number of proactive pushes, adoption rate, user dismiss/ignore rate	Calibrates the “proactivity threshold” to avoid disruption
Security	Approval rejection rate, red-line block count, anomalous call count	Reveals policy gaps and attack attempts
Cost	Average inference cost per task, connector call count	Optimizes model tiering and scheduling frequency
Latency	End-to-end latency from event occurrence to result delivery	Assesses whether timeliness meets business needs

For evaluation methods, a three-step approach of “offline replay + shadow mode + canary release“ is recommended:

• Offline replay: reconstruct scenarios from historical data and check whether the agent’s plans and actions match expectations.
• Shadow mode: a new version only generates decision recommendations without actually executing them, and they are compared against human decisions.
• Canary release: open a new capability to a small share of users first, and expand gradually once the metrics are stable.

8. Phased Rollout Roadmap

Building an always-on agent from scratch can proceed in phases:

• Phase 1: read-only assistant. Connect the calendar, documents, and business data sources, and provide a daily briefing and anomaly alerts. This phase involves almost no write operations, carries the lowest risk, and is best suited to validating the value of proactivity.
• Phase 2: reversible writes with approval. Open up undoable operations such as creating events, drafting documents, and organizing spreadsheets, along with an approval center and audit logs.
• Phase 3: controlled multi-agent collaboration. Introduce specialized sub-agents (e.g., “data collection,” “analysis,” “copywriting”) coordinated by a lead agent, all sharing the same permission and audit framework.

At the infrastructure level, this kind of system generally relies on common cloud capabilities: object storage (for reports, artifacts, and memory data), message queues (for event-driven decoupling), a container platform (for running agents and sandboxes), a gateway service (for unified connector egress and rate limiting), a secrets manager (for access tokens), logging services (for audit and troubleshooting), and monitoring and alerting (for metrics and anomalies). Cloud platforms such as Baidu AI Cloud provide these general-purpose capabilities and can be considered one candidate foundation for agent infrastructure.

9. Conclusion

The competitive focus of always-on AI agents is shifting from “how strong the model is” to “how trustworthy the system is.” The model determines how good the plans the agent can come up with are, while the permission model, sandbox, memory governance, and audit mechanisms determine whether those plans can be turned into reality safely.

For developers and enterprises, the most worthwhile investment is not chasing more connectors but clearly defining “which actions the agent can take on its own and which must be decided by a human,” and then codifying that into software. A clear “non-delegable” red line, together with an auditable tiered permission model, is the foundation on which this type of product earns users’ long-term trust.