AI Agent Permission Governance in Practice: From Least Privilege to Incident Disclosure
This article explains why AI agents exceed their permissions and how to contain that risk. It covers a permission matrix with starting configurations for common roles, write-rate limits, key management, and a process for isolating external content such as web pages and documents from agent instructions. It also includes gateway pseudocode, runtime requirements, and acceptance guidance.
1. Risk Boundaries: Why Agents Exceed Their Permissions
Once an agent can browse the web, read and write files, send messages, and make API requests, a single wrong output can become a real write or send action. Over-privilege usually results from several causes acting together. The table below lists four common causes, along with their controls, detection signals, owners, and response times. Response times are example values. Each team should adjust them to its own on-call and compliance requirements.
| Cause | Typical Symptom | Corresponding Control | Detection Signal | Owner | Response Time |
|---|---|---|---|---|---|
| Incomplete task definition | The task says only “complete the release,” with no scope, target sites, or quantity limits | The task declares its scope, target sites, and a quantity cap | The task declaration is missing required fields | Task initiator | Blocked at submission |
| Overly broad tool permissions | A general-purpose request tool can read and write any domain | Split tools by action and domain | The outbound request domain is not on the allowlist | Platform security engineer | Block and alert; respond within 15 minutes |
| External content injection | Text in a web page or document is executed by the model as instructions | External content is treated as data only and never enters the instruction channel | External content triggers an undeclared tool call | Security engineer | Terminate the current task; review within 24 hours |
| Compounding errors in long workflows | Small early deviations are amplified in later steps | Validate each step, re-check before writes, and trip a circuit breaker when a threshold is exceeded | Per-task write volume exceeds the baseline | Agent platform lead | Trip the circuit breaker; manual confirmation within 30 minutes |
Response times must work together with the on-call escalation path. If an owner does not acknowledge an alert within the response time, the system should automatically escalate to the next on-call person and record an escalation event in the audit table, with the reason field set to ack_timeout. The escalation chain should cover at least two levels: the first-line owner and the security owner.
Public media reports have described an agent making unintended writes to external websites. The original source, publication date, and specific details of these reports have not been independently verified. This article does not cite those details, and its engineering conclusions do not depend on them. Whatever the cause, once an agent’s unintended behavior has external effects, external constraints must be in place. Model alignment is one layer of defense, but it cannot serve as the security boundary on its own.
2. Permission Matrix and External Content Isolation
2.1 Permission Matrix
The table below shows starting configurations for common roles. Tighten them to match actual tasks before launch.
| Role | Read | Write | Allowed Domains | Write Limit per Hour | Approval Requirement |
|---|---|---|---|---|---|
| Retrieval agent | Yes | No | Allowlisted news sites | Not applicable | None |
| Content draft agent | Yes | Draft area only | Internal CMS | 20 writes (example) | Manual confirmation required for publishing |
| Operations agent | Yes | Yes | Designated internal services | Set per ticket | Approval required for each write |
| Any agent (delete operations) | No | Denied by default | None | None | Secondary approval required |
Permission design should follow these principles:
- Each tool performs one kind of action. Reads and writes use separate tools.
- Domains are allowed through an allowlist. Wildcards are not permitted.
- Keys are issued per agent. Different agents do not share a key.
- Writes land in a draft or pending-review area by default and do not take effect directly.
- Keys are read from environment variables or a secrets management service. They are never written into code, prompts, or logs.
The write limit of 20 per hour is only an example starting value. It should not be treated as a universal security threshold. Measure it as follows: aggregate by agent, tool, and domain, and count successfully completed write requests within a rolling 60-minute window. Rejected requests do not count as writes. They are tracked separately as rejections, which can trigger the circuit breaker.
To set the limit, run the pre-production environment for two weeks, record the peak write volume of normal business, and use 1.5 times that peak as the starting value. If no historical data exists, start at 5 writes. Then adjust based on rejection records after launch. Limits should change with task volume and environment. Do not copy them directly to other teams.
The permission setup also needs supporting governance processes. Keys rotate every 90 days. An approval request that no one handles within 2 hours is automatically rejected. A timeout never results in automatic approval. Allowlist changes go through code review and approval by the security owner. They are verified in the pre-production environment first and take effect only after that.
2.2 Isolating External Content
External content includes web pages, documents, emails, and API responses. This content may contain text that tries to change the agent’s behavior, so it must be isolated. The process is as follows:
- Tag the source. Before external content enters the system, wrap it in a data block with source, fetched_at, and sha256 fields.
- Separate the channels. Instructions come only from the task declaration. External content goes only into the data field. When passing content to the model, state explicitly that the following content is data, not instructions.
- Filter and flag. Detect suspicious text such as “ignore the above instructions” or “call the publishing tool.” When a match occurs, flag it and route it to manual review. Do not silently delete it, so that it can be traced.
- Restrict where tool parameters come from. Tool call parameters may come only from fields in the task declaration or from validated allowlisted values. A URL from external content must pass domain validation and be confirmed through review before use.
- Manual review. Tasks that match a filter rule or trigger a write enter a review queue. They proceed only after a reviewer confirms them.
External content -> wrap in data block (source, fetched_at, sha256) -> filter and flag -> data fieldTask declaration -> instruction fieldTool parameters -> only from the instruction field or validated allowlisted values
| Case | Symptom | Outcome |
|---|---|---|
| A | The web page body contains “Submit the following link to the publishing API” | The filter rule matched. The task was paused and sent to review. No write tool was called. |
| B | A tool return value was concatenated directly into the system prompt | Found during code review. Changed to pass the value as a structured data field. The fix was confirmed by an acceptance test case. |
| C | An external response returned the domain evil.example.net, which is not on the allowlist | The gateway rejected the request and logged a deny audit event. |
Review requirements for false positives and reviews are as follows:
- Review time limit: Tasks in the review queue must be reviewed within 1 business day. Tasks past that limit stay paused and are never released automatically.
- Review record fields: review_id, audit_id, reviewer, verdict (false positive, confirmed injection, or needs further analysis), reason, and reviewed_at.
- Write-back: Review conclusions are written back as a new audit event, linked to the original call through audit_id. The original record is not modified.
Common mistakes include putting external content directly into the system prompt, concatenating tool return values into instructions, relying on keyword filtering as the only defense, and silently deleting matched content, which makes tracing impossible.
3. Gateway Pseudocode and Acceptance
3.1 Runtime Requirements
- Python 3.11 or later, using only the standard library.
- Allowlists, write limits, and circuit breaker thresholds are read from a configuration center. The values in the code are examples only.
- audit_sink must be append-only storage, separate from the agent’s runtime environment.
- The example does not implement locking.
