AI Agents Should Execute, Not Decide: Managing the Risks of Autonomous Decision-Making

1. Introduction: From Answering Questions to Taking Action

Over the past two years, the way large language models are used has changed significantly. Early models mainly generated text, after which users reviewed the responses, made their own decisions, and carried out the necessary actions. Today, more systems can operate computer interfaces, call APIs, read and write files, and complete tasks across multiple steps.

These systems are commonly known as AI agents. They can open applications, fill out forms, and submit requests, making them similar to digital employees that can perform actions rather than simply provide information.

Greater capabilities can improve efficiency, but they also change the risk profile. When a model only produces text, the worst outcome may be that it misleads the reader. Once the model has access to tools and account permissions, an error can directly result in data changes, financial transactions, or service disruptions. The key question should therefore shift from “What can the model do?” to “What should the model be authorized to do?”

2. The Capability Boundary of AI Agents

To determine whether a task is suitable for an AI agent, it can first be divided into two parts:

• Execution: Completing clearly defined steps, such as organizing data, generating drafts, or applying formatting changes in bulk.
• Decision-making: Determining whether a goal is appropriate, setting priorities, or deciding whether to abandon a task or change a business rule.

Execution has clear inputs and outputs, and its results can generally be verified. It is therefore suitable for delegation to an agent. Decision-making, however, involves value judgments and accountability, and should remain with an authorized person. Giving an agent decision-making authority effectively allows an entity that cannot be held accountable to make business judgments on behalf of the organization.

3. Three Core Risks of Autonomous Decision-Making

3.1 Instability

The same task may produce different results when the wording of a prompt changes slightly, when an additional piece of context is introduced, or when the task is run at a different time.

Traditional software is expected to produce the same output for the same input, making test results reproducible. Agent outputs are subject to variability, so passing a single test does not demonstrate that the system will behave consistently in production.

3.2 Lack of Explainability

When an outcome is incorrect, teams may have difficulty determining why the agent selected a particular approach. The reasoning process may not be transparent, intermediate steps may be omitted, and tool-call records may be incomplete.

Without a traceable decision chain, investigating the cause of an incident and fixing the relevant component becomes much more difficult.

3.3 Unclear Accountability

When an employee makes a mistake, an organization can conduct a review, provide training, or adjust the person’s responsibilities. When an agent makes an error, the cause could be an improperly designed prompt, truncated context, an abnormal tool response, or random variation in the model itself.

Without complete logs, a team may not even be able to determine which part of the system needs to be changed.

Together, these risks mean that AI agents should not handle critical decisions without human review.

4. From Saying the Wrong Thing to Doing the Wrong Thing

Earlier approaches to model safety focused primarily on whether generated content violated rules or contained inaccurate information. With AI agents, safety concerns extend to behavior.

An agent may delete files, send a message to the wrong recipient, modify a configuration, or access a system without proper authorization. The risk has therefore shifted from content risk to operational risk.

Protective measures must expand accordingly. Content filtering alone is not enough. Effective controls also require permission management, operation auditing, and execution isolation.

5. The Right Division of Responsibility: Humans Define the Rules, Agents Execute Them

In practice, organizations can adopt a model in which humans define the rules and machines perform the execution:

• Humans define objectives, operational boundaries, approval requirements, and rollback procedures.
• The agent performs specific actions within those predefined boundaries.
• The system records every input, output, tool call, and approval result.

The following pseudocode illustrates a basic permission gate in which all high-risk actions require human confirmation:

DEFINE high_risk_actions = [delete_data, send_external_message, modify_permissions, deploy_to_production]
FUNCTION execute_agent_action(action, parameters):
    record_audit_log(action, parameters, current_time)
    IF action IS IN high_risk_actions:
        approval_result = request_human_approval(action, parameters)
        IF approval_result IS NOT approved:
            record_audit_log(action, "rejected")
            RETURN rejected
    result = call_tool(action, parameters)
    record_audit_log(action, result)
    RETURN result

This design contains three important elements:

1. An action allowlist limits the operations the agent can call.
2. High-risk operations must go through human approval.
3. Every call is recorded in an audit log for later review.

6. Five Steps for Implementation

1. Inventory Tasks and Resources

List all systems, data, and accounts that the agent may access, and identify whether each permission is read-only or allows modification.

2. Apply Permission Levels

Classify actions according to their potential impact:

• Read-only operations
• Reversible changes
• Irreversible operations

Set different approval requirements for each level.

3. Run the Agent in a Sandbox

Start in an isolated environment. Confirm that the output is sufficiently stable before gradually granting access to production resources.

4. Establish Auditing

Record the prompt version, a summary of the context, tool-call parameters, and tool responses. This ensures that every decision and action can be traced.

5. Evaluate the System Regularly

Use a fixed test set to measure success rates and error types. When performance drift is detected, tighten permissions promptly.

7. Recommendations for Business Teams

• Do not equate higher efficiency with permission to make autonomous decisions.
• Keep human checkpoints in critical processes involving funds, customer data, and external communications.
• Make agent instructions as specific as possible. Clearly state the objective, constraints, prohibited actions, and failure-handling procedures.
• Treat the agent as an execution system that requires ongoing supervision, not as an employee that can be configured once and then left unattended.

8. Conclusion

The value of AI agents lies in freeing people from repetitive execution so they can focus on judgment and trade-offs. Handing complete decision-making authority to an agent introduces instability, limited explainability, and unclear accountability into the core of the business.

A safer approach is for humans to define the rules and boundaries while agents execute tasks efficiently within those limits. Auditing and continuous evaluation can then be used to keep the system aligned. This model provides the benefits of automation while preserving organizational control.