Designing Prompts for Long-Running AI Agents: 12 Practical Rules

Introduction: From Process Control to Outcome Contracts

Many teams still use prompting habits from the short-conversation era: repeatedly asking a model to “think step by step,” reminding it to “be more careful,” or attempting to micromanage its reasoning in natural language.

For newer models with long-running execution, multi-step workflows, and always-on reasoning capabilities, these practices often create three problems:

• More noise in the context
• Unnecessary token usage
• Premature termination before the task is actually complete

Prompt design should shift from controlling how the model thinks to defining what must be done, when to stop, and what evidence is required.

The following 12 rules are not syntax tricks. They are control-plane design principles for long-running AI agents.

1. Replace Vague Goals with an Acceptable Definition of Done

The most important rule for an autonomous agent is to define what “complete” actually means. Instead of writing “migrate the authentication system,” specify verifiable acceptance criteria, escalation conditions, and stopping boundaries.

Task: Migrate the authentication flow to a unified authentication protocol.
Definition of done:
- All authentication endpoints use the unified authentication protocol;
- All existing tests pass;
- The new authentication flow includes tests for both normal and error paths;
- The legacy authentication implementation has been removed;
- Related documentation has been updated.
Pause and request confirmation only when an issue requires a human decision,
permissions are missing, or an irreversible high-risk action is required.

The definition of done should be verifiable whenever possible. Test results, file diffs, API responses, and documentation changes can all serve as evidence.

Vague objectives cause an agent to stop when the result merely “looks good enough,” or to spend too much time exploring minor details without a clear endpoint.

2. Stop Micromanaging Reasoning; Guide the Agent with Goals and Constraints

Instructions such as “think carefully,” “analyze every possibility in depth,” and “do not rush” have limited value for models that already support extended reasoning. They may even encourage unnecessarily long responses.

A better approach is to use prompts to define:

• Objectives
• Constraints
• Evidence requirements
• Output structure

Reasoning depth should be controlled through configurable effort levels.

If faster responses are needed, do not simply tell the model to “think less.” Lower the effort level and use an evaluation set to determine whether quality remains acceptable. The prompt should specify what the reasoning must accomplish, not prescribe every step of the reasoning process.

3. Start with Medium Effort and Tune It with Data

Do not automatically set the effort level to its maximum. For long-running tasks, higher effort usually means greater latency and token consumption, but it does not guarantee a proportional improvement in quality.

A practical approach is to classify tasks by risk:

• Exploratory analysis can use lower effort.
• Critical migrations and complex debugging may require higher effort.
• Irreversible operations should receive the highest level of scrutiny.

A more engineering-oriented method is to build a quality-cost curve. Select representative tasks, run them at low, medium, and high effort, and record:

• Completion rate
• First-pass success rate
• Human intervention rate
• Average execution time

Then assign default effort levels by task type.

4. Reduce Ambiguity with Structured Inputs and Outputs

Long-running tasks need outputs that are recoverable, parseable, and auditable. Prompts should require the agent to return status, evidence, unresolved questions, and the next action in a fixed structure.

For example:

Output contract:
status: completed | blocked | failed | in_progress
summary: What was completed in this phase
evidence: Test results, diff summary, or log summary
open_questions: Issues requiring human decisions
next_action: The next phase or the reason for stopping

Structured output reduces the risk of receiving natural language that sounds reasonable but cannot be verified. It also allows downstream processes to automatically determine whether to continue, roll back, or escalate.

5. Define Boundaries, Prohibited Actions, and Permission Scope

“Be careful” is not a boundary. Boundaries should specify the allowed read and write scope, available capabilities, prohibited actions, and actions that require confirmation.

For example:

• Only modify files within the specified directory.
• Do not delete production data.
• Deployment, rollback, and permission changes require human confirmation.
• Sensitive data must not be written to logs.

Least-privilege access and explicit prohibitions reduce risk more effectively than repeatedly telling the agent to act cautiously.

6. Require an Evidence Chain, Not Just a Conclusion

In long-running workflows, the most dangerous outcome is not failure. It is claiming success without sufficient support.

Prompts should require every important conclusion to include verifiable evidence, such as:

• Test results
• A summary of file differences
• Reproduction steps for failures
• Relevant log excerpts
• A completed checklist

Evidence does not need to be lengthy, but it must support the conclusion. If a claim cannot be verified, it should be labeled as an assumption and accompanied by a proposed verification method.

7. Define Tool-Calling Contracts and Failure Handling

Agents depend on tools to complete long-running tasks. Each tool call should have a clear contract covering:

• Purpose
• Inputs
• Successful output
• Retryable errors
• Non-retryable errors
• Idempotency requirements
• Retry limits

For example:

Tool: Run tests
Input: Test scope and timeout budget
Success: Return a pass/fail list and summary
Retryable: Temporary resource exhaustion
Non-retryable: Compilation errors or assertion failures
Failure handling: Record evidence, narrow the scope, and escalate when necessary

Without failure handling, tool calls can lead to infinite retries or premature abandonment. A tool contract turns “the agent can call the tool” into “the agent can use the tool reliably to complete the task.”

8. Manage Context and State with Regular Checkpoints

Long-running tasks cause context to grow over time. Prompts should require the agent to produce regular checkpoints containing:

• Completed work
• Outstanding work
• Key decisions
• Risks
• Locations of supporting evidence
• The next action

Checkpoints can be used both to compress context and to resume work after an interruption. For multi-stage tasks, state should be stored in a structured form to prevent repeated exploration and context drift.

9. Define Stop Conditions, Escalation Rules, and Human Decision Points

“Ask when you encounter a problem” is too vague. Prompts should distinguish among at least four stopping conditions:

1. Normal completion
2. Blocked and requiring escalation
3. Failure termination
4. Budget exhaustion

Human decision points should be tied to specific situations, including:

• Missing permissions
• Conflicting requirements
• High-risk irreversible operations
• Insufficient evidence that cannot be verified automatically

Clear stopping rules prevent two common failures: continuing when the agent should stop, which can increase risk, and stopping too early when the task could have continued safely.

10. Control Long Tasks with Milestones and Incremental Delivery

Break large tasks into verifiable phases:

1. Read-only analysis
2. Small-scope changes
3. Broader changes
4. Full validation

Each milestone should have a deliverable, acceptance criteria, and a rollback point.

Incremental delivery reduces the rollback cost of large, all-at-once changes. It also gives the agent feedback after each phase, allowing it to correct its direction before the task expands further.

11. Make Cost, Latency, and Token Budgets Explicit

Long-running tasks should include explicit resource constraints, such as:

• Time budget
• Token budget
• Tool-call budget

When a budget is exceeded, the prompt should define a fallback strategy. The agent may need to:

• Narrow the scope
• Produce an intermediate report
• Request human confirmation
• Stop while preserving the available evidence

Budgets should not be treated only as metrics monitored after execution. They should be part of the prompt and guide the agent’s decisions when resources are limited.

12. Establish Offline Evaluation and Prompt Regression Testing

Prompts are engineering assets and should be versioned and tested for regressions.

Create a representative task set and define evaluation metrics such as:

• Completion rate
• First-pass success rate
• Average number of interaction turns
• Number of unnecessary tool calls
• Human intervention rate
• Cost per task

After each prompt change, run the regression suite and determine whether higher cost has produced a meaningful quality improvement.

Failure cases should be categorized, for example, as:

• Ambiguous requirements
• Missing boundaries
• Tool errors
• Insufficient evidence
• Unclear stopping conditions

Prompt revisions can then target the specific failure category instead of relying on general reminders.

Implementation Checklist

A practical implementation sequence is:

1. Write a verifiable definition of done.
2. Add an output contract, permission boundaries, evidence requirements, and stopping conditions.
3. Configure effort levels and budgets according to task risk.
4. Build an evaluation set and regression process.

The core principle is simple: control less of how the model thinks and define more of what must be done, when to stop, and how success must be proven.

This approach helps long-running agents operate more reliably, controllably, and audibly across complex workflows.