From Tool Calls to Long-Running Workflows: Engineering Reliable AI Agents
Reliable AI agents must do more than make correct first tool calls. They need to track durable state, distinguish success from uncertainty, recover safely from timeouts and restarts, manage scheduling and throughput, and know when to pause for human review. This article presents practical design principles for building agents that deliver verifiable outcomes rather than merely generating convincing responses.
A reliable agent is not defined by whether its first tool call is correct. The real test comes midway through a task: when data is missing, an API times out, or a process restarts, can the agent determine what has already been completed, what should happen next, and when it should stop and ask for human confirmation?
This is the fundamental difference between a conversational application and an execution system. A conversation primarily produces a response. An execution system produces verifiable results in the outside world. Whether a report was written to the correct location, an approval is still valid, or a timed-out write actually took effect cannot be confirmed by the model’s own statement. A reliable agent should let the model interpret goals and handle uncertainty, while the execution system manages permissions, state, and delivery.
1. Define Completion Conditions Before Asking the Model to Plan
A long-running task is not simply a longer prompt. It is a sequence of dependent operations organized into a workflow. Consider the task of generating a daily report for the previous business day. At minimum, the system must determine the reporting date, read the data, verify that all required source tables are available, perform the aggregation, validate the result, and publish the report. Without explicit completion conditions, a model may mistake “the report content has been generated” for “the report has been delivered as required.”
The task entry point should define four things:
- The input scope
- The final deliverable
- The conditions that must be satisfied before delivery
- The procedure to follow when information is insufficient
For example, a reporting task should specify the business timezone, the required data sources, and the reconciliation rules for the aggregated results. If critical data has not arrived, the correct behavior may be to suspend the task and wait—not to fill missing values with zeros, or replace validation with a note and continue publishing.
After the goal is decomposed into steps, each step should have verifiable preconditions and outputs. Independent data reads can run in parallel. Result validation depends on the aggregation output. Formal publication must wait for validation and authorization.
The model can propose steps, explain discrepancies, or suggest alternatives. However, explicit rules should determine which steps are allowed to execute and which results can move the workflow to its next stage. Fixed workflow logic does not need to be replanned by the model every time. Reserving the model for exception handling and unstructured information processing generally makes the system easier to test and audit.
2. Treat Tool Calls as Interface Engineering, Not Just Prompt Design
Tool calling has at least three dimensions of correctness:
- Selecting the right operation
- Passing the right parameters
- Producing the expected result
A syntactically valid request is not necessarily valid for the business. Two dates may both match the required format while referring to the wrong reporting window. A write request may pass parameter validation but point to an unauthorized directory. The executor should therefore validate field types, value ranges, resource ownership, and operation permissions before making a call. After the call, it should verify the returned value or the actual external artifact.
Tool responses must also be explicit enough to support correct decisions. “No data,” “data is not ready,” and “the call failed” are different conditions. If all three are represented as an empty response, the model will struggle to choose the right next action.
A practical tool response should include:
- A status
- A reference to the result or artifact
- A meaningful error category when applicable
This allows the execution system to distinguish between continuing to the next step, waiting and retrying, requesting more information, and ending the task. Natural-language content returned from documents, web pages, or data records should be treated as business input. Its presence in a tool response does not make it a new execution instruction.
A Write Timeout Does Not Mean the Write Failed
Suppose the executor publishes a report but receives no response. The external service may already have saved the file even though the response was lost. Retrying immediately could create a duplicate file or overwrite a version that has already been reviewed.
A safer approach is to query the result using the operation identifier before retrying. For idempotent writes, retries should reuse the same idempotency key so that the receiving service can recognize duplicate submissions. If the result cannot be queried and the operation cannot be made safely repeatable, the system should mark the state as result unknown, pause automatic retries, and request human verification.
3. Use Persistent State to Track Progress
A long context helps the model read supporting material and understand prior events, but it is not workflow state. Repeatedly sending all conversations, query results, and logs back to the model increases latency and cost. It can also expose the model to conclusions from earlier messages that are no longer valid.
The system—not the model’s memory of a conversation—should record and retrieve task progress.
A practical task record should answer at least these questions:
- What is the task identifier?
- Which steps have passed?
- Where are the resulting artifacts?
- Which exceptions remain unresolved?
- Is the task currently eligible to continue?
Detailed data and complete logs can be stored separately. The model only needs a summary, key fields, and references to artifacts that are relevant to the current decision. A summary supports understanding, but it does not replace source data or audit evidence. For exact values such as amounts and quantities, calculations and verification should use structured results.
Recovery logic must also distinguish between:
- Not submitted
- Submitted but result unknown
- Confirmed successful
If a process exits after an external write completes but before the internal state is updated, the old state does not prove that the write failed. On restart, the system should reconcile the external result first, then decide whether to record success, resubmit, or escalate to a human.
For artifacts that require approval, the approval decision should be bound to a specific version. If the report changes after approval, the old approval must not automatically apply to the new version.
4. Let the Execution System Manage Time and Throughput
A model can understand “generate the previous day’s report every morning,” but it should not be responsible for triggering tasks precisely, managing wait windows, or handling missed runs.
The scheduling layer should distinguish among:
- Planned trigger time
- Actual execution time
- Business date represented by the data
If a morning task resumes in the afternoon after a delay, it should normally process the originally scheduled reporting date rather than recalculating the date based on the recovery time. The system also needs explicit deduplication rules when the task for the same business date is triggered multiple times.
When data is not ready, the task should enter a waiting state with a recorded next-check time. The model should not repeatedly query the same source within a single conversation. Temporary failures can be retried with backoff and a limited retry count. A task that has exceeded its delivery deadline should be alerted, terminated, or routed to a backfill process according to defined rules.
This distinction is important: some tasks are still meaningful if continued later, while others would only produce an outdated result.
For batch workloads, the bottleneck may not be the model. Excessive query concurrency can affect the data source, concentrated file writes can slow storage, and repeated retries can amplify downstream pressure. Concurrency limits, queue capacity, and execution budgets can be configured separately for each resource. Monitoring should include queue time, tool latency, and model latency.
If most of the elapsed time is spent waiting for source data, increasing the context window or switching models will not reduce delivery time. Adjusting the scheduling window or triggering execution when the data becomes available may be more effective.
5. Example: Turning a Scheduled Report into a Recoverable Workflow
When a reporting task starts, the system first records the business date, reporting rules, and publication target. The executor reads authorized data and verifies that all required sources are available and that the reporting interval is correct.
If the data is incomplete, the task enters a waiting state. Once the preconditions are satisfied, it continues without replanning the entire task.
The system then uses deterministic calculation logic to generate the summary and reconcile totals for amounts, quantities, and categories. The model can write explanations and summarize anomalies based on validated data. It should not invent missing values or use natural-language reasoning as a substitute for calculation.
After generating a draft, the system records the artifact version and validation results. If approval is required, the task pauses in a pending-approval state. Once that specific version is authorized, the system publishes it and verifies the write using a stable operation identifier.
If the publication request times out, the system should first check whether the corresponding version already exists at the target location. If it does, it should not publish another copy. If the result cannot be confirmed, the system should preserve the result unknown state rather than incorrectly reporting successful delivery.
This workflow does not require a model call at every step. Data retrieval, aggregation, validation, and publication can be handled by conventional programs. The model can focus on understanding requirements, explaining anomalies, and drafting narrative content.
This separation also improves maintainability. When reporting rules change, the calculation logic can be tested independently. When the quality of the explanatory text declines, the entire execution chain does not need to be redesigned.
6. Evaluate End-to-End Outcomes, Not Just Call Success Rates
Evaluation should cover both normal tasks and failures that can create disagreement about system state. In addition to the complete-data path, isolated tests should simulate:
- Late-arriving data
- Empty tool responses
- A lost response after a successful write
- Process restarts
- Duplicate triggers
- Invalidated approvals
Each test should define the expected outcome in advance: continue, wait, stop, or escalate to a human. It should also specify which external side effects must never occur more than once.
Metrics should be recorded at multiple levels. Task completion rate should be based on accepted deliverables, not on the model claiming that the task is complete. Recovery success rate should measure whether the system can continue after a failure without producing duplicate side effects. Human intervention rate should distinguish required approvals from unexpected manual fallbacks.
These metrics should be evaluated together with end-to-end latency, retry counts, and the cost of each successful delivery. This makes it possible to determine whether a problem originates in model decisions, tool interfaces, scheduling policies, or downstream dependencies.
When comparing different models, keep the task set, tools, permissions, and execution budget as consistent as possible. Otherwise, differences in the execution framework may be incorrectly attributed to the model.
For initial deployment, choose a low-risk scenario with clear acceptance criteria and replayable execution. Validate read-only analysis first, then enable draft writes, and only later decide—based on actual operational results—whether controlled publication should be allowed.
Agent reliability does not mean never encountering errors. It means that after an error, the system can accurately state where the task stopped, which operations have taken effect, and whether the next action can be executed safely.
