AI Agent Evaluation and Selection, and How to Fix Long-Task Memory Loss
This article explains why many AI agents stall at the demo stage and how to move them into reliable production use. It covers choosing the right scenario before choosing a model, evaluating agents by real-task success rather than benchmark scores, and the key metrics to track during selection. It then examines why scheduled, long-running tasks gradually "forget" their instructions and presents an engineering approach: externalize the task contract and state, make time anchors explicit, and add memory anchors, idempotency, checkpoint resume, and guardrails—so the model decides and the system remembers.
1. The Problem: Where Agent Deployment Gets Stuck
Agents are moving from “being able to chat” to “being able to do actual work.” Users want them to automatically complete entire workflows described in natural language—information aggregation, data organization, periodic inspection, and so on. Yet many attempts stall at the demo stage: sometimes steps jump around, sometimes the agent gets stuck at a tool call, and sometimes it simply forgets the original instructions partway through.
Breaking this down, effectiveness usually hinges on two factors: the underlying model and the Skills design. The former is the agent’s “brain,” responsible for understanding instructions, decomposing steps, and selecting and invoking tools. The latter is the “operating manual,” defining what the agent knows, what it can do, and which boundaries it must respect. Missing either one sharply degrades the experience. Without a capable model, you cannot write good skills or execute them well; without appropriate skills, even a powerful model lacks the process and boundaries to follow. At that point, handling things manually is actually simpler.
1. The first step is not choosing a model—it’s choosing a scenario. Agents are not suited to every task. An ideal deployment scenario usually has four characteristics: a relatively stable process, results that are easy to verify automatically, controllable cost of failure, and a low cost of human fallback. Examples include periodically aggregating public information, generating structured daily reports, and inspecting system metrics to produce anomaly reports. For steps that require complex judgment or irreversible operations, human approval should be retained. When the scenario is chosen well, both the requirements on the model and the trial-and-error cost drop significantly.
2. Beware the intuition that “a bigger model means a better experience.” A model that shines in general benchmarks is not necessarily optimal inside an agent workflow. Agents face open-ended, multi-step, long-horizon tasks whose success depends on three things holding simultaneously: each step is executed correctly; context stays consistent from step to step; and the established rules are followed consistently across the entire task cycle. The more links in the chain, the higher the probability that errors accumulate.
Suppose the per-step accuracy is 95%. Roughly speaking, the probability that 10 consecutive steps are all correct is about 0.95¹⁰ ≈ 60%, and for 20 steps it drops to about 0.95²⁰ ≈ 36%. This is an illustrative estimate—models are correlated with one another, and verification can intercept some errors. But it is enough to show that, even with automatic error correction, a small drop in per-step quality gets significantly amplified over a long chain. Conversely, the long waits caused by an inefficient model further raise the risk of timeout interruptions.
2. Rethinking Evaluation: From “Answering” to “Getting It Done”
2.1 How Traditional Evaluation Differs from Agent Capability Evaluation
Traditional evaluation mostly revolves around knowledge and reasoning, with relatively self-contained questions where answering does not really affect the external environment. Agent evaluation is closer to end-to-end task acceptance: the focus shifts from “is the answer correct” to “did the task get done, and was it done reliably.”
| Dimension | Traditional Evaluation | Agent Capability Evaluation |
|---|---|---|
| Evaluation target | Knowledge and reasoning ability | Task completion ability in real workflows |
| Sample source | Question banks, annotated datasets | Cases and logs generated during actual runs |
| Judgment method | Comparing answers against a reference answer | Result verification, script- or rule-based judgment |
| Key metrics | Accuracy, score | Task success rate, step count, latency, cost |
| Focus | Single-turn output quality | Tool invocation, multi-step planning, state retention, rule adherence |
2.2 Four Characteristics of a High-Quality Agent Evaluation Set
- Samples come from real runs. Select high-frequency, representative cases from actual task logs, covering varying complexity and common failure modes. Hand-written ideal examples tend to overestimate capability.
- Automatically judgeable. Each case should have a clear completion standard and a scriptable verification method—for example, whether the output contains required fields, whether the target data is accurate, and whether an inappropriate tool was called. Samples that rely on subjective human scoring should be kept to a reasonable proportion.
- Layered by difficulty. At minimum, divide cases into four categories: single-step tool invocation, multi-step planning, cross-system coordination, and long-horizon scheduled tasks. This lets you pinpoint whether the capability problem is “cannot do it” or “forgets while doing it.”
- Reproducible. Fix the inputs, control the execution environment, and pin the evaluation and model versions so that different models, or different versions of the same model, can be compared side by side.
2.3 Evaluation Needs Multi-Dimensional Observation
Success rate is certainly the primary metric, but looking at success rate alone easily leads to one-sided conclusions. It is worth recording the following dimensions as well:
- Task success rate: the proportion of tasks that pass verification, further split into first-attempt success rate and final success rate;
- Average step count: reflects how concise the planning is. Too many steps means both cost and latency increase, and there are more chances to err;
- Average latency and timeout rate: related to the user’s waiting experience and whether scheduled tasks finish on time;
- Cost per task: determined jointly by average number of calls, input/output size, and per-call price;
- Human takeover rate: the proportion of tasks that require human correction midway, directly measuring “how much manpower was saved”;
- Failure type distribution: distinguishing instruction misreading, incorrect tool-call parameters, timeout interruptions, forgotten rules, and so on, providing a basis for targeted optimization.
2.4 Selection Principles: Balancing Success Rate and Inference Efficiency
Inference efficiency usually refers to the speed and cost-effectiveness with which a model produces results under given resources. It directly affects the number of tasks processed per unit time, the cost per call, and whether the failure rate climbs as runtime grows. The two need to be weighed together, following this path:
- Match model tier to task difficulty. Layer the evaluation set by difficulty, compare models of different sizes at each tier, and find the lowest tier “good enough” to complete that layer’s tasks;
- Set a response budget by latency sensitivity. Interactive tasks are more sensitive to latency, while background batch tasks can be relaxed somewhat;
- Tune the scenario first, then consider upgrading the model. Adding skill definitions around a specific scenario, optimizing prompts and tool descriptions, and introducing verification and retries are often more cost-effective than simply switching to a larger model;
- Track continuously. Build a fixed regression evaluation set and keep comparing success-rate and efficiency metrics during gradual rollouts and small-scale pilots.
It is worth emphasizing that scale is never the only deciding factor. Whether a model has been optimized for agent behaviors such as tool invocation and multi-step planning, whether instruction-following is stable, and whether tool-call parameter formats are well-formed and reliable all directly affect end-to-end performance. So even models of the same size class can differ markedly in real-workflow usability.
3. Scheduled-Task “Amnesia”: Why Long Tasks Break Down
Many developers use agents for scheduled tasks—for example, aggregating data every morning or checking resource usage every week. But in actual runs, a common pattern emerges: everything works fine for the first few days, then the agent gradually drifts from the original settings, or produces nothing for several days in a row.
The problem usually isn’t “bad memory” but rather instruction-following over long contexts: as history accumulates, the original agreements get buried; when the context is compressed, the original rules may be rewritten or even lost; and at some step, the task has forgotten “why it started” and “what to do next.” At its core, it is a stack of three engineering problems.
1. Context expansion. Each execution appends inputs, outputs, intermediate results, and tool return values, so the history keeps growing longer. The share of constraints relevant to the whole task shrinks, and the model’s attention to key rules gets diluted.
2. Instruction decay. In repeatedly compressed or summarized history, the original instructions may distort, key constraints get simplified, and subtle conditions are merged or dropped. Once qualifiers such as time, scope, and format are lost, execution gradually goes off track.
3. Missing time semantics. The model itself has no stable clock and cannot preserve state between two scheduling runs. “Every Monday” becomes “which weekday last Monday was,” and “before the deadline” becomes “after it expired”—problems that usually stem from time anchors not being explicitly maintained.
Notably, this problem appears across models of all sizes, not just small ones. A structural memory gap must be filled with structural engineering means, rather than pinned on a larger context window.
4. The Solution: Let the Model Decide, Let the System Remember
The core idea is a single sentence: do not entrust memory to the model—put it outside the model.
- Externalize the task contract. Write the task goal, trigger conditions, execution steps, completion standard, and exception-handling policy into a versioned configuration file, fully loaded before each run and not dependent on conversation history.
- Externalize state. Store progress, completed steps, pending confirmations, and intermediate artifacts in persistent storage—for example, a database, object storage, or a state record in a message queue. The model reads only the necessary slices and writes back the latest state after finishing.
- Make time anchors explicit. At the scheduling layer, compute the time baseline for this run—current date, the cycle it belongs to, last successful run time, time remaining to the deadline—and provide it to the model in an explicit form such as “Today is Month X, Day X, Year X,” rather than asking the model to calculate it.
- Memory anchors. In a long task, re-inject the task definition and current progress every fixed number of rounds. This is like a person reviewing their “to-do list” before starting work, and it effectively counteracts context drift.
- Structured contracts. Agree on a fixed output structure for the model—for example, action type, target object, parameters, and a self-check item—then parse it programmatically before execution, reducing the ambiguity of free-form text.
- Guardrails and verification. Set global safety rules, a tool allowlist, and human-confirmation steps, and automatically verify the output of each run.
The following illustrative code shows how to combine “rule injection” and “periodic re-anchoring” in an executor:
# Illustrative code: inject global rules into the executor and periodically re-anchor the task contract# execute is a placeholder function standing in for "let the model run one round of inference / tool invocation"def run_with_anchor(task, agent, anchor_interval=5):# task["contract"] is the task contract from external storage, containing the goal, steps, and completion standardstate = load_state(task["id"]) # read task state from persistent storagedeadline = get_deadline(task) # compute the time baseline for this run at the scheduling layerfor step in task["contract"]["steps"]:if state["status"] != "running": # task was paused or cancelled, exit immediatelyreturn stateprompt = GLOBAL_RULES # global safety rules: highest priority, cannot be overriddenprompt += f"\n[Time anchor] Current time: {now()}, deadline this run: {deadline}"prompt += f"\n[Task progress] Completed {state['done']}/{len(task['contract']['steps'])} steps"if state["step_index"] % anchor_interval == 0:# Memory anchor: re-inject the full contract every few steps to resist context driftprompt += f"\n[Task contract] {render_contract(task['contract'])}"result = execute(prompt + f"\n[Current step] {step['name']}", tools=task["tools"])if not result["ok"]: # step failed: no retry, no advance, wait for human handlingstate["status"] = "paused"state["error"] = result["error"]save_state(task["id"], state)notify_oncall(task["id"], result["error"]) # notify on-call staffreturn statestate["done"] += 1state["step_index"] += 1save_state(task["id"], state) # persist after each step for checkpoint resume and auditingstate["status"] = "completed"save_state(task["id"], state)return state
5. Engineering Practice: Making Automation Workflows Trustworthy
In automation, an unreliable process that runs frequently often does more harm than an occasional manual operation. So an agent workflow needs to treat itself like a production service:
- Enforce task agreements. Agent tool calls are inherently probabilistic, and the same instruction can produce different paths across runs. Every run must be given explicit constraints: what to execute, what not to execute, how to handle failure, and where the boundary for human intervention lies.
- Idempotency and deduplication. Generate a unique identifier for each task, such as “task name + cycle + date.” Before actually executing, check whether the identifier has already been processed, to prevent retries, duplicate scheduling, or system restarts from running the same task multiple times.
- Checkpoint resume. Persist progress after each completed step, so an interrupted task resumes from the checkpoint instead of starting over. This saves cost and reduces the side effects of repeated operations.
- Timeout and retry policy. Set a timeout per step and a deadline for the whole task; use exponential backoff with an upper bound for retries, and once the retry budget is exceeded, hand over to a human.
- Observability and alerting. For each run, record time, model version, latency, number of calls, and result status, and connect to unified logging and monitoring/alerting. Automatically alert on consecutive failures, prolonged lack of output, abnormally rising latency, and similar conditions.
- Permission isolation and human fallback. Follow the least-privilege principle for tool calls, require human confirmation for sensitive data or irreversible operations, and for high-risk operations, run in an isolated environment first and act on the real environment only after verification passes.
- Cost and capacity governance. Periodically analyze call counts, input/output size, and timeout distribution, remove redundant steps, and set reasonable limits on concurrency and resources.
5.1 Idempotency and Checkpoint Resume (Illustrative)
# Illustrative code: use "task name + cycle + date" as the idempotency key to prevent duplicate executiondef run_scheduled_job(name, func, period="daily"):key = f"{name}:{period}:{today()}" # idempotency key: repeated triggers on the same day run only onceif is_done(key):return "skipped"try:mark_running(key) # mark "running" first to avoid concurrent duplicate entryresult = func() # func saves progress step by step internally, supporting checkpoint resumemark_success(key, result)return "ok"except Exception as exc:mark_failed(key, str(exc)) # record failure for alerting and human investigationreturn "failed"
5.2 The Boundaries of Exception Handling
Automatic error correction needs clear boundaries. For “safe-failure” tasks—tasks that cause no side effects even when they fail, such as reading and aggregating data—automatic retries are acceptable. For “unsafe-failure” tasks, such as executing a change or sending a message externally, the cause of failure is often unknown, and blindly retrying can cause duplicate submissions or erroneous sends; in these cases you should stop and hand over to a human. It must be clear that automatic retry is not a panacea: it cannot fix task-definition errors, nor can it compensate for a model’s capability gap at a given step; it only improves the system’s resilience against occasional jitter.
6. Implementation Advice: A Phased Approach
- Phase 1: Choose a scenario. Review existing processes, pick high-frequency, low-risk, easy-to-verify tasks as pilots, and define completion standards and human fallback.
- Phase 2: Build an evaluation set. Distill typical cases from real run records, layer them by difficulty, and build an automatically judgeable, repeatable regression evaluation set.
- Phase 3: Do the selection. Run the same evaluation across candidate models, comprehensively compare success rate, latency, cost, and human takeover rate, choose a model tier on the principle of “just enough,” and polish skills and tool definitions around the scenario.
- Phase 4: Engineer it. Implement externalized task contracts and state, idempotency, checkpoint resume, timeout and retry, alerting, and permission isolation, and bring the agent’s execution process into a unified observability system.
- Phase 5: Govern it. Continuously track metrics, periodically review failure samples, dynamically retire inefficient scenarios, and keep human confirmation for high-risk operations.
7. Conclusion
The value of an agent lies not in how long an answer the model can write, but in whether it can complete a whole task stably and controllably. Evaluation should focus on real-task success rate and execution efficiency; selection should be guided by scenario needs; and the “amnesia” problem in long-horizon tasks must, in the end, be solved by an engineered memory mechanism.
When the model handles understanding and decision-making, skills define boundaries and processes, and the scheduling system handles memory and fallback, an automation workflow truly has the foundation for stable long-term operation.
