Long-Running AI Agents for Reliable Delivery of Complex Tasks

Long-running AI agents are not chat windows with longer prompts. They are work systems designed to be recoverable, auditable, and verifiable.

They are particularly useful for three connected types of work:

• Converting existing mathematical proofs into machine-checkable formal text
• Organizing scientific hypotheses, derivations, and reproducible computations
• Continuously modifying, migrating, and delivering code in repositories governed by tests and review requirements

Latency, throughput, or price in a single demonstration cannot substitute for correctness. The following workflow focuses on engineering practices rather than any particular model or vendor.

Define Completion Before Starting a Long-Running Task

Most complex tasks fail not because the model is incapable, but because the definition of “done” is vague. Before work begins, create a one-page task brief that specifies four areas.

Define a Verifiable Deliverable

For formalization tasks, completion means that the proof assistant reports zero unresolved goals and that important lemmas can be replayed independently.

For scientific tasks, completion means that the assumptions, units, boundary conditions, and numerical results are reproducible—not merely that the explanation sounds plausible.

For software tasks, completion means that tests, static analysis, builds, and review findings have all been addressed.

Specify Constraints Before Implementation

Constraints should include:

• Directories that may be modified
• Secrets and production configuration that must not be touched
• Dependency and licensing requirements
• Performance and resource limits
• External behavior that must remain unchanged

High-risk changes, such as language migrations or kernel and driver work, should also define rollback points and observability metrics in advance.

Budget for Effective Work

Estimate the budget based on successful work rather than list price alone. Deeper reasoning generally increases latency and context size, while failed retries can multiply costs.

Track at least:

• Number of tasks
• Retry count
• Number of human interventions
• Percentage of tasks that pass acceptance

If unit prices decrease while retries increase, the total cost may still rise.

Make Stop Conditions Explicit

The system must stop and escalate to a human when:

• The failure set does not shrink after repeated attempts
• Unauthorized external data is required
• A security classification boundary is reached
• A subtask begins expanding without a clear path to completion

Do not allow subtasks to grow indefinitely simply because the agent can continue generating output.

Turn Different Problems into the Same Work Pattern

Mathematical formalization, scientific derivation, and large-scale programming look different on the surface, but they share a similar operational structure:

1. Retrieve existing knowledge.
2. Generate candidate steps.
3. Use an independent tool to determine correctness.
4. Convert failures into constraints for the next attempt.

Mathematical Formalization

Consider formalizing a classical theorem in number theory. The main effort is usually not the final line of the theorem, but whether the library contains the required algebraic structures, whether the proof strategy is stable, and whether intermediate lemmas depend too heavily on manual patches.

A practical decomposition is:

1. List definitions and axiomatic dependencies.
2. Generate lemmas chapter by chapter.
3. Check each lemma independently.
4. Attempt the main theorem only after the supporting structure is stable.

Treating “the proof finishes after dozens of hours” as a capability metric is not meaningful. Library completeness and the cadence of verification determine the schedule.

An agent can help retrieve proof patterns, fill in boilerplate lemmas, and revise strategies based on error messages. It does not acquire mathematical correctness automatically.

Scientific Analysis

Scientific work must guard against the gap between a correct-sounding explanation and an incorrect calculation. Problems involving black-hole accretion, perturbations near an event horizon, or stability analysis may combine analytical approximations with numerical experiments.

A reliable collaboration pattern is to:

• Fix the coordinate system, sign conventions, and boundary conditions.
• Break derivations into steps that can be differentiated or checked dimensionally.
• Recompute key intermediate quantities with an independent script.
• Repeat experiments with randomized initial conditions and preserve the random seeds.

Physical conclusions must be supported by independent computation or expert review. An agent can accelerate the organization of assumptions and experimental scripts, but it cannot replace reproducible scientific evidence.

Software Delivery

Software delivery is judged by whether maintainers can accept and maintain the change, not by whether the code compiles once.

If evaluation only checks the model’s statement that a task is complete, results will be systematically overestimated. A more realistic process is to:

1. Apply the patch in an isolated working copy.
2. Run the project’s existing tests and new regression tests.
3. Perform human or rule-based review for interface, licensing, and security violations.

High-intensity reasoning may sometimes improve first-pass success rates relative to a more conservative baseline. However, if failures cause the context to expand, the cost per task can quickly exceed that of an apparently cheaper approach.

At minimum, compare:

• Pass rate
• Number of lines changed manually
• Total cost

Orchestration: One Scheduler and Multiple Short-Lived Workers

Long-running tasks should not depend on a single context that writes everything from beginning to end. A practical design uses one scheduler and multiple short-lived workers.

The scheduler maintains only the objective, dependencies, and acceptance status. Each worker receives one clearly bounded subtask, such as implementing a driver interface, proving one lemma, or rerunning one group of parameters.

Task brief → Scheduler
  ├─ Retrieval worker: read-only access to references and existing interfaces
  ├─ Implementation worker: modify code or proofs in an isolated working copy
  ├─ Validation worker: run tests, proof checks, or numerical verification
  └─ Review worker: identify defects against the constraints; never simply declare success
Failures return to the scheduler, which narrows the scope of the next change

Execution should be isolated. Run builds and tests in containers or virtual machines. Store artifacts in object storage. Pass subtasks through a message queue. Record each input summary, tool exit code, and diff patch in logs and monitoring alerts.

This allows the system to resume from the last passing checkpoint after an interruption instead of repeating the entire task.

Running thousands of subtasks in parallel is not automatically better. Without dependency analysis and conflict resolution, parallelism only creates branches that cannot be integrated. A useful rule is to make the main path pass sequentially first, then parallelize independent lemmas, modules, or parameter sweeps.

Extremely Long Tasks

Generating a bootable system from an empty directory or rewriting a critical service in another language are both extremely long tasks. They can expose weaknesses in planning, testing, and review, but should not be presented as production-ready results by default.

For a new system, each layer—kernel, disk and input devices, persistence, graphical interface, and installation—needs a minimal runnable slice and fault injection.

For a language migration, first lock down the behavioral contract, including:

• Startup latency
• Memory usage
• Error semantics
• Concurrency model
• Observability metrics

Without benchmarks and rollback procedures, a claimed speed improvement cannot be confirmed and should not be included in capacity planning.

The Validation Loop Matters More Than Generation Quality

Every iteration should produce exactly one of three outcomes:

1. Passed
2. Failed, with reproducible evidence
3. Blocked, with a clear explanation of the missing information

A vague intermediate state that merely “looks close enough” must not enter the main branch.

For proofs, the authority is the proof assistant’s batch checker, not the model’s confidence.

For numerical work, validation should include fixed-seed reruns, dimensional analysis, and comparison with known limiting cases.

For code, validation consists of tests, static analysis, builds, and diff review.

Implementation and review workers should be separate. The same context should not both write the change and judge whether it is acceptable.

Place Human Review Where Automation Cannot Decide

Human review is required for questions such as:

• Whether an axiom has been replaced or weakened improperly
• Whether a physical approximation has exceeded its valid range
• Whether an interface has changed its external promise
• Whether generated content includes large unlicensed excerpts

Review comments should be written back into the task brief and become constraints for the next iteration, rather than remaining as one-time remarks in a chat transcript.

Prevent Drift During Long Runs

Long-running jobs need regular checkpoints. Every few hours, record:

• The current diff
• Checks that have passed
• Open questions
• Resource consumption

For security audits, architecture prototypes, or proof engineering that runs overnight, checkpoints are more useful than simply allowing the process to continue. They reveal when the work has drifted away from the original objective.

If external feedback says that the system can “run for many hours,” translate that claim into measurable properties:

• Can the task recover from interruption?
• Does context compression preserve constraints?
• Are failed subtasks being submitted repeatedly?

Manage Reasoning Intensity, Latency, and Cost Separately

Reasoning intensity, generation speed, and price are three different controls. They should not be collapsed into a single conclusion that one system is simply “stronger.”

High-intensity reasoning is appropriate for steps with few valid branches and high failure costs, such as:

• Selecting a proof strategy
• Defining migration boundaries
• Reviewing concurrency defects

Lower-intensity settings are usually sufficient for format conversion, boilerplate tests, and documentation drafts. Stable, lower-cost configurations are suitable for much routine coding and knowledge organization. Higher intensity should be reserved for open-ended problems with tight constraints.

Fast configurations typically trade higher cost for lower latency. They are useful for tight human-in-the-loop workflows, but not necessarily for unattended batch proofs or full-repository migrations. Batch work should prioritize checkpoints and caching rather than peak generation speed.

Cost estimates should include:

• Input and output size
• Retries
• Tool execution time
• The percentage of failed subtasks discarded
• Human review hours

Comparing only the price per million tokens ignores context expansion under high-intensity reasoning. When third-party measurements differ from vendor self-reports, use your own task set, fixed acceptance scripts, and complete billing records. Do not rely on scores that cannot be independently verified.

Use Risk Tiers Instead of Capability Showmanship

Requests related to cybersecurity and life sciences can have dual-use implications. A responsible deployment uses the same general foundation with different safeguards, rather than removing restrictions from every interface.

The default path should retain classification and refusal policies for broad developer and paid-use scenarios. It should not provide step-by-step procedures, payloads, or parameters for unauthorized intrusion, nor actionable instructions for producing or enhancing pathogens or toxins.

When legitimate research or defensive testing requires additional access, use a separate process with:

• Qualification review
• A stated purpose
• Log retention
• Human approval
• Scope limited to authorized assets and experiments

Relaxed restrictions should never be the default, and they must not be bypassed through fictional framing or claims that the work is “for research.”

Operationally, classifiers should apply both at task entry and at tool invocation. An agent may discuss the impact of vulnerability classes, remediation principles, threat modeling, and the organization of security testing. Once a request becomes a reproducible attack procedure, the scheduler should terminate the worker and raise an alert.

The same principle applies to biology. Public, mechanism-level scientific explanations may remain available, while details involving synthesis, screening evasion, or enhancement of harmful effects must be stopped.

An Executable Acceptance Checklist

Before launch or external reporting, confirm each item:

• The completion criteria in the task brief can be judged automatically, and passing logs have been saved.
• Every subtask patch or proof script can be replayed from a checkpoint without relying on a particular conversation.
• Parallel changes have been merged, and conflicts were not resolved by simply overwriting one branch.
• Performance, correctness, and rollback are supported by evidence. Unsupported claims of acceleration or completion must not enter the final conclusion.
• Costs are calculated per successfully accepted task and compared with a lower-intensity configuration.
• High-risk requests have classification records. Samples that were refused or escalated to humans are reviewed rather than silently discarded.
• Licenses, secrets, and unpublished data have not entered prompts or logs.

Conclusion

The value of long-running AI agents lies in turning open-ended problems into recoverable work.

Formal proofs require lemma-level validation. Scientific problems require independent recomputation. Software changes require tests and review.

Reasoning intensity, parallelism, and generation speed are only means to that end. When acceptance criteria, cost accounting, and risk tiers are built into orchestration rather than marketing language, teams can determine whether an agent is delivering verified results—or merely producing a larger volume of unvalidated output.