Agent Execution Frameworks: Enabling LLMs to Move from Conversation to Action

Introduction: Models Understand; Execution Frameworks Make Things Happen

Large language models are becoming more capable, but a practical gap remains: many models “know a lot” but may not be able to “get things done.” They can write a complete remediation plan but cannot directly modify files; they can analyze error messages but may still require users to copy logs manually; they can plan complex workflows but lack a channel to call real systems.

The key to closing this gap is not just a larger model, but an agent execution framework. It is also commonly called a harness. In simple terms, a harness is an intermediate layer that gives the model “hands and feet.” It allows the model not only to output suggestions, but also to call tools, read and write files, execute commands, observe results, and continue adjusting based on feedback.

This article focuses on the engineering practice of agent execution frameworks. It answers several key questions: What is a harness? How do you build a minimal viable version? How is context compressed? When should the loop stop? How are dangerous operations controlled?

1. What Is a Harness? A Bridge Between Models and Applications

If an LLM is viewed as a smart but isolated advisor, the harness is the interface that lets the advisor enter the real world. It does not directly generate intelligence, but it determines which systems intelligence can reach, which actions it can perform, and which constraints it must follow.

A typical agent execution framework usually contains the following modules:

Module	Main role	Key design points
Perception layer	Converts files, API responses, terminal output, web content, and other information into context the model can understand	Information filtering, structuring, summarization
Decision loop layer	Drives the model through an iterative “think—act—observe—think again” process	Loop control, termination conditions, state management
Execution layer	Turns model intent into real calls, such as reading or writing files, executing commands, or sending requests	Parameter validation, timeout control, error handling
Security layer	Performs permission checks, blocks dangerous operations, and supports confirmation and rollback	Least privilege, allowlists, denylists, audit
Observability layer	Records model decisions, tool calls, execution results, and error information	Logging, tracing, replay, alerting

Among these, context management is easy to overlook but very important. Each model decision consumes tokens, while real-world information can be very large. If all logs, directories, and API responses are directly given to the model, it increases cost and may interfere with judgment. Therefore, a harness is not just an executor; it is also an information filter.

2. How to Build a Minimal Viable Harness

A minimal viable agent execution framework does not need to be complex. The core is to form a closed loop: perceive the environment, call the model, execute an action, observe the result, and decide whether to continue.

A minimal architecture can contain four parts

1. Tool definitions: Tell the model what capabilities are available, such as reading files, writing files, executing shell commands, or calling HTTP APIs.
2. Action parsing: Convert the text output by the model into structured actions, such as tool names, parameters, and target paths.
3. Executor: Actually executes actions and returns standardized results.
4. Loop controller: Decides whether to continue, pause for confirmation, or terminate the task.

Simplified pseudocode

max_steps = 20
for step in range(max_steps):
    observation = collect_environment()
    context = compress_context(observation)
    decision = model.invoke(context, tools=tool_schemas)
    action = parse_action(decision)
    if is_dangerous(action):
        ask_user_for_confirmation(action)
    result = executor.run(action)
    log(step, action, result)
    if task_completed(result):
        break
    if failed_too_many_times():
        break

This pseudocode is simple, but it already contains the key elements of a harness: tool calls, context compression, action execution, safety confirmation, logging, and termination conditions.

Tool definitions should be clear

The tool descriptions given to the model should not just say “read file.” They should include:

• Tool name: for example, read_file
• Description: reads the content of a specified file
• Parameter structure: path, encoding, maximum read length
• Return format: file content, truncation marker, error information
• Risk level: read-only, write, delete, external call

Structured tool definitions are more reliable than vague natural-language prompts. Models are more likely to generate valid parameters, and the execution framework can more easily perform validation and auditing.

3. Why Programming Is Only a Starting Point

Early agent execution frameworks were often associated with coding assistants. This is not surprising: code is structured text, compilers, tests, and terminals can provide quick feedback, and programmers are more familiar with command-line interfaces. Therefore, “letting the model take action” is often easier to form a closed loop in programming scenarios first.

However, if a harness is understood only as a programming tool, its general value is underestimated. A framework that can read and write files, execute commands, and call interfaces can also be used for:

• Automated operations inspections: read logs, check service status, and generate reports;
• Batch data processing: scan directories, clean data, and output result files;
• Document generation and publishing: organize materials, generate Markdown, and upload to a knowledge base;
• Cross-system workflow orchestration: retrieve data from form systems, then call approval or notification APIs.

A common but not necessarily rigorous observation is that in the early stages of a technical paradigm, people tend to judge whether new capabilities are necessary based on existing tools; as feedback signals increase and product forms mature, solutions once considered “good enough for now” may be re-evaluated. Therefore, the design of an execution framework should not focus only on a single scenario, but should consider general capabilities and security boundaries.

4. Common Strategies for Context Compression

A core challenge of agent execution frameworks is that the information seen by the model must be sufficient, but not overwhelming. Common compression strategies include:

1. Directory summaries first

Do not give the model the entire file tree at once. You can first provide a directory structure summary and then expand specific files as needed. For example:

The project root contains:
- src/: source code directory
- logs/: log directory
- docs/: documentation directory
- config/: configuration files
Which directory should I inspect?

2. Long-text truncation and summarization

For logs, API responses, or web content, retain key information: error codes, stack summaries, time windows, and exception statistics. Raw content can be stored in object storage or temporary files, and the model only reads summaries or fragments on demand.

3. Structured responses

Tools should be required to return structured results, such as JSON, when possible. The model can focus on necessary fields instead of parsing large amounts of free-form text.

4. Retrieval-based reading

If the environment has a knowledge base or historical records, you can first retrieve the most relevant fragments before handing them to the model. This is more economical than putting all materials into the context.

5. Preserve error signals

Compression does not mean discarding critical information. When execution fails, at least the following should be retained:

• The failed command or API;
• Exit code or status code;
• Core error message;
• Number of retries;
• The most recent successful step.

Only in this way can the model decide whether to continue, roll back, or request human intervention.

5. Loop Termination Conditions: Avoid Infinite Loops

One of the easiest places for an agent to lose control is continuing “try—fail—try again.” A robust harness must have clear termination conditions.

Common termination strategies include:

1. Goal achieved: The task is completed, such as a file has been generated, tests pass, or an API returns success.
2. Maximum steps: A preset maximum step count is reached. A minimal viable version can first set 10–20 steps, then adjust based on task complexity.
3. Token limit: Prevents unbounded growth of context and invocation costs.
4. Consecutive failures: For example, if the same action fails 2–3 times in a row, the agent should pause and request human confirmation.
5. Insufficient permissions: If the model tries to execute unauthorized actions, it should be blocked directly.
6. User intervention: The user cancels, modifies, or takes over the task.

For high-risk scenarios, a “cooling-off period” should also be set: the model cannot directly execute deletions, publications, or changes; it must first generate a plan and wait for confirmation.

6. How to Identify Irreversible Operations?

Deletion is not the only dangerous action. The following actions usually require stricter control:

• Deleting files, directories, or database records;
• Overwriting existing configurations or code;
• Modifying production environments;
• Publishing content to external platforms;
• Sending emails, messages, or external notifications;
• Changing permissions, keys, or access policies;
• Creating compute resources that may incur costs;
• Calling irreversible third-party APIs.

A tiered strategy is recommended:

Risk level	Example	Control method
Low	Reading files, querying status	Execute directly and log
Medium	Creating temporary files, calling read-only APIs	Restrict paths, timeouts, and call counts
High	Writing configurations, modifying production data	Requires human confirmation
Very high	Deleting data, publishing external content	Disabled by default; requires allowlist and approval

At the same time, dry-run support should be provided whenever possible. Let the model simulate execution first, output the actions to be performed and the scope of impact, and then truly execute after confirmation.

7. Engineering Practice Checklist: Making Harnesses More Stable and Secure

The following checklist organizes common practices for direct implementation during development:

Recommendation	Purpose	Minimal implementation action
Do not treat the harness as a one-off prompt	Ensure tasks can continue	Implement loops, state, termination conditions, and logging
Build a clear feedback loop first	Improve the model’s ability to self-correct	Record tool exit codes, error summaries, and result status
Enforce security constraints early	Reduce the risk of mistakes	Use least privilege by default, confirm dangerous operations, and require approval for deletion
Actions must be traceable	Facilitate debugging, auditing, and replay	Record task ID, step number, tool name, argument summary, and result
Reuse mature infrastructure	Reduce duplicate construction	Use object storage, message queues, logging services, and monitoring/alerting
Limit context size	Control cost and interference	Use summaries, directories, retrieval, and on-demand expansion
Set failure exit conditions	Prevent infinite loops	Limit steps, tokens, and consecutive failures

The log fields can at least include:

task_id: unique identifier of the current task
step: current step
tool: name of the called tool
args_summary: argument summary; avoid recording sensitive information
status: success / failed / blocked
risk_level: low / medium / high
duration_ms: execution duration
error_summary: error summary

In general cloud environments, mature infrastructure can also be reused: object storage can be used to store file assets, message queues can decouple asynchronous tasks, logging services can trace execution processes, and monitoring/alerting can detect anomalies. This allows teams to focus more on agent logic rather than underlying infrastructure.

Conclusion: Execution Frameworks Determine Whether Agents Can Be Deployed in Practice

The value of an agent execution framework is not in how new the concept is, but in whether it can turn “understanding” into “action.” It allows models to move from single-turn conversation to real system operations, and it also requires developers to carefully handle permissions, feedback, termination, observability, and security.

From an engineering perspective, a reliable harness should at least achieve three things: actions can be executed, processes can be observed, and risks can be controlled. The model determines the upper limit of intelligence, while the execution framework determines whether this intelligence can be stably deployed in real environments.