Why AI Agents Are Getting Smarter: Moving from Tool Calling to Code Execution

Why Agent Capability Depends on More Than the Model

In recent years, AI agents have advanced rapidly in scenarios such as information retrieval, document processing, business system operations, and automated analysis. When evaluating how capable an agent is, people often focus only on the model’s parameter size, reasoning ability, or answer quality.

In real-world business, however, the final result depends not only on whether the model can “think,” but also on how efficiently it can use external tools.

A complete agent typically needs to perform the following steps:

1. Understand the user’s goal;
2. Determine which tools to call;
3. Construct tool parameters;
4. Retrieve data returned by external systems;
5. Decide the next action based on the results;
6. Loop when necessary, handle exceptions, and produce the final output.

When a task involves only a simple query, traditional tool calling usually works fine. But when a task needs to connect multiple business systems and handle large intermediate data sets, serious efficiency bottlenecks emerge.

For example, a user might request:

Read a meeting note from the document system, extract the key information, and write it into a specified record in the CRM system.

This looks like a simple “read once, write once” operation. But if the meeting note is very long, a traditional approach would return the entire document to the model, and the model would then reorganize its content into the input for the next tool. Large amounts of data pass back and forth through the model’s context, resulting in high computational cost and noticeable latency.

For this reason, agent architecture is shifting from “the model directly calls tools” toward “the model writes code, and an execution environment calls the tools.”

Why Traditional Tool Calling Creates Context Bottlenecks

1. All tool definitions are injected up front

The traditional approach usually places the names, parameters, return values, and usage instructions of all available tools into the model’s context at the start of a task.

If a system connects to only two or three tools, this design is acceptable. But in an enterprise environment, an agent may connect to many systems at once:

• Document systems;
• Databases;
• CRM systems;
• Ticketing systems;
• Messaging systems;
• Object storage;
• Logging and monitoring services;
• Internal business APIs.

Each system may expose dozens of tools or more. The tool definitions alone consume a large amount of context, and the model still has to filter out which tools are relevant to the current task.

This increases input length and causes two additional problems: the model must locate the right tools among a large volume of irrelevant descriptions, increasing its comprehension burden; and the more tool definitions and complex parameter structures there are, the higher the probability of incorrect calls.

2. Intermediate results repeatedly pass through the model’s context

The traditional flow looks roughly like this:

User request
  ↓
Model selects a read tool
  ↓
Tool returns the full document
  ↓
Full document enters the model's context
  ↓
Model generates parameters for the write tool
  ↓
Parameters are sent to the business system again

The biggest waste here is that intermediate data merely moves from one tool to another, yet it must first pass through the model’s context.

If a document contains tens of thousands of lines of text, the model may need to receive the full content first, then reorganize most of it into the input for the next tool. The model never needed to read the data word by word, nor to regenerate it—but the traditional architecture forces it through this step anyway.

3. Complex control flows are hard to express in natural language

When a task involves loops, conditionals, exception retries, and data filtering, relying solely on natural language instructions is unreliable.

Examples include:

• Iterating over multiple documents;
• Processing only records from the last month;
• Skipping data that has already been synced;
• Retrying failed requests three times;
• Writing to the business system only when confidence exceeds a threshold;
• Submitting results in batches to avoid oversized single requests.

If all of this logic is implemented through consecutive model calls, the system accumulates many intermediate steps. Each step requires the model to make a fresh decision, which increases latency and the risk of the process going out of control.

The Core Idea of Code Execution Mode

Code execution mode does not give the model unlimited permissions. Instead, the model generates a controlled program, and an isolated execution environment performs the actual operations.

The basic flow can be summarized as:

User request
  ↓
Model analyzes the task and generates a program
  ↓
Execution environment loads tool capabilities on demand
  ↓
Program calls external systems
  ↓
Intermediate data flows within the execution environment
  ↓
Results or a summary are returned to the model

Illustrative code:

// The following is an architectural illustration, not tied to a specific product API
const source = await documentService.read({
  id: "document-id"
});
const importantParts = source.content
  .split("\n")
  .filter(line => line.includes("action items"));
for (const item of importantParts) {
  await recordService.append({
    target: "business-record",
    content: item
  });
}

This code reflects three key changes:

1. The model does not need to read all tool definitions up front;
2. Document content can be processed directly inside the execution environment;
3. Filtering, looping, and exception handling are handled by the program rather than by the model making step-by-step decisions.

The model’s role shifts from “personally moving all the data” to “planning how the data should be processed.”

Four Main Benefits of Code Execution Mode

1. Tools are loaded on demand

In a code execution architecture, tools can be organized like code libraries, modules, or capability catalogs. The model first learns an overview of the available capabilities and only reads the detailed usage instructions of a specific module when it is actually needed.

This resembles module imports in programming, rather than printing the entire enterprise system’s API documentation to the screen all at once.

The advantages include:

• Shorter initial context;
• Less interference from irrelevant tools;
• Easier addition and removal of tools;
• Tool documentation that can be maintained independently;
• Better scalability in complex systems.

Note that on-demand loading does not mean tool documentation can be omitted carelessly. Each module should still provide clear input constraints, return structures, error descriptions, and permission boundaries.

2. Intermediate data no longer passes through the model repeatedly

In a code execution environment, data returned by one tool can be fed directly into another tool.

For example, a document’s read results can be filtered, summarized, and format-converted in memory, and then the processed output can be written to the business system. Unless the model genuinely needs to understand or judge the content, raw data does not need to be fully returned to the model.

This change is especially well suited to scenarios such as:

• Batch processing of large documents;
• Merging and filtering multiple files;
• Structured data cleaning;
• Log analysis;
• Bulk record synchronization;
• File format conversion;
• Multi-step data orchestration.

A more sensible data flow is:

External system A → variables in the execution environment → filtering and transformation → external system B

Instead of:

External system A → model context → model regenerates the data → external system B

3. Programmatic control flow is more stable

Code natively supports conditionals, loops, functions, exception handling, and concurrency control.

For example, a synchronization task can use logic like this:

for (const item of items) {
  if (item.status === "completed") {
    continue;
  }
  try {
    const result = await process(item);
    await saveResult(result);
  } catch (error) {
    await recordFailure(item, error);
  }
}

If the model had to make every judgment individually, the more steps a task had, the more opportunities for errors. By letting programs handle deterministic logic, the model can focus on the parts that truly require semantic understanding.

A more reasonable division of labor is:

• The model handles intent recognition, plan design, and semantic judgment;
• Programs handle loops, filtering, sorting, retries, and data movement;
• External tools access real business resources;
• The execution environment handles permission control, resource limits, and runtime auditing.

4. Easier to control cost and latency

Reducing data transfer through the context typically reduces the model’s input size. Cutting unnecessary intermediate reasoning also shortens the task chain.

This is especially evident in batch processing. The model only needs to generate a program once or a few times, and the execution environment can then perform a large number of repetitive operations without invoking the model for every single record.

However, cost and speed improvements are not automatic. They also depend on:

• Whether tool responses support field filtering;
• Whether paginated reads are available;
• Whether bulk submissions are possible;
• Whether caching is implemented;
• Whether redundant reads are avoided;
• Whether concurrency levels are set appropriately;
• Whether long texts are processed in layers.

This Architecture Does Not Mean “Letting the Model Run Code Freely”

Code execution capability must rest on strict security controls. Otherwise, model-generated programs could cause unauthorized access, data leakage, or resource abuse.

1. The execution environment must be isolated

Code should run in a controlled sandbox with restrictions on:

• File system access scope;
• Network access targets;
• Allowed system commands;
• CPU, memory, and runtime limits;
• Ability to create subprocesses;
• Readable environment variables;
• The set of business tools that can be called.

The execution environment should not directly inherit all host machine permissions.

2. Tool permissions must be minimized

Different tasks should use different permission sets. For example, a document-reading task should not have delete permissions by default, and a data analysis task should not automatically be able to modify production records.

Tool permissions can be divided into:

• Read-only;
• Create;
• Modify;
• Delete;
• Configuration management;
• Access to sensitive data.

High-risk operations should additionally require human confirmation, secondary verification, or an approval workflow.

3. Both inputs and outputs must be validated

Before code execution, check whether the model-generated program contains dangerous operations. During tool calls, validate parameter formats, data ranges, and target objects. After execution, verify that returned results meet expectations.

Examples:

• Pause when the write volume exceeds a threshold;
• Require human confirmation for bulk deletions;
• Automatically mask sensitive fields on access;
• Cap the number of exception retries;
• Split into batches when a single processing volume exceeds the limit.

4. The entire process must be observable

Enterprise-grade agents should not merely return “task completed.” They should also record:

• What plan the model generated;
• What code was actually executed;
• Which tools were called;
• Which resources were accessed;
• At which step exceptions occurred;
• How much compute was consumed;
• What changes were ultimately made.

Robust logging, auditing, and tracing are essential foundations for troubleshooting and meeting compliance requirements.

Designing Tool Interfaces Suited for Code Execution

Whether tool interfaces are suited for programmatic calls directly affects the overall result. High-quality tools typically share these characteristics:

1. Clear input structure

Parameters should use explicit structured formats rather than mixing multiple meanings into a paragraph of natural language.

2. Composable return values

Return values should be easy for programs to continue processing—structured objects, lists, and status fields—rather than large blocks of human-readable text only.

3. Field filtering and pagination support

High-volume APIs should support returning only the necessary fields and allow paginated reads, avoiding pulling everything at once.

4. Actionable error messages

Errors should explain the cause of failure, whether a retry is possible, and how to fix parameters—not just return a vague failure notice.

5. Explicit side effects

Tools that write, modify, or delete data should clearly indicate the impact of their operations and, where possible, provide preview, dry-run, or rollback mechanisms.

The Best Division of Labor: Model, Program, and Tools

Code execution is not meant to replace the model; it redivides responsibility between the model and the program.

Work suited to the model:

• Understanding user intent;
• Identifying task goals;
• Interpreting unstructured content;
• Choosing processing strategies;
• Generating program plans;
• Performing semantic analysis of exceptions.

Work suited to programs:

• Iterating over data;
• Conditional filtering;
• Sorting and aggregation;
• Format conversion;
• Batch calls;
• Retry and timeout control;
• Result caching;
• Validation and statistics.

Work suited to external tools:

• Accessing business data;
• Performing real writes;
• Querying system status;
• Uploading and downloading files;
• Triggering business workflows.

The division can be summarized as:

The model makes judgments, the program executes, and the tools connect to the real world.

Conclusion: Agent Competition Is Moving from “Calling” to “Orchestrating”

Traditional tool calling solved the problem of connecting agents to external systems. But as tool counts, data volumes, and task complexity keep growing, simply exposing all tools to the model can no longer meet enterprise-grade requirements.

The value of code execution mode is not just saving context—it fundamentally changes data flow and control flow:

• Tools shift from being fully exposed at once to being loaded on demand;
• Intermediate data shifts from passing through model context to flowing within the execution environment;
• Multi-step operations shift from consecutive natural language decisions to composable program logic;
• The model shifts from a data mover to a task planner and intelligent controller.

Going forward, excellent agent systems will not only pursue “human-like answers.” They will place greater emphasis on orchestration efficiency, permission security, execution stability, data governance, and end-to-end observability.

A truly powerful agent is not necessarily the one that calls tools the most often, but the one that completes complex tasks reliably with the fewest model interactions, the cleanest program structure, and the strictest permission boundaries.