From Tool Calling to Code Execution: A New Efficiency Paradigm for AI Agents

Tool calling has become a standard capability in agentic systems. However, many teams find that as the number of tools and the complexity of tasks increase, context consumption grows rapidly, driving up latency and cost.

One increasingly relevant approach is to stop having the model call tools one by one. Instead, the model writes code that invokes typed tool APIs. Although this may appear to be a change in calling syntax, it actually redefines the boundary between model-driven decision-making and execution-environment data processing.

1. The Cost Structure of Traditional Tool Calling

Traditional agent tool calling typically follows this loop:

1. Put the definitions, parameter descriptions, and response schemas for all available tools into the model context.
2. Have the model read the definitions and decide which tool to call and which parameters to provide.
3. Execute the tool and insert the complete result back into the context.
4. Have the model read the result and decide what to do next.

This workflow is intuitive and broadly applicable, but its cost structure is not always efficient. The main costs come from three sources.

• Tool-definition overhead: Each tool may include a name, description, parameter schema, examples, and other metadata. When an agent connects to multiple tool servers, each containing dozens of tools, the definitions alone can consume a substantial number of context tokens.
• Intermediate-data overhead: Large blocks of text, tables, logs, or file contents returned by tools are often loaded into the model context in full. The model may then include the same data in the parameters of the next tool call, causing repeated transmission.
• Generation overhead: The model must generate a tool call for every execution step. Multi-step tasks therefore require multiple rounds of generation and waiting.

Consider a typical example. Suppose an agent connects to a cloud storage service and a customer relationship management (CRM) service, each exposing dozens of tools. The tool definitions alone may consume hundreds of thousands of tokens.

Now suppose the task is to download meeting notes from cloud storage and write them into a sales lead’s notes field in the CRM system. In a traditional workflow, the model first calls the storage service and receives the complete transcript of a two-hour meeting—perhaps around 50,000 tokens. Those tokens are added to the context. The model then generates another call that includes the same 50,000-token transcript in the CRM update parameters, creating another large context cost.

A seemingly simple task can therefore consume more than 100,000 tokens and require two model-generation rounds. Tool definitions, intermediate data, and parameter construction combine to create a significant “token black hole.”

2. Code-Executing Agents: Turning Tool Servers into Code APIs

The core idea behind code execution is to expose tool servers not only through descriptions intended for direct model calls, but also as importable, callable, typed code libraries.

Instead of outputting individual tool-call instructions, the model generates code. That code runs in a controlled execution environment, imports only the required tool APIs, processes intermediate data, manages control flow, and returns only the results or summaries that the model actually needs.

For example:

// Import the code libraries corresponding to the required tool servers
import * as drive from './servers/cloud-drive';
import * as crm from './servers/crm';
// The model writes code to complete the task
const transcript = (
  await drive.getDocument({ documentId: 'doc_123' })
).content;
await crm.updateRecord({
  objectType: 'Meeting',
  recordId: 'rec_456',
  data: { notes: transcript },
});

The key differences from traditional tool calling are as follows:

• Tool definitions do not need to be inserted into the context all at once. The execution environment can load them on demand from a file system or registry.
• The complete meeting transcript can move directly from the storage API to the CRM API inside the execution environment without entering the model context.
• The model generates one piece of orchestration code instead of repeatedly reading full intermediate results and generating the next call.
• Loops, conditional logic, exception handling, and concurrency controls can be expressed through programming constructs rather than natural-language instructions.

This does not mean the model no longer needs to understand the tools. It still needs to know which libraries are available, along with their important functions and types. However, it does not need to place every complete tool definition and every complete result into the context. More of the context budget can therefore be reserved for understanding the task, making decisions, and validating results.

3. Comparing the Two Approaches

Dimension	Traditional tool calling	Code-executing calls
Tool exposure	All definitions are placed in the context	Definitions are loaded on demand from a file system or registry
Intermediate data flow	Passes through the model context	Remains inside the execution environment
Control flow	Natural-language instructions such as “process every record”	Programming constructs including variables, loops, exceptions, and concurrency
Context usage	Tool definitions and intermediate results consume context	Significantly lower; only summaries or required results are returned
Testability	Depends largely on prompts and model behavior	Code can be unit-tested and statically checked
Error handling	Relies on model recognition and retries	Can use exception handling, retries, and idempotency strategies
Best suited for	Simple single-tool calls and exploratory tasks	Multi-tool orchestration, large-file processing, and batch tasks

Code execution is therefore more than a different tool-call format. It extends the agent’s execution model from step-by-step conversational decisions to code-based orchestration. The model generates high-level orchestration logic, while the execution environment performs the data flow and tool operations deterministically.

4. Three Areas of Efficiency Improvement

More predictable context usage

In the traditional model, tool definitions and intermediate data compete for space in the context window. Code execution can keep large objects in the execution environment and return only references, summaries, or paginated results.

For example, after reading a large document, the runtime might return a message such as “Read complete: N pages processed, with the following key fields,” rather than placing the entire document into the model context.

More reliable control flow

A natural-language instruction such as “process all records in a loop and retry failures three times” does not provide deterministic execution. Code can directly express loops, branches, exception handling, and concurrency limits, reducing drift during multi-step tasks.

Stronger software engineering practices

Generated code can be statically checked, unit-tested, audited, and reused. Tool APIs can provide type definitions, versioning, and structured error codes. The execution process can also record the call chain.

Compared with purely prompt-driven tool calling, code execution is easier to integrate into established software engineering practices.

5. Architecture and Security Considerations

A code-executing agent does not mean allowing the model to run arbitrary code without restrictions. A production implementation should carefully design the following components:

1. Tool-code registration layer: Expose tool servers as typed SDKs or function libraries with clear parameter schemas, return structures, and version information.
2. Sandboxed execution layer: Run model-generated code in an isolated environment with limits on CPU, memory, execution time, and file-system access.
3. Permission and policy layer: Grant only the minimum permissions required for each task. Sensitive operations should require approval. Credentials should be managed by the execution environment rather than directly exposed to model-generated code.
4. Network egress controls: Restrict the external addresses that the runtime can access to prevent data leakage and unauthorized service access.
5. Auditing and observability: Record the code version, tool calls, input and output summaries, errors, and execution times for tracing and review.
6. Result-return strategy: Return only the summaries, status codes, or references needed for the model’s decisions, rather than sending large results back into the context.

In cloud deployments, general-purpose container or function-based sandboxes can be combined with object storage, logging, monitoring, alerting, and secrets-management capabilities. For example, capabilities provided by Baidu AI Cloud—including general-purpose container platforms, object storage, logging and monitoring, and secrets management—can be used to host the execution environment, store large objects, record call chains, and manage sensitive credentials.

The core principles remain least privilege, isolated execution, and end-to-end auditing.

Security risks must also be treated seriously. Model-generated code is effectively untrusted input. Systems should therefore use strong sandbox isolation, resource limits, network allowlists, input validation, timeout and retry limits, idempotency controls, and human approval workflows by default.

High-risk operations such as deletion, payments, and bulk updates should require a second confirmation or human approval. They should not rely solely on the model’s judgment.

6. Best Practices for Tool API Design

Code execution places higher demands on tool API design. The following principles are recommended:

• Use an appropriate interface granularity: Avoid functions that perform too many unrelated operations, while also avoiding excessive fragmentation that makes orchestration unnecessarily complex.
• Return references instead of complete payloads: For large files and datasets, return an object reference or pagination cursor and retrieve summaries only when needed.
• Define stable types: Parameters and return values should have clear, stable type definitions. This helps the model generate correct code and supports static analysis.
• Standardize error codes: Distinguish retryable errors, permission errors, parameter errors, and business conflicts so that the runtime can handle them deterministically.
• Support idempotency and retries: Update operations should support idempotency keys to prevent duplicate writes during retries.
• Manage versions: API changes should include compatibility strategies so that previously generated code does not fail unexpectedly.
• Return structured summaries: After execution, return a structured summary to the model instead of the complete raw data.

7. Appropriate Use Cases and Conclusion

Code-executing agents are particularly suitable for multi-tool orchestration, large-file processing, batch loops, complex conditional logic, and execution chains that require auditing.

Traditional tool calling remains useful for simple single-tool operations, exploratory question answering, and high-risk tasks that are difficult to sandbox. In practice, systems can adopt a hybrid model: use direct tool calls for simple steps, delegate complex orchestration to code execution, and introduce approval workflows for high-risk operations.

The significance of this approach is not simply the introduction of a new concept. It is the redistribution of responsibilities within an agent system:

• The model understands the goal.
• The model generates orchestration logic.
• The model validates the result.
• The execution environment handles deterministic data flow and tool execution.

Moving control flow into code while reserving context for decision-making provides a clear path toward more efficient, reliable, and engineering-oriented agent systems.