From Function Calls to Code Execution: A More Efficient AI Agent Paradigm

1. The Overlooked Context Cost: Why Traditional Tool Calling Is Expensive

In many current AI agent implementations, tool calling follows a “full definitions + step-by-step interaction” pattern. At startup, the system injects the definitions of all available tools, including names, parameters, and descriptions, into the model’s context window. The model reads all the definitions, chooses a tool, receives the tool result back in context, and then decides the next action. This loop repeats until the task is complete.

This approach is manageable when only a few tools are involved. As more systems are connected, however, the cost grows rapidly. If an agent connects to a document storage service and a customer relationship management system, each side may expose dozens of tools. The tool definitions alone can consume tens of thousands or even hundreds of thousands of tokens. More importantly, intermediate data must also cross the model context repeatedly: the model first generates tool parameters, then reads the complete tool result, and then generates the next action based on that result. Data is repeatedly moved between generation and reading.

A typical task illustrates the issue. Suppose the agent needs to download a meeting record from a document service and save it as a sales lead in a business system. Under the traditional approach, the model first calls the document API. The API returns the full transcript of a two-hour meeting, say around 50,000 tokens, and that content enters the model context. The model then needs to pass this content to the business system, so it effectively has to regenerate roughly another 50,000 tokens. A task that looks like a simple transfer operation may therefore consume more than 100,000 tokens and involve two full model generation waits.

The underlying problem is that the model context is responsible not only for decision making but also, unintentionally, for data transport. When the data volume is large and the workflow has many steps, token usage and latency can quickly become uncontrollable.

2. A Paradigm Shift: Turning Tools into Callable Code APIs

The core idea of the new approach is to change external tools from interfaces that the model selects directly into libraries that can be imported and called from code. Instead of choosing tools one by one, the model writes code to complete the task, similar to how a developer would write an ordinary program.

The following example illustrates the pattern:

// Import the code modules corresponding to the tools on demand
import * as docs from './tools/doc-store';
import * as crm from './tools/crm-system';
// Orchestrate the task using full programming constructs
const transcript = (await docs.getDocument({ documentId: 'abc123' })).content;
await crm.updateRecord({
  objectType: 'SalesMeeting',
  recordId: '00Q...XYZ',
  data: { Notes: transcript }
});

Under this model, tool definitions do not need to occupy the entire context. They can be read from the file system only when needed. More importantly, intermediate data, such as the meeting transcript in the example, can flow entirely within the code execution environment and does not need to pass through the model context.

3. A Systematic Comparison of the Two Paradigms

Dimension	Traditional Function Calling	Code Execution
How tools are exposed	All definitions are injected into context at once	Loaded on demand from the file system
How intermediate data moves	Passes through the model context throughout the workflow	Passed only inside the code execution environment
How control flow is expressed	Relies on natural-language prompts such as “please loop over…”	Uses full programming constructs: variables, loops, exceptions, and conditions

The table highlights more than token savings. It also reflects an expansion of capability.

4. Capability Gains from Code Execution

1. Implicit passing of intermediate results
   Under the traditional approach, if the output of tool A needs to become the input of tool B, it usually has to pass through the model: the model reads and understands the output, then expresses it again as the next tool input. This consumes tokens and increases the chance of error. With code execution, variables pass directly within the program. The model can focus on business logic rather than on moving data around.

2. Precise control flow
   Loops, conditional branches, and exception handling can be expressed ambiguously in natural language but can be defined precisely in code. For example, “download all meeting records and archive them one by one” is more reliable as a loop in code than as repeated natural-language instructions.

3. On-demand filtering of intermediate results
   If only one field from a document is needed, the code can filter or extract it and return only the concise result to the model. This avoids pulling a long document into the context unnecessarily.

4. Error handling and retries
   Programming constructs naturally support mechanisms such as try-catch, retries, and timeouts. These robustness mechanisms do not depend on the model “remembering” to handle exceptions in the prompt.

5. Privacy and data boundaries
   Sensitive data can be processed inside the execution environment, and only necessary conclusions need to be exposed to the model. This reduces the exposure of data in the model context.

5. Engineering Requirements for Production

For this paradigm to be reliable in practice, several engineering practices are important:

• Make tools code-ready: Wrap each tool as a function with clear type signatures and documentation so the model can understand how to call it rather than guessing from a natural-language description alone.
• Support on-demand loading: Use file retrieval or metadata indexing so the model reads a tool definition only when needed, avoiding full-context injection.
• Isolate execution securely: Code generated and executed by the model must run in a controlled sandbox. Limit file system access, network access, and permissions to prevent unintended operations or injection risks.
• Improve observability: Log, monitor, and alert on code execution so teams can track token consumption, execution duration, and failure causes. These capabilities can often be supported by general cloud capabilities such as logging and monitoring.
• Define a result-return strategy: Decide which intermediate results should be returned to the model and which should be consumed inside the execution environment, controlling context size from the source.

6. Not a Silver Bullet: Tradeoffs to Consider

The code execution paradigm is not without costs. It requires the model to have strong code-generation ability, and the generated code must be correct and safe. The execution environment also requires additional build and maintenance effort. Debugging changes from “inspecting prompts” to “inspecting code and runtime results.”

A more practical approach is often not to choose one pattern exclusively, but to mix them according to task characteristics. Code execution is better suited to tasks with many steps, large data volumes, or complex logic. For simple one-off calls, direct tool calling can still be sufficient.

7. Conclusion

Turning tools from a “menu the model selects from” into “functions in code” essentially relieves the agent from the burden of both making decisions and moving data. Decisions can be handled by the model, while data flow and control logic can be handled by code. This division can significantly reduce token consumption and latency, while programming constructs can improve the precision and robustness of task execution.

For teams building AI agent applications, understanding and adopting this paradigm is an important step toward improving context engineering efficiency.