Code Execution for Agent Tool Use: Lower Token Consumption and Higher Efficiency

A recent engineering article proposes a simple but powerful optimization for AI agent architectures: instead of selecting and calling one tool at a time, let the model write code that calls tools. The idea is to turn external tool servers into importable code libraries, so the model no longer enumerates tools one by one but writes a program to get the job done. The headline may sound simple, but it targets one of the most expensive structural inefficiencies in current agent designs.

To see why this matters, consider the typical tool-calling loop in an agent:

1. On every request, the system places the names, descriptions, and parameter schemas of all connected tools into the context window.
2. The model reads the definitions, decides which tool to use, and generates one tool call.
3. The result of the tool execution is written back into the context.
4. The model reads the result, decides the next step, and returns to step 2.

This loop is tolerable in demos. In real enterprise systems, it quickly becomes costly.

1. Fixed cost of tool definitions

Assume an agent is connected to two kinds of external systems: a document collaboration platform and a customer relationship management system, each exposing dozens of tools. A single tool’s name, purpose description, and parameter schema often consume hundreds of tokens. Across dozens or hundreds of tools, the “tool catalog” alone can easily consume tens of thousands of tokens. In enterprise environments with dozens of servers and hundreds of tools, the scale grows further. Worse, this catalog must be paid for again on every turn and every tool call.

2. Intermediate results pass through the model

Use the classic example from the article: the task is to download meeting notes from the document system and attach them to a sales lead in the CRM system.

• The model calls the document-reading tool. The tool returns the full transcript of a two-hour meeting, assumed to be about 50,000 tokens.
• Those 50,000 tokens enter the context.
• The model then copies those 50,000 tokens into the parameters of the write API.
• That creates another 50,000 tokens of consumption.

A task that is essentially “copy data from A to B” consumes 100,000 tokens and requires the model to generate twice. In this case, the model is not a reasoner; it is an expensive, error-prone, pay-per-token data pipeline.

3. Latency grows with round trips

Every tool call creates a serial round trip: the model thinks, the tool executes, the result returns, and the model thinks again. A ten-step task stacks ten generation latencies, which users experience as the system becoming slower over time.

4. Lack of expressive power

Natural-language tool loops cannot express control flow elegantly. Logic such as “retrieve only records from the last thirty days whose titles contain ‘budget’ and whose transcripts exceed one thousand words, batch-write them, and group them by owner” would be broken into many tool calls. Most of those calls add no reasoning value; they only move data.

II. The core idea: three replacements

The new approach is not mysterious. It translates the “tool protocol” into a “programmable interface” through three replacements.

Replacement 1: Injecting all definitions → on-demand imports and progressive disclosure

The model no longer sees the full schema of every tool at the start. Instead, it first sees a minimal “capability index” containing only the names of tool servers, their general responsibilities, and the import paths of the corresponding code libraries. When needed, it imports a specific module and inspects a specific signature.

Replacement 2: Model calls tools directly → model writes code to call tools

The model produces executable code:

// Import the code libraries for the two external systems on demand.
import * as docs from './servers/doc-store';
import * as crm from './servers/customer-crm';
// Complete the task directly without the model moving data around.
const transcript = (await docs.getDocument({ documentId: 'abc123' })).content;
await crm.updateRecord({
  objectType: 'SalesMeeting',
  recordId: '00Q5f000001abcXYZ',
  data: { Notes: transcript },
});

In this code, transcript is an ordinary in-process variable. The 50,000-token meeting transcript flows from reading to writing without passing through the model context. The model’s token consumption is proportional to the few lines of code it writes, not to the size of the data it is handling.

Replacement 3: Writing intermediate state back into the model → in-process variables and control flow

Filtering, aggregation, sorting, grouping, retries, conditional branches, and batch loops are all handled by code. This is the key way the model’s language capabilities are amplified: models are good at writing programs, so they should not be forced to act as data movers.

III. Where the benefits come from

This optimization is widely discussed because its benefits are not mysterious; they have clear sources.

• Definition cost drops from “total number of tools” to “number of symbols actually used.” Complexity changes from O(all tools) to O(used symbols). The benefit becomes more obvious as tool integrations grow.
• Large intermediate data no longer counts against model tokens. Data moves inside sandbox memory, and only necessary summaries, assertions, or statistical results are returned to the model.
• Round trips drop from N to 1. A ten-step chain of tool calls can be compressed into one code-generation step, reducing latency by orders of magnitude.
• Errors can be retried locally. If code execution fails, only the failed branch needs to be rerun; the model does not have to replan the whole task.
• The context stays cleaner for longer. Attention is no longer diluted by large raw payloads, and subsequent reasoning accuracy usually improves as well. In practice, this can be more valuable than the saved cost.

IV. Six key engineering decisions for production

Turning this idea into a production system requires more than a prompt. It needs supporting engineering infrastructure.

1. Make tool descriptions machine-readable assets. Tool documentation should be maintained as machine-readable signatures and type declarations, serving as the single source of truth. From this source, generate two views: a “capability index” and a “full reference.” If tool descriptions are unclear, the generated code will almost certainly be wrong.

2. Use an execution sandbox. The code-execution environment must provide process isolation, CPU and memory limits, execution timeouts, and temporary filesystem isolation. This is a non-negotiable security prerequisite: the model generates executable code, not merely a suggestion.

3. Apply credentials and least privilege. Tokens should not be exposed to the model or written into generated code. Instead, the execution environment should inject credentials at runtime. Authorization should be scoped to individual tools, and write operations should require explicit confirmation or approval by default.

4. Control network egress. Use allowlists for external system domains and prohibit arbitrary outbound network access, so the sandbox cannot be used as a data-exfiltration channel.

5. Make writes idempotent and replayable. Write-oriented tools should require idempotency keys. Execution traces should be persisted so that a failed task can be replayed rather than rerun from scratch.

6. Provide observability. Log the tool name, latency, return-value size, and token consumption for each step. Turn “how much data was placed into the context” into a measurable metric. Logging, metrics, and alerting can reuse mature general-purpose components from cloud platforms, such as log services and monitoring/alerting tools. Sandbox orchestration and result persistence can also integrate with general cloud capabilities such as object storage, message queues, and container platforms. Managed cloud platforms provide these kinds of general-purpose infrastructure building blocks, and agent frameworks only need to connect through standard interfaces.

The result-return strategy also deserves separate design. After code execution finishes, do not let the full raw return value go back into the context. A better practice is for the model to explicitly declare what it needs to know at the end of the code, for example: only the number of successfully written records, plus the IDs and error codes of failed items.

V. Limitations that must be acknowledged

Any engineering judgment should be honest: this approach is not unconditionally better.

• It significantly increases the requirement for model coding ability. Generated code may call nonexistent interfaces or guess parameter types incorrectly. Mitigation includes type constraints, static checks, pre-execution compilation validation, and strictly limiting the model’s visibility to real signatures.
• Debuggability becomes worse. Errors change from “the tool returned an error” to “a runtime exception occurred.” Stack traces and execution-trace replay must be supported.
• Small tasks become heavier. For single-step, single-parameter tool calls, the native tool protocol is simpler. A hybrid mode is recommended: lightweight tasks use native tool calling, while multi-step tasks involving data transformations or loops use code execution.
• The permission model becomes more complex. Once the model can write control flow, the permission decision point moves from “a single tool” to “an entire piece of code,” requiring more fine-grained policy enforcement.

VI. Why coding capability matters so much for agents

This technical direction also explains a long-standing phenomenon: when agents are given the same external tools, models with stronger coding ability often achieve higher task completion rates. The reason is not that they are “smarter.” The task format has changed. Instead of “select a tool and fill in parameters,” the model must design a correct program with error handling and type constraints. This requires long-chain syntactic correctness, variable tracking, reasonable inference about API semantics, and the ability to debug its own generated code.

The practical implication for teams is that model selection should not rely only on general benchmark scores. Evaluate models on your own real toolset at small scale and measure three indicators: the first-run success rate of generated tool-calling code, token consumption per task, and end-to-end latency. These are more persuasive than any abstract ranking.

VII. Implementation checklist

• Inventory existing tools and rank them by “definition token size × call frequency.” Prioritize the most expensive integrations.
• Create machine-readable signatures for tool servers and generate tiered views: an index and a full reference.
• Build a sandbox with resource limits and network allowlists. Start with read-only access, then enable writes.
• Enforce result truncation: define the maximum return size and summarization strategy for each tool.
• Build a token and latency dashboard, and establish a comparable baseline before and after optimization.
• Use hybrid routing: dynamically choose native tool calling or code execution based on task step count and data volume.

In one sentence: this shift changes agents from “reading the entire toolbox before tightening a single screw” to “opening the toolbox and writing a script to finish the job.” Context is a scarce resource, and code execution is currently one of the most economical ways to manage it.