How Code Execution Mode Improves Agent Tool-Calling Efficiency

In agent systems, “being able to call tools” is often treated as a core capability. In production, developers quickly find that the issue is not only whether the model can choose a tool, but the context cost of the tool-calling path. The usual pattern is to inject every tool definition into the context window. The model reads a large set of definitions, picks a tool, the result is written back into context, and the model decides the next step. Single-step tasks can still be acceptable. Multi-step tasks quickly become a token sink.

This article discusses a simple but effective shift: wrap tool servers as code APIs so the model does not call tools one by one, but generates code that orchestrates those calls. This is not merely a different interface. It changes how an agent expresses a task. The examples below are illustrative and do not refer to any specific product interface or performance claim.

Why traditional tool calling grows so quickly

Suppose an agent is connected to both a document library and a CRM, each exposing dozens of tools. For the model to choose among them, names, descriptions, and parameter schemas are often all injected into context. Tool definitions alone can consume a large number of tokens. If the task also reads long documents or writes business records, cost grows further.

Consider a hypothetical task: download a meeting transcript from the document library and write it into a CRM sales record. A traditional flow might look like this:

1. The model reads tool definitions and selects a “get document” tool.
2. The tool returns a two-hour transcript of about 50,000 tokens.
3. Those 50,000 tokens are loaded into the model context.
4. The model then places that content into an “update record” tool call, producing another ~50,000 tokens.
5. After the tool runs, the model continues with the next decision.

A seemingly simple task can consume more than 100,000 tokens and wait for multiple model generations. The root problem is not the tools themselves, but large intermediate objects repeatedly entering the model context. The model is forced to “carry” data it does not need to understand word by word, which is slow, expensive, and more likely to lose focus as context grows.

What changes in code execution mode

The new idea is to treat each tool server as a library or SDK. The model no longer selects tools directly. It writes a short program that calls those APIs:

// Illustrative pseudocode; not any specific product interface
import * as docs from './servers/doc-service';
import * as crm from './servers/crm-service';
const transcript = (
  await docs.getDocument({ documentId: 'doc_abc123' })
).content;
await crm.updateRecord({
  objectType: 'MeetingNote',
  recordId: 'rec_12345',
  data: { Notes: transcript }
});

Here the model only generates orchestration logic. Document content flows at runtime from the document service to the CRM service and never fully enters the model context. The model only needs a result summary, such as “updated 1 record, document length about 50,000 tokens, no errors.” Context then holds the plan, the code, and the summary—not the full transcript.

Three changes matter:

• Tool definitions are shown on demand: the model sees code interfaces, types, and a few examples rather than full descriptions of every tool.
• Intermediate data does not pass through context: large objects move inside the sandbox runtime; the model handles only necessary fields and summaries.
• One generation can orchestrate multiple steps: loops, conditions, filtering, aggregation, and retries can live in code, reducing round trips.

Efficiency comes from architecture, not “the model suddenly codes better”

Code execution is often misread as “stronger coding ability therefore better tool use.” A more accurate view is that programming becomes the language of tool orchestration. Describing “fetch this, then decide that, then write this” in code is more compact than deciding in natural language turn by turn.

Concrete benefits include:

1. Compressed context: full tool definitions and intermediate results are no longer injected. Gains are especially large for long documents, large tables, and batch records.
2. Fewer generation rounds: one snippet can include multiple calls, loops, and branches, so the model does not reread context at every step.
3. Better testability: generated code can be logged, reviewed, replayed, and unit-tested. Multi-turn tool calling is more like a black-box conversation and harder to reproduce.
4. Clearer error handling: try/catch, retries, timeouts, and idempotency checks can live in code. On a write failure, the program can query record state before retrying instead of having the model guess again.
5. Richer dataflow: filtering, mapping, pagination, and aggregation can run in the runtime; the model receives structured summaries.

This does not fully replace traditional tool calling. Simple one-step tasks, tool exploration, and open-ended cases that need real-time model judgment may still fit direct calls. Code execution is a high-density path for complex multi-step work.

Engineering issues that must be solved

Turning tools into code APIs raises the bar for security and governance. At minimum:

• Sandboxing: code must run in isolation with limits on CPU, memory, duration, and network; only allowlisted tool services by default.
• Least privilege: short-lived credentials and scoped access, with read/write separation. High-risk writes should add approval or a second confirmation.
• Data privacy: redact sensitive fields; keep large objects in the runtime; avoid logging full business payloads.
• Observability: record code versions, call chains, latency, failure reasons, and return summaries for audit.
• Stable interfaces: SDKs need clear types, pagination, rate limits, idempotency, and error codes so generated code is less brittle.
• Fallback: if execution fails or the task is uncertain, fall back to traditional tool calling or human review.

A practical flow: the model first produces a plan, then calling code; the sandbox runs the code and returns a structured summary; the model then completes, retries, or issues follow-up queries. For writes, dry-run or validate before commit.

Scope and implementation advice

Code execution fits multi-step, batch, large-data tasks that need loops, filters, and aggregation—document sync, record cleanup, query-then-summarize, cross-system state checks. Simple one-shot queries may still be better as direct tool calls. High-risk irreversible operations need approval and idempotency.

Start with low-risk reads, then expand to controlled writes. Stabilize tool interfaces first, then add sandbox execution and result summaries, while keeping traditional calling as a fallback. The goal is not “the model writes code for everything,” but letting the agent use a denser way to complete work and avoid pointless context shuttling.

In short, code execution improves tool-calling efficiency not because of a single model skill, but because orchestration moves from round-trip context passing to runtime code execution. When large objects no longer repeatedly cross the model context, agents can keep lower cost, fewer rounds, and more controllable engineering behavior on complex tasks.