Why “Stuffing Everything into Context” Drags Agents Down

Most agents today work like this: descriptions of all available tools are written into the prompt once, the model reads the full list and picks one to call; the tool’s result is written back into the same context, and the model decides the next step. The loop repeats.

This pattern is fine when you have a few tools with small responses. But once you connect multiple business systems, each exposing dozens of operations, the tool list alone consumes a large chunk of the context. If one step returns a long document, a detailed table, or a call transcript, that entire payload has to pass through the model again before it can be used as arguments for the next call. Typical problems emerge:

• Tool definitions fill the context, squeezing out space for task goals, constraints, and intermediate reasoning.
• Intermediate data must “pass through the brain”: large blocks of raw text flow in and out of the model, with cost and latency rising linearly with data size.
• Control flow can only be described in natural language: intents like “loop over each item” or “retry on failure” are easy to misinterpret, and branching, batching, and exception paths are hard to express precisely.

So the efficiency problem isn’t just “the model isn’t strong enough” — it’s poorly designed tool exposure and data flow paths.

Protocol-Based Integration: Turning External Systems into Composable Capabilities

The industry has widely adopted open tool protocols: a standalone process exposes a set of standardized capabilities, and the agent connects to external systems — file systems, business databases, office documents, customer records — through a uniform calling convention. The protocol itself isn’t tied to any particular model. The key points:

1. Capabilities are decoupled from the model: who implements the server and who consumes the capabilities can evolve independently.
2. Permissions and auditing become practical: each server can handle its own authentication, rate limiting, and logging.
3. Composition over accumulation: real value comes from multiple systems working together, not from an ever-longer list of individual tools.

One caveat: the protocol only solves “how to connect.” It doesn’t automatically prevent the context from blowing up afterward. If you still dump all interface documentation into the prompt at once, protocol-based integration just means more tools and a bigger black hole.

The Key Shift: Treat Tools as a Code Library, Not a Prompt Appendix

The more efficient path is: don’t make the model face a flat list of tools. Instead, map each tool server to an importable code module, and let the model write short programs that perform the orchestration in an isolated execution environment.

A sketch of what this looks like (interface names are generic examples, not specific to any product):

# Load modules as needed, instead of preloading all definitions
from integrations import docs as doc_api
from integrations import crm as crm_api
transcript = (await doc_api.get_document(document_id="...")).content
await crm_api.update_record(
    object_type="sales_meeting",
    record_id="...",
    data={"notes": transcript},
)

Compared with “the model reads the full meeting transcript, then passes that same text as arguments to generate a second call,” the differences are:

• Long text never enters the model’s context — it moves only through the execution environment’s memory or temporary storage.
• The model generates control logic (which document to fetch, which field to write) rather than reproducing 50,000 words of source text.
• A single program expresses sequencing, conditions, and error handling, reducing the back-and-forth of “look at the result, then decide the next step.”

A Three-Dimension Comparison

Dimension	Traditional Tool Calling	Code Execution Orchestration
How tools are exposed	All definitions written into context	Read on demand from modules/file system
How intermediate data flows	Everything passes through model context	Passed internally within the execution environment
How control flow is expressed	Natural language instructions	Variables, loops, branches, exceptions

“On-demand reading” means: only the signatures of modules the current task might use enter the prompt; the rest stay in a searchable directory or documentation that the model consults while writing code. This mirrors how human developers work — look at the project structure first, then import the specific packages.

Separating the Data Plane from the Control Plane

Agent interaction can be split into two layers:

Control plane: The model understands the goal, picks modules, writes orchestration code, and explains failures. What it needs here is interface shapes, business constraints, and a few examples — not the full business data.

Data plane: Document bodies, table rows, and binary attachments flow through the sandbox, object storage, or message channels. The execution environment handles downloading, truncating, transforming, and batch writing; the model only receives summaries, validation results, or error codes.

Once separated, the cost structure of common tasks changes. Take “sync meeting notes to the customer record”: the traditional path is often “pull the full text into context, then generate an update call with the full text as a parameter” — the same large object gets processed by the model twice. The code path is one I/O operation and one write; the model side only generates a few dozen lines of logic.

The effect is even more pronounced for batch processing: looping over a hundred records inside the execution environment is far more stable than asking the model in natural language to “process each one” across a hundred round-trip calls, and it’s much easier to apply uniform retry and idempotency logic.

Engineering Capabilities You Must Add

Simply treating “let the model write code” as a trick isn’t enough. Without supporting mechanisms, you’ve just traded a token problem for a security and operability problem.

1. Least-privilege execution sandbox
Code can only call authorized modules, with no network access and no arbitrary file writes by default. Write operations against external systems should require confirmation policies or change tickets, so one bad generated script can’t cause irreversible updates.

2. Tool catalog with progressive disclosure
Provide a module index (name, one-line responsibility, input/output types), with detailed parameter documentation expanded on demand. Avoid permanently stuffing every field description into the system prompt “just in case the model doesn’t know.”

3. Result trimming and structured returns
The execution environment should return status, affected row counts, and validation summaries — not paste raw payloads back into the conversation. When the model does need to read content, extract, chunk, or retrieve first, then feed only the key fragments.

4. Observability and replay
Log which modules were imported, the actual call sequence, timing, and errors. When something goes wrong, you should be able to replay the same piece of code rather than just reading the conversation transcript, making it easier to distinguish an API failure from an orchestration logic bug.

5. Coexistence with existing integration patterns
Not every step belongs in a program. Exploratory Q&A and classifications that require semantic understanding can still use single tool calls. Code execution is best suited for data movement, batch transformation, and transactional orchestration across multiple systems.

Why This Improves Stability on Complex Tasks

Once control flow is expressed in code, requirements that were previously hard to articulate in a prompt become executable: timeouts and retry counts, compensation on partial failure, field mappings and null handling, validate-before-commit. These are routine software engineering structures — they shouldn’t depend long-term on the model “remembering an operations manual.”

Meanwhile, the context window can be reserved for higher-value information: business rules, user preferences, a few critical examples. The tool list is no longer coupled to task difficulty, so as you onboard more systems, the cost curve doesn’t simply explode as “number of tools × definition length.”

Criteria for Deciding Whether to Adopt It

Evaluate whether a code execution middleware layer makes sense using a few testable criteria, not slogans:

• In a single task, does it frequently happen that “the tool response is far larger than the fields you ultimately need to write back”?
• Do you often need to loop, filter, or stitch together results from multiple systems?
• Has the total size of tool definitions approached or exceeded the task description itself?
• When something fails, can you distinguish a business validation failure from the model miscopying a large block of text?

If most answers are “yes,” prioritize moving high-frequency, high-volume, strongly structured paths down into the execution environment, and let the model focus on generating and fixing the orchestration code.

Summary

A large part of the gap in agent programming ability comes from how tools are used, not how long the tool list is. Protocalizing external systems into modules, keeping intermediate data inside the execution environment, and expressing control flow as code can simultaneously reduce token consumption, cut round trips, and improve controllability for batch and exception scenarios. Implementation must be paired with sandboxing, progressive disclosure, result trimming, and auditing to turn “a seemingly smarter way of calling tools” into a production-ready engineering practice. When deploying in the cloud, this typically combines with object storage, logging services, monitoring and alerting, and containerized execution environments to ensure isolation, observability, and elasticity — without dumping all business data into the model’s context.