Local Protocol Conversion Gateway: Bridging AI Agent Tools and Heterogeneous LLMs

The Problem: Protocol Silos and Model Lock-In

The current AI coding tool ecosystem exhibits a notable contradiction: on one hand, command-line agent tools keep emerging and growing more powerful; on the other, these tools are often deeply bound to specific LLM services, creating de facto “model lock-in.”

What makes things trickier is that different tools adopt different communication protocols. Some use the Anthropic Messages protocol, some use OpenAI Chat Completions, and others rely on the OpenAI Responses protocol or the Gemini protocol. When a developer wants to point a tool at a different model service, protocol incompatibility becomes the first obstacle in the way.

A natural idea follows: can we build a lightweight local gateway where all protocols converge, get converted, and get dispatched? That is exactly the technical approach explored in this article.

Solution Overview: One Gateway Plus One Configuration Manager

The solution consists of two core components.

Component 1: The local protocol conversion gateway

The gateway listens on a loopback address (for example, 127.0.0.1:3425) and simultaneously accepts all four API protocols:

• OpenAI Chat Completions
• OpenAI Responses
• Anthropic Messages
• Gemini

These four protocols can be converted into one another, and the conversion fully preserves key semantics such as streaming output, tool calling (Function Calling / Tool Use), and reasoning content.

In practice, this means an AI coding tool that only speaks the Anthropic protocol can call a model service that only offers an OpenAI-compatible interface, and vice versa. Protocol is no longer a constraint, and the freedom to combine tools with models increases substantially.

Component 2: The agent configuration manager

The gateway solves the communication problem, but there is another easily overlooked piece: how should the tool’s configuration file be written?

AI coding tools typically read local configuration files to determine the API endpoint, authentication credentials, and default model. Manually editing these files is not only tedious but also error-prone—especially when the file is TOML with extensive comments, where a single mistake can break the entire configuration.

The configuration manager’s value lies here: it offers model selection in a graphical interface. Once the user picks a model, it modifies only the few key-value pairs that the tool’s config file actually requires, while comments, key order, and indentation style are all preserved verbatim. Writes are also atomic, preventing file corruption if the process is interrupted midway.

Protocol Conversion in Detail

Semantic Mapping Across the Four Protocols

Although all four protocols serve the same purpose—interacting with LLMs—they differ in request structure, response format, and streaming event types. The gateway needs to handle the following core mappings:

Semantic aspect	Anthropic Messages	OpenAI Chat Completions	OpenAI Responses	Gemini
System prompt	system field	system-role entry in messages	instructions	systemInstruction
User messages	messages array	messages array	input array	contents array
Tool definitions	tools array	tools array	tools array	tools array
Streaming events	content_block_delta	choices[].delta	response.output_text.delta	candidates[].content
Reasoning content	thinking blocks	reasoning_content	reasoning field	thought field

When a request arrives, the gateway first identifies the source protocol, parses the request body into a unified internal representation, and then serializes it into the target protocol’s format before forwarding. Responses are processed the same way in reverse.

Preserving Streaming Output

Streaming output is central to the coding tool experience—responses that arrive token by token let developers see generated content immediately instead of waiting for the entire response. When converting between protocols, the gateway must process streaming chunks event by event, mapping each source protocol event type to the corresponding target protocol event type.

For example, when the source protocol is OpenAI Chat Completions and the target is Anthropic Messages, a choices[].delta.content event must be converted into a content_block_delta event while maintaining the correct event sequence numbers and indices.

Preserving Tool Calls

Tool calling (Tool Use) is the key capability that lets agents perform real tasks. The model returns tool names and parameters in its response, and the execution results are fed back to the model as input in the next conversation round.

The difficulty in protocol conversion is that protocols express tool calls differently. Some use a tool_calls array; others use tool_use blocks inside content. The gateway must identify and convert these structures precisely while keeping call IDs consistent, so that tool execution results can be correctly associated back with the original request.

How the Configuration Manager Works

Targeted Edits Instead of Full Rewrites

The configuration manager’s core design principle is simple: modify only the required key-value pairs and touch nothing else.

Take a JSON configuration file as an example. Suppose the original content is:

{
  // Authentication settings
  "env": {
    "ANTHROPIC_BASE_URL": "https://original-endpoint",
    "ANTHROPIC_AUTH_TOKEN": "original-key",
    "ANTHROPIC_MODEL": "original-model"
  },
  // All other settings remain unchanged
  "theme": "dark"
}

When making changes, the configuration manager does not re-serialize the entire JSON object (which would discard comments and formatting). Instead, it performs precise text edits, replacing only the values of the target keys. Comments, indentation, and key order are all preserved.

Atomic Writes

Configuration file writes are atomic: the manager first writes to a temporary file and renames it to the target file only after the write succeeds. Even if the process is interrupted unexpectedly, the original configuration file remains intact.

Where Configuration Is Written for Each Tool

Different AI coding tools store their configuration files in different locations and formats, so the configuration manager must handle multiple cases:

Tool type	Write location	What is modified
Claude Code-style tools	~/.claude/settings.json	ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, and model variables under env
Codex-style tools	~/.codex/config.toml	Adds a [model_providers.magpie] table and model; the model list is written to ~/.codex/magpie-models.json
Gemini CLI-style tools	~/.gemini/settings.json, ~/.gemini/.env	GOOGLE_GEMINI_BASE_URL pointing to the gateway, API key authentication, and model
OpenCode / Pi / Crush-style tools	Their respective config files	Adds a provider entry, with models written as magpie/provider/model

Typical Use Cases

Scenario 1: Letting an Anthropic-Only Tool Call Other Models

Suppose an AI coding tool only supports the Anthropic Messages protocol, but the developer wants to use a model service that only offers an OpenAI-compatible interface. With the local gateway:

1. The tool’s configured API endpoint points to the gateway (for example, http://127.0.0.1:3425)
2. The gateway receives requests in the Anthropic protocol and converts them into OpenAI Chat Completions format
3. The gateway forwards the request to the target model service
4. Upon receiving the response, it converts it back to the Anthropic protocol format and returns it to the tool

The entire process is transparent to the tool—it never knows which model is actually running on the backend.

Scenario 2: Driving a Responses-Protocol Tool with a Subscription-Based Model Service

Suppose a tool uses the OpenAI Responses protocol while the developer holds a subscription to a different model service. The gateway converts the Responses protocol into the protocol supported by the target service, completing the integration without modifying the tool’s code.

Scenario 3: Multiple Tools Sharing the Same Model Backend

Different team members may use different coding tools but want to standardize on a single model service. With the gateway, each tool only needs to point its API endpoint at the local gateway, which then routes requests to the corresponding backend based on the model identifier in each request.

Security and Operations Considerations

Network isolation: The gateway listens on the loopback address and is accessible only from the local machine, preventing exposure to the public internet.

Credential management: Authentication tokens are written into tool configuration files by the configuration manager; the gateway itself does not persist sensitive credentials.

Logging and observability: The gateway can log request metadata (protocol type, model, latency) to aid troubleshooting, but it should not log full request/response bodies to avoid leaking sensitive information.

Failure recovery: The configuration manager’s atomic writes guarantee configuration file integrity, and the gateway is stateless—simply restarting it restores normal operation.

Conclusion

The combination of a local protocol conversion gateway and a configuration manager addresses a foundational problem in the AI coding tool ecosystem: the mismatch between protocol diversity and model diversity. By abstracting protocol conversion and configuration management into independent components, developers can flexibly combine tools and models without modifying tool source code or being locked into a particular model service, gaining far greater freedom in technology choices.

The core value of this approach lies not in any single technique but in the decoupling idea behind it: communication protocols, model services, and developer tools should never be tightly bound in the first place. A lightweight local middleware layer can unlock considerable combinatorial flexibility. For teams that frequently switch models or need to manage configurations across multiple tools in a unified way, this approach is well worth considering in technology selection.