Connecting Heterogeneous LLMs to AI Coding Tools: A Practical Guide to Protocol Adapter Layers

Introduction: Why “Just Switch the Model” Is Harder Than It Sounds

The ideal setup for many developers looks like this: a coding assistant with solid context engineering and a pleasant day-to-day developer experience, paired with the freedom to swap the backend model at will — a domestic model with a free tier for routine completions today, a stronger model for the hard problems tomorrow. It sounds like flipping one config flag. In practice, people get stuck: either the tool refuses to connect, or it connects and then throws errors constantly.

The root cause is neither the network nor the API key. It is that the protocols don’t agree. An AI coding tool only knows how to issue requests in one particular model API specification, while many cost-efficient models expose an entirely different dialect. They disagree on the request path, the authentication header, the message body structure, parameter names, streaming events, and even the tool-calling format. Making them work together requires a dedicated layer of translation: a protocol adapter / gateway.

This article does not walk through the setup steps for any specific tool. Instead, it explains the general technical problem behind that translation layer — because once you understand the protocol differences, supporting any provider is just a matter of writing another set of mappings.

1. Root Cause: A Breakdown of the Two Dominant Model API Specifications

The LLM APIs on the market fall into roughly two styles. One exposes the conversation endpoint under /v1/messages (the “Messages spec” below); the other uses /v1/chat/completions (the “Chat Completions spec”). Both carry the same underlying idea — multi-turn conversation — but the way they express it differs enough that requests simply cannot be exchanged between them.

1.1 Request Path and Authentication

The most superficial difference lives at the HTTP layer:

• Messages spec: requests go to /v1/messages, credentials sit in a custom header such as x-api-key, and a protocol-version header (something like anthropic-version) is usually required to negotiate behavior.
• Chat Completions spec: requests go to /v1/chat/completions, and authentication follows the standard Authorization: Bearer <token>.

If the gateway does not handle this layer, requests land directly on a 404 or 401. The adapter’s first job is therefore to rewrite the URL path and the authentication header.

1.2 Message Body: Where Does the System Prompt Live?

This is where people trip up most often, and it is also where the philosophies of the two specs diverge most clearly.

The Chat Completions spec stuffs every role into the messages array, and the system prompt is just a message with role: "system":

{
  "model": "<target-model>",
  "messages": [
    {"role": "system", "content": "You are a senior engineer"},
    {"role": "user", "content": "Write a quicksort for me"}
  ]
}

The Messages spec promotes the system prompt to a top-level standalone field, system, and leaves only the user / assistant roles inside messages:

{
  "model": "<target-model>",
  "system": "You are a senior engineer",
  "max_tokens": 1024,
  "messages": [
    {"role": "user", "content": "Write a quicksort for me"}
  ]
}

When adapting, you must lift the top-level system out and turn it into a system message (or merge it back into the top level in the reverse direction). Otherwise the model “loses” the entire system instruction, which shows up as an abrupt change in answering style or as constraints being ignored.

1.3 Content Blocks and Parameter Naming

In the Messages spec, content can be a structured array of content blocks, for example [{"type":"text","text":"..."}], leaving room for multimodal material (text, images, tool results interleaved). The Chat Completions spec historically used just a string or fairly flat content parts. The adapter needs a flatten / expand normalization pair to convert in both directions.

Parameter naming is equally divergent:

• max_tokens is required in the Messages spec; in the Chat Completions spec it is usually optional, and newer versions lean toward max_completion_tokens.
• Stop sequences: one side calls them stop_sequences, the other calls them stop.
• stream, temperature, and similar parameters mean the same thing on both sides and can be passed straight through.

Omitting the required max_tokens field is the most common source of 400 errors for newcomers integrating the Messages spec.

1.4 Tool Use / Function Calling

What makes an AI coding tool useful is letting the model call external tools — reading and writing files, running commands — and the two specs express “tool calling” in almost entirely different ways:

• Chat Completions spec: a tool definition looks like {"type":"function","function":{"name","parameters"}}; the model returns tool_calls; tool results are sent back as a message with role: "tool".
• Messages spec: a tool definition is simply {"name","input_schema"}; the model returns a content block with type: "tool_use" (carrying id, name, input); results are sent back as a content block with type: "tool_result", paired with the original call via tool_use_id.

The adapter has to maintain bidirectional mappings for both tool definitions and tool results, and in streaming mode it must correctly assemble the incrementally delivered partial_json fragments into complete arguments before handing them to the coding tool — the single most likely place for subtle bugs in the whole adapter.

1.5 Streaming Output and the Event Model

Whether the coding experience feels good depends heavily on how smoothly streaming (SSE) works, and the two specs model streaming events very differently:

• Chat Completions spec: a flat stream of choices[0].delta.content text increments, with finish_reason marking the end.
• Messages spec: a structured event stream — message_start → content_block_start → content_block_delta → content_block_stop → message_delta → message_stop.

So the gateway cannot simply pass raw bytes through; it has to perform event-level translation: wrap the simple delta increments into the full event sequence the other side expects, and align finish_reason with stop_reason values such as end_turn, max_tokens, tool_use, and stop.

2. Designing the Protocol Adapter Layer

Once the differences are clear, the skeleton of the adapter is straightforward: it is a bidirectional translation layer. Taking the case of “a coding tool that speaks the Messages spec calling a model that speaks the Chat Completions spec”, the data flow is:

coding tool --(Messages request)-->  adapter gateway --(Chat Completions request)--> target model
coding tool <--(Messages response)--  adapter gateway <--(Chat Completions response)--  target model

Externally the gateway impersonates the protocol the tool already understands; internally it translates into the protocol the target model understands. The core is two functions — request_inbound → request_outbound (request conversion) and response_outbound → response_inbound (response conversion) — with one variant each for streaming and non-streaming.

A skeleton of the request conversion looks like this:

def to_chat_completions(messages_req: dict, target_model: str) -> dict:
    messages = []
    # 1) top-level system field -> a message with role=system
    if messages_req.get("system"):
        messages.append({"role": "system", "content": messages_req["system"]})
    # 2) normalize content: concatenate structured blocks into plain text
    for m in messages_req["messages"]:
        messages.append({
            "role": m["role"],
            "content": flatten_content(m["content"]),
        })
    return {
        "model": target_model,
        "messages": messages,
        "max_tokens": messages_req.get("max_tokens", 1024),  # fill in the required field
        "temperature": messages_req.get("temperature", 1.0),
        "stream": messages_req.get("stream", False),
    }
def flatten_content(content):
    if isinstance(content, str):
        return content
    return "".join(
        b.get("text", "")
        for b in content
        if isinstance(b, dict) and b.get("type") == "text"
    )

3. Implementation Details and Engineering Pitfalls

Once it ships, a handful of details decide whether the adapter is merely usable or genuinely good:

1. Enumerate the stop-reason mappings. Turn the stop_reason ↔ finish_reason correspondence into an explicit lookup table covering end_turn ↔ stop, max_tokens ↔ length, tool_use ↔ tool_calls, and so on. Miss one branch and the client may hang on an endless spinner, or silently truncate an answer without telling the user.

2. Streaming tool-call assembly. In the Messages spec, tool arguments arrive as partial_json fragments and must be accumulated by content_block_index into complete JSON before the coding tool can execute them; in the Chat Completions spec, function.arguments are accumulated by index inside tool_calls. This logic deserves its own unit tests, because when it goes wrong it does not raise an error — it quietly produces truncated arguments.

3. Align error codes and exception semantics. Rate limiting (429), 5xx failures, and authentication failures from the target model need to be translated into an error structure the coding tool can recognize, while preserving the original error information for debugging. Never pass upstream errors through verbatim, or the tool side will only see an unparseable blob of bytes.

4. Token accounting and context-window differences. Context limits differ across models. The adapter should ideally do a coarse estimate and trim at the entry point, so that a smaller window on the target model does not cause an outright error or silently drop the system prompt.

5. Model capability alignment. Not every model supports the same tool-calling or multimodal features. The adapter should let you declare a capability set per target model and degrade gracefully when a feature is unsupported (for example, falling back to plain text) rather than failing hard.

As for where the backend model comes from: mainstream cloud platforms, including Baidu AI Cloud, now expose model services through relatively standardized interfaces. As long as the target model’s API fits one of the two specs above, writing the corresponding mapping is enough to integrate it. That is exactly why “understanding protocol differences” has more long-term value than “memorizing one configuration”: once the protocol layer is solved, model selection is genuinely free.

4. Conclusion

The journey from “I want to use the free quota” to “protocols are the real issue” reveals a fact that is often overlooked: what makes models interchangeable in an AI coding tool is not the API key or the quota, but semantic incompatibility between protocols.

Three takeaways: first, the two dominant model API specs differ across six layers — path, authentication, message structure, parameters, tool calling, and streaming events — and a missing mapping at any one of them produces a tool that connects but cannot be used. Second, the right fix is a bidirectional protocol adapter gateway that impersonates the tool’s protocol outward and converts inward, covering the whole path with two conversion functions (plus their streaming counterparts). Third, the engineering difficulty concentrates in streaming tool-call argument assembly, stop-reason mapping, and error-semantics alignment — the “no error raised, but the result is wrong” class of problems — which is why unit tests are essential.

Once you understand this layer, you are no longer locked to a single vendor: switching models, running canary comparisons, or picking a model with a free tier in cost-sensitive scenarios all come down to changing one set of mappings.