AI Response Verification: How to Detect Model Substitution in API Relay Services
When you call a third-party LLM relay service, you pay for one model but may unknowingly receive answers from a cheaper one swapped in along the chain. This article explains why naive checks—asking the model who it is, comparing writing style, or validating response fields—cannot reliably catch this. It then walks through verifiable inference mechanisms that turn a response into a verifiable object: request-response binding, cryptographic commitments and hash chains, digital signatures, verifiable computation and zero-knowledge proofs, and deterministic inference, illustrated with pseudocode. It also covers the real-world limits of these techniques and offers enterprises and developers practical guidance on vendor selection, full-chain logging, anomaly detection, cross-verification, and traceable agreements.
1. The Root Problem: Invisible “Swapping” Along the Relay Chain
When you use a third-party model relay service, callers typically care about only two things: I paid, and I got an answer. But between your request and the actual model, the request may pass through multiple relay nodes. Along this chain lies a risk that is not new but is hard to defend against: model substitution. You are billed at the rate for a high-tier model, but the relay node forwards your request to a cheaper model and passes the result straight back.
Why is this so hard to detect? Because “capability differences” are a fuzzy perception. For the same advertised model name, some responses are well-structured and rigorously reasoned, while others miss the point with loose logic. This gap may come from a model being swapped out, or from parameters being dialed down (for example, a lower reasoning budget, shorter context, or reduced thinking depth). To the caller, all of these look like “service degradation” that is difficult to attribute.
At a higher level, this risk is related to model capability distillation: replicating the behavior of a strong model onto another one through large-scale interaction. Whatever the motive, the direct harm to an ordinary caller is the same—you cannot confirm whether the response you received really came from the model you paid for.
2. Why Naive Detection Methods Fall Short
Faced with this problem, the most intuitive idea is to “just ask the model who it is.” But this is almost guaranteed to fail: as long as the relay node injects a line like “You are [some high-tier model]” into the system prompt, any substituted model will answer accordingly. Self-reported identity is therefore completely untrustworthy.
Several detection approaches common in the community each have their ceilings:
- Response style comparison: Judging model style by statistics on wording, sentence structure, and punctuation habits. But style can be induced by prompts and can drift with sampling parameters, leading to a high misjudgment rate.
- Return field validation: Checking whether certain fields in the response structure exist and are correctly formatted. Fields are protocol-level conventions that a knowledgeable relay node can fully forge.
- Reasoning-trace signature verification: Checking whether the signature on the model’s “chain of thought” is valid. This can prove the response came from a legitimate provider, but it cannot answer a key question—whether the model running behind the scenes is the top-tier model or a cheaper model from the same vendor.
In short, existing methods can at most verify that “the response did come from some official channel,” but they cannot verify that “the response was actually generated by the tier of model you paid for, with the parameters you expected.” To clear this hurdle, the problem needs to shift from “guessing style” to “verifiable cryptographic proof.”
3. The Core Idea: Turning “Responses” into Verifiable Objects
In recent years, the industry has accumulated several general-purpose technical components in the direction of verifiable inference. Their common goal is to make it impossible for a service provider to swap models or tamper with results without leaving a trace. The following mechanisms have been widely discussed in both academic and engineering practice and can be combined.
3.1 Request-Response Binding, Preventing Tampering and Replay
The first step is to bind each “request-response” pair into an inseparable relationship. The approach is to generate a one-time nonce for each request, then concatenate a canonical hash of the request content, the nonce, the model identifier, and the hash of the response content into a single whole. If anything is changed, the hash changes and is immediately detected. This both prevents a response from being replaced with a cached result from another call (replay) and prevents the content from being tampered with in transit.
3.2 Cryptographic Commitments and Hash Chains, Locking in Generation-Time State
When generating a response, the service provider can publish a commitment to a digest of the key reasoning parameters at that time (such as the random seed, sampling parameters, and model version number), and attach the corresponding hash value to the response. Multiple responses can also be chained together with hash links, so the caller can verify their order and integrity. The value of a commitment is that it makes it hard for the provider to deny things afterward—once a commitment is published, claiming to have used different parameters would conflict with that commitment.
3.3 Digital Signatures, Establishing Non-Repudiation of Origin
The service provider signs the whole thing—“request digest + model identifier + response hash”—with a private key, and the caller verifies it with the corresponding public key. A successful signature proves that the response was indeed generated by the party holding the private key and that the content and parameter digest were not tampered with midway. This step solves “trustworthy origin,” but it does not by itself solve “which tier of model was used”—so it needs to work with the mechanism below.
3.4 Verifiable Computation and Zero-Knowledge Proofs, Proving “It Was Computed Correctly”
To prove that “this response was indeed computed by the specified model according to the specified procedure,” the strongest tool is verifiable computation: the service provider attaches a succinct proof to the response, and the caller can verify the correctness of the computation without re-running the entire inference. The zero-knowledge proof is one representative form; it can also prove “I did execute this computation” without revealing model details. In engineering practice, because large-model inference is enormous in scale, the overhead of general-purpose zero-knowledge proofs remains high; more practical scenarios adopt a compromise of trusted execution environment (TEE) + remote attestation: performing inference inside the provider’s trusted hardware and outputting verifiable hardware-level attestation credentials.
3.5 Deterministic Inference, the Prerequisite for Verifiability
All the proofs above share an implicit premise: given the same input and the same seed, the inference result must be reproducible. This is “deterministic inference.” When the sampling temperature is 0, the random seed is fixed, and the computation graph and operator implementations are consistent, the same model should give consistent output for the same request. Only with reproducibility do commitments and proofs have meaning; otherwise every output is different and there is nothing to verify. So a truly verifiable inference service will typically proactively expose and commit to its deterministic inference capability.
Here is some illustrative pseudocode (only to explain the verification flow, not any specific vendor’s interface):
import hashlibdef canonical_json(obj):# Canonical serialization to keep key order and whitespace consistent for stable hashing...def build_binding(request_payload, nonce):# Bind request content with a one-time nonce to prevent replay and substitutionraw = canonical_json(request_payload) + noncereturn hashlib.sha256(raw.encode()).hexdigest()def verify_response(response, provider_public_key):# response contains at least: original request, nonce, model ID, parameter digest,# response content, signature, and commitment/proofcontent_hash = hashlib.sha256(canonical_json(response["content"]).encode()).hexdigest()signed_material = (build_binding(response["request"], response["nonce"])+ response["model_id"]+ response["param_digest"]+ content_hash)# 1) Verify the signature: confirm the response was generated by the private-key holder# and was not tampered withif not verify_signature(provider_public_key,signed_material, response["signature"]):return False# 2) Verify the commitment/proof: confirm the inference parameters and computation# flow meet the agreementif not verify_commitment_or_proof(response):return Falsereturn True
4. Real-World Limits: It Is Not a Silver Bullet
After understanding how this scheme works, it is also important to view its limits objectively; otherwise it is easy to overestimate its anti-fraud power.
- Depends on the provider’s active cooperation: All signatures, commitments, and proofs need to be implemented by the model vendor or service provider along the chain. If the upstream does not provide these capabilities, the caller cannot construct a trustworthy proof unilaterally. In other words, the switch for verifiability is in the provider’s hands.
- Deterministic inference has a cost: Strictly pursuing reproducibility often means sacrificing some sampling diversity and creates tension with performance optimizations (such as batching and dynamic scheduling). Whether to commit to determinism is an engineering tradeoff for the provider.
- General-purpose proof overhead is high: Running general-purpose zero-knowledge proofs over the full inference of very large models is still considerable in compute and time cost; TEE approaches are constrained by hardware availability and trust models.
- System prompt pollution still exists: Even if you prove “it was generated by a certain model,” if the relay node forcibly changes the model’s behavior through the system prompt, the proof can only reflect that “this behavior was indeed produced by that model under that prompt,” but it cannot prevent inducement at the prompt level. This is also the fundamental reason why self-reported identity is unreliable.
So response authentication is more like a means to substantially raise the bar for wrongdoing and leave traceable evidence of substitution, rather than a master key that “identifies all tricks in one click.”
5. Practical Advice for Enterprises and Developers
For production environments, build defenses from three angles: “observable, providable, and accountable.”
- Prioritize service channels that offer verifiable capabilities: When evaluating procurement and integration, include “whether it provides response signatures, parameter commitments, and deterministic inference” in your technical selection criteria.
- Full-chain logging: Uniformly collect the request digest, response hash, model identifier, latency, and billing information for each call, making post-hoc comparison and evidence-gathering easier. This can be done with general capabilities such as logging services, monitoring and alerting, object storage, and message queues. General cloud product categories from cloud platforms such as Baidu AI Cloud can serve as the vehicle for such data pipelines.
- Establish baselines and anomaly detection: Maintain a baseline of “normal response quality and latency” for critical business lines, and use monitoring and alerting to spot abnormal degradation or latency spikes and trigger manual review.
- Cross-verify critical conclusions: For high-risk, high-value scenarios, use multi-channel routing or sampling review to reduce the impact of a single relay chain being swapped.
- Sign traceable service agreements: Clarify model specifications, parameter commitments, and liability terms in contracts and technical agreements, so that technical and governance measures work together.
6. Summary
Model substitution along the relay chain is essentially a trust problem of “information asymmetry plus difficulty in providing evidence.” Naive approaches like pressing for identity and comparing style fail because they can be easily bypassed by prompt and field forgery. Verifiable inference offers a more rigorous path: through request-response binding, cryptographic commitments, digital signatures, verifiable computation, and deterministic inference, it turns “whether this response was generated by the model you paid for” from subjective guesswork into objective, verifiable proof. It depends on provider cooperation and has performance and cost constraints, but it can indeed expose substitution under accountable evidence. For developers and enterprises, the real defense is not any single “anti-fraud tool,” but systematic engineering that includes verifiable capabilities in selection, does full-chain logging solidly, and puts quality monitoring and governance agreements into practice.
