Verifying LLM Response Provenance Through API Proxies

API proxies let applications avoid direct connections to model services, but they also hide “which tier you paid for and which model actually ran” inside an invisible forwarding path. Two endpoints that claim to serve the same premium model can differ by an order of magnitude in answer quality: one can handle multi-step reasoning and long-context constraints, while the other can only repeat surface instructions. The price difference is often not a prompting trick; the request is redirected mid-path to a cheaper model and returned as-is.

Why “Who Are You?” Is Not Evidence

Asking a model “what model are you?” has almost no discriminative power. System prompts, prefix injection, or output post-processing can impose a fixed self-description. Any model compatible with a chat format will repeat it. Temperature, refusal policy, and knowledge cutoff can also be imitated or trimmed. An identity claim is therefore generated content, not a verifiable source.

What needs verification is not whether the wording resembles a certain class of model, but whether the response can be cryptographically bound to the declared model service, this request, and the time it was returned. Without that binding, the model name in a log is merely a field filled in by the proxy.

Typical Deviations in Proxy Chains

Common deviations fall into several categories, and they have different verification goals.

1. Model substitution: billing is at the high-capability tier, inference falls to a low-cost tier, and the response format remains compatible.
2. Silent degradation: context is truncated, tool calls are disabled, or the reasoning budget is reduced, while the same model name is still shown.
3. Response stitching: some sentences come from a cache, a rule template, or another model, and are mixed into one message.
4. Capability transfer: an upstream provider uses a genuinely high-capability model to generate training or alignment data in bulk, but offers its own model as the service. This last category goes beyond “whether a single response came from the declared model” and requires data governance and contractual constraints; it cannot be solved by one online signature check.

On the user side, what can be observed reliably is mainly whether the final text is consistent with the declared service in the first three categories.

What a Proof Must Bind

A usable response proof should cover at least the following elements. Missing any one of them leaves a bypass.

• Service identity: proof that the proof was issued by a party holding the private key for a model service, not merely declared by a proxy.
• Request commitment: a hash over the canonicalized messages and key parameters (such as model tier, temperature, and maximum length), to prevent “signing request A but returning answer B.”
• Response commitment: a hash over the full output (including finish reason and tool-call structure), to prevent signing only a summary while the body is changed afterward.
• Time and sequence: limits proof replay; the same proof cannot be used to cover another call.
• Policy statement: if cache hits are allowed, the relationship between the cache key and the original proof should be explicit; otherwise the cache itself becomes a substitution entry point.

Canonicalization must be defined in advance: field order, whitespace, digest algorithms for multimodal attachments, and how streaming deltas are concatenated into final text. Otherwise the same semantics can produce different hashes and verification will always fail, or a proxy can insert content in positions that do not change the hash.

A Practical Verification Structure

Without binding to any specific vendor API, a workable industry approach is: the inference party signs, the caller verifies, and the proxy only forwards non-forgeable material.

1. The caller generates a request identifier and a random value, and submits them together with the canonicalized request.
2. After generation, the inference side holding the model service key signs “request hash ‖ response hash ‖ time ‖ request identifier ‖ model tier identifier.”
3. The response returns both the body and the proof. The proof only adds provenance; it does not replace business content.
4. The caller verifies the signature with a pre-distributed public key and recomputes both hashes locally. If either is inconsistent, the caller refuses to bill at that tier or pass the response downstream.
5. The audit system stores the request identifier, both hashes, the public-key identifier, and the verification result. It need not retain full text long term, reducing sensitive-data retention.

Public-key distribution must use a trust anchor separated from business traffic: internal configuration, a certificate system, or prior exchange between the parties. A proxy must not “bring its own public key and self-certify” in the same response. Otherwise the proxy can generate a temporary key, sign the output of a cheap model, and still pass verification.

The following illustrative verification logic shows only the check order; it is not an interface from any particular product:

canonical = canonicalize(request_messages, params)
req_hash = hash(canonical)
out_hash = hash(final_output)
ok = verify(public_key,
            signature,
            req_hash || out_hash || timestamp || request_id || tier_id)
ok = ok and fresh(timestamp, request_id) and tier_id == expected_tier

In streaming scenarios, verify the signature over the complete output after the stream ends. Intermediate chunks can at most undergo integrity checks; a model self-claim in each chunk must not be treated as the final proof. If the stream is interrupted, the incomplete output should be marked as not billable at the full tier.

Key Management, Rotation, and Proxy Boundaries

Private keys must exist only in the environment that actually performs inference at that tier. Proxies, log collectors, and prompt orchestration layers should see only public keys and proofs. If a private key is issued to a proxy, provenance verification immediately fails: the proxy can generate text with any backend and then sign it.

Keys must be rotatable. Proofs should carry a key identifier; verifiers select the public key by identifier and reject revoked identifiers. Handle clock skew with a short time window and request-identifier deduplication, so normal latency is not mistaken for replay.

With multi-hop proxies, each hop can add its own forwarding signature to prove “I forwarded the downstream proof unchanged,” but it must not rewrite the downstream signature over the model tier. The caller should verify the innermost model proof, then verify the proxy chain as needed. Trusting only the outermost proxy’s “model field” is equivalent to returning to a verbal promise.

What It Can and Cannot Prove

Provided the model service cooperates, the private key is not leaked, and both sides implement canonicalization consistently, the mechanism can detect: a proxy substituting a backend that does not hold the private key; body tampering after the response is returned; replay of an old response; and a proof that does not match this request.

It cannot by itself prove: that no cache was used inside the inference process; that model weights were not copied; that the service provider did not use outputs from other models during training; or that prompts were not rewritten by a proxy before entering the signed scope. The last point must be addressed by “signing the caller’s original request and detecting proxy rewrite”: if a proxy has the right to rewrite messages, the rewritten content must appear in the signed canonical request, and the caller can compare the original with the signed content to detect it.

Nor can it be used to compare which of two models is smarter. Quality evaluation still requires test sets and human or automated review. Provenance verification only answers “whether this was issued by the declared tier of service,” not “whether that tier is worth the price.”

Engineering Priorities

First, split billing tier, logged model name, and actual inference identity into three fields, and forbid a single string from serving all three. For premium tiers, require proofs; if missing, issue a downgrade alert or reject before storage. For failed verifications, sample and retain canonical requests and hashes to distinguish implementation errors from malicious substitution.

Clients and gateways should share a canonicalization library, so the two sides do not concatenate strings independently. Hash multimodal and tool results before including them in the request, to avoid placing large files directly into the signature buffer. When proof verification fails, users should be shown “provenance not verified,” rather than sending unverified text into automated decision-making.

Contracts and access control should state: proxies must not hold model service private keys, must not inject identity claims outside the signed scope, and caches must be traceable to the original proof. Technical verification addresses the provenance of a single response; permissions and auditing address bulk abuse and data forwarding.

Conclusion

The trustworthiness of a model response cannot rest on self-introduction. Bind the request, output, tier, time, and service public key together, and keep the private key at the actual inference endpoint; then substitution and tampering in the proxy chain become detectable events. This mechanism depends on service-provider cooperation and strict key boundaries, and it cannot replace quality evaluation or data-use constraints. Only by designing these three concerns separately can teams still know which generation, from which tier, they bought when using an API proxy.