Verifiable LLM Inference: Practical Anti-Counterfeiting for Model Outputs

Once large model capabilities are offered at scale through open APIs, the call chain is rarely a simple direct connection from the user to the official inference service. Enterprise gateways, aggregation interfaces, secondary wrapper platforms, and various internal forwarding layers split a single conversation request into authentication, routing, caching, billing, logging, and other stages. From the end user’s perspective, the page may promise a certain capability tier and context length, while the model actually running behind the interface could be cheaper, weaker, or entirely different. The problem is made harder by the fact that questions such as “Who are you?” cannot reliably reveal the truth. With a small constraint in the system prompt, any model can claim to be a specified identity. Common community approaches—style comparison, response field inspection, and chain-of-thought signature checks—can at best suggest that an upstream service “looks like” a certain type of service. They can hardly prove in a cryptographic sense that a particular answer was actually generated by the declared model and weights. As a result, output authenticity has long remained a matter of commercial trust: vendors find it difficult to prove themselves when capability is quietly reduced, and callers find it difficult to produce evidence when a model is swapped. Replacing trust with verifiable proof is the core problem that verifiable inference aims to solve.

Why Verifiable Model Outputs Matter

The incentive for an API proxy to swap models is straightforward. Higher-capability tiers cost more per unit, so forwarding a request to a lower-cost model and returning the result as if it came from the original tier turns the price difference into profit. On the user side, the subjective experience is that the service “sometimes acts like an expert and sometimes like it has just learned to speak,” yet the user lacks evidence that can be submitted to an auditor. Distillation and impersonation are another risk category. Large numbers of fake accounts may collect conversations from a target model, use the collected results to train a knockoff model, and still claim the same origin capability externally. Even if a vendor does not intentionally reduce capability, once public discussion escalates, the vendor may lack a mechanism that can be publicly checked within a short time to clarify that “the weights and decoding policy have not changed.” Traditional detection methods are limited because they are mostly heuristic and can be easily bypassed through prompts, sampling parameters, and post-processing. They usually depend on experience with private fields from one vendor’s API and are difficult to turn into a reproducible cross-vendor standard. The goal of cryptographic proof is to let any third party use only a public verification key and a proof to determine whether an answer is consistent with the committed model, input, and generation process, without obtaining the weights and without re-running the entire inference.

How Zero-Knowledge Proofs Act as an Anti-Counterfeiting Code

A verifiable answer is typically divided into three steps: commitment, proof, and verification.

The first step is commitment. When publishing or upgrading a model, the inference provider makes a public commitment to the weight hash, tokenizer version, decoding constraints—such as temperature, top-p, maximum length, and whether a specific reasoning mode is enabled—and a software stack fingerprint, and writes them into an auditable manifest. The commitment itself does not reveal the weights; it only binds future outputs that claim to come from that model to this fingerprint.

The second step is proof. For each generation, or each batch of generations, the proof circuit must cover key computation segments. At minimum, this includes the backbone operation that maps input tokens to hidden states, the output layer that produces logits, and the sampling or greedy decoding completed according to the declared policy. A zero-knowledge succinct non-interactive proof, commonly discussed in the industry as a type of zkSNARK, allows a verifier to confirm that the circuit was executed honestly without seeing intermediate activation values.

The third step is verification. A caller or an independent audit tool uses the public verification key, input digest, output text, and proof file to complete verification in milliseconds to seconds. If verification passes, the answer is bound to the committed model. If verification fails, there may be model substitution, truncated computation, tampered sampling, or forged output.

Engineering Constraints That Must Be Addressed

Putting an entire forward pass of a hundred-billion-parameter model into a general-purpose proof circuit is extremely costly under current hardware and proof systems. Latency and expense are difficult to compare directly with ordinary chat APIs. As a result, practical deployment almost inevitably uses layering and approximation: proving key layers while using trusted execution environments or multi-party computation to assist with the remaining layers; using reproducible pseudorandom commitments for the sampling process to avoid having to prove irreproducible randomness for every input; and using chunked commitments and incremental proofs for long contexts to avoid a linear explosion of the context window.

Circuit design must also handle numerical consistency. Floating point, mixed precision, and kernel fusion can cause a mismatch between “running once in plaintext” and “running again inside the circuit.” It is necessary to define fixed-point rules or allowed error bounds explicitly and write them into the commitment. Proof generation can be asynchronous: return the text first to preserve the interactive experience, then attach the proof afterward. For high-risk scenarios—contract generation, code merging, and compliance question answering—verification should block synchronously, and unverified results should not be admitted into downstream systems. Key and commitment rotation, model canary releases, and A/B routing must all be included in the same version manifest; otherwise, verifiers cannot tell whether a failure is an attack or normal iteration.

Detecting Fraud in API Chains

Verification can be implemented as gateway middleware in the call chain. Before a request is sent, record the input commitment; after the response returns, verify the proof. On failure, raise an alert, reject the response, or degrade to a direct connection channel. For aggregation platforms, each upstream can be required to submit a model commitment when onboarding and to attach proofs per request or per batch. Billing and reconciliation should be based on successful verification. For enterprise internal multi-model routing, verification can prevent the case in which a request claims to use a high-capability tier but actually runs on a distilled small model.

It is important to emphasize that a proof binds “the declared computation.” If the official provider itself lowers the capability of a tier and updates the commitment, verification will still pass. What verification catches is undeclared substitution, not a replacement for product review and experience evaluation. For private deployments that are completely off-chain and do not publish commitments, verification can only be completed inside the organization and has no external credibility. Therefore, verifiable inference is a supplementary audit capability, not a replacement for contracts, SLAs, and manual spot checks.

Working with Logs, Monitoring, and Evaluation Systems

Proof files should be stored as audit objects in object storage and indexed to the request ID, model version, and input hash. Monitoring and alerting should focus on verification failure rate, proof latency, and circuit version mismatch. Evaluation sets can run both “plaintext score” and “verification pass rate” to avoid optimizing only benchmark scores while turning off proofs. On the security side, the supply chain for proof circuits and verification keys must be protected against tampering; otherwise, an attacker could forge both the model and the verifier. On the privacy side, the zero-knowledge property means verifiers do not need to see weights, but input may still appear in public verification materials. User data should therefore be processed with digests or selective disclosure. In compliance scenarios, verification logs can serve as non-repudiation records to support post-incident accountability.

Practical Recommendations and Boundaries

If you are a model provider, prioritize verifiable paths for high-price and high-risk interfaces. Start the circuit with decoding and the output layer, then gradually cover more layers. Publish commitments through the same pipeline as version releases, and avoid verbal claims. If you are a caller, treat verification failure as an event at the same level as authentication failure, not as an ignorable warning. If you are a platform, use a neutral verification protocol to connect multiple upstreams, and avoid writing detection logic as a crawler for one private response format.

Do not expect prompt probes or style classifiers alone to solve model substitution. Do not invent a nonexistent “one-click detection of all fake interfaces” capability. Numerical error, proof cost, asynchronous windows, and key rotation will all turn into incident tickets under real traffic. Write these constraints into the architecture so that verifiable output can move from a concept to an operable capability.

Conclusion

LLM answers have long relied on trust because there was no cryptographic bridge binding weights, input, and decoding policy to an individual output. Zero-knowledge schemes built around commitment, proof, and verification turn the question “Was this answer generated by the declared model?” into a publicly checkable proposition. This reduces the room for API proxy substitution, capability-reduction denial, and impersonation-based distillation. It cannot eliminate all fraud, nor can it replace experience evaluation, but it can move disputes from arguments to reproducible verification logs. For developers and enterprise users, the benefit is an additional hard metric when choosing an interface, additional evidence when an incident occurs, and an automatable gate for governance. What will ultimately determine effectiveness is whether circuit coverage, cost budget, and version governance can keep pace with model iteration.