How Zero-Knowledge Proofs Can Authenticate Large Language Model Answers and Prevent Model Substitution

1. A Common but Uncomfortable Reality: AI Answers Are Hard to Verify

When using large language model APIs, many developers and enterprise users have encountered a familiar problem: requests labeled as coming from a flagship model may sometimes return precise and fluent results, while at other times they appear noticeably “dumber,” with confused logic and weaker instruction-following ability.

The root cause is usually not mystery or speculation. It is that the trustworthiness of the inference pipeline cannot be externally verified. Common risk scenarios include:

• Model substitution: A customer pays for an expensive flagship model, but an intermediary or proxy service routes the request to a lower-cost model in the backend and returns the response as if it came from the original model.
• Silent degradation: A service provider reduces quantization precision, trims context, or downgrades inference configurations to control costs, while users cannot detect the change.
• Distilled impersonation: Some parties train imitation models through large-scale dialogue data and then present those distilled models as the original model.

The underlying issue is this: an LLM answer lacks cryptographic evidence strongly bound to its generation process and verifiable by a third party. Everything ultimately falls back to trust: trust that the vendor did not cut corners, and trust that the service provider did not insert cheaper intermediaries.

2. Why Traditional Authenticity Checks Are Mostly Ineffective

Common detection approaches in the community generally fall into three categories, but none of them solves the problem at its root.

1. Asking the model “Who are you?”
   This is almost useless. If a system prompt states that the model is a certain product or model type, almost any model can answer accordingly. Model identity is soft information that can be freely disguised through prompts.

2. Comparing answer style
   This is unreliable. The output style of the same model can change dramatically depending on the prompt, temperature settings, and context. Different models may also behave similarly on specific tasks. Style similarity therefore provides only weak evidence.

3. Checking returned fields and reasoning signatures
   This can only prove that the upstream service belongs to a certain vendor’s inference service. It cannot prove whether the actual execution used the flagship version or a cheaper lightweight version, which is exactly what users care about most.

The fundamental reason is that these methods verify either the surface appearance of the answer or the source label, not the generation process behind the answer. True authentication requires a mechanism that can mathematically or cryptographically demonstrate that “this answer was indeed produced by a specific model according to a given input.”

3. Core Idea: Replace Trust with Proof

Verifiable computation in cryptography is designed for this problem. It addresses the following scenario: when a less powerful party, such as a user, delegates a computation task to a more powerful party, such as a server, how can the user verify at extremely low cost that the returned result was indeed produced correctly by the claimed computation process?

The most important tool here is the zero-knowledge proof, especially zkSNARK. It allows a prover to convince a verifier that “I did execute a certain computation and obtained a certain result,” with three key properties:

• Completeness: A correct inference can always pass verification.
• Soundness: An incorrect inference, such as model substitution or process tampering, is almost impossible to fake into a passing proof.
• Succinctness: Verification is very cheap, usually far cheaper than rerunning the inference itself.

In other words, it turns the empty promise “you have to trust that I used the flagship model” into an actionable fact: “you can run the verification algorithm yourself and confirm it.”

4. Technical Workflow: Commit, Infer, and Prove

A zero-knowledge-proof-based authentication scheme for LLM answers can usually be divided into three stages.

Step 1: Commitment

The model vendor generates and publishes a cryptographic commitment to critical information such as model weights, model identity, and version number. This commitment has two important properties:

• Hiding: It conceals the original content.
• Binding: Once published, the vendor cannot later deny it or replace it with another model.

This public commitment becomes the trust anchor for all later verification.

Step 2: Challenge and Inference Execution

The user sends an inference request with a specific input and declares that verification proof is required. The server executes inference on the real model and configuration corresponding to the commitment, obtains the answer, and retains the internal states of the inference process as a witness.

Step 3: Proof Generation and Verification

The server generates a zero-knowledge proof based on the witness and returns it together with the answer. Any third party, including the user, an audit organization, or the open-source community, can run a lightweight verification algorithm offline using the public commitment, input, output, and proof.

If verification succeeds, it means that “the answer was indeed generated by the model corresponding to the commitment, using the given input, without being substituted or tampered with.”

The flow can be expressed more intuitively with pseudocode:

# Illustrative pseudocode: LLM answer authentication using commitments and zero-knowledge proofs
# Stage 1: The model provider publishes a cryptographic commitment once.
commitment = commit(model_weights, model_id, version)
publish(commitment)
# Stage 2: The user sends an inference request with a proof requirement.
request = InferenceRequest(prompt="...", require_proof=True)
# Stage 3: The service runs inference on the committed model and generates a proof.
answer, witness = model.generate(request)      # Actual inference
proof = prove(commitment, request.prompt, answer, witness)
# Stage 4: Any third party verifies offline at a cost far lower than rerunning inference.
is_authentic = verify(
    commitment    = commitment,
    prompt        = request.prompt,
    answer        = answer,
    proof         = proof,
    public_params = zk_setup_params
)
# is_authentic == True:
#   The answer was generated by the committed model and weights,
#   following the given input, without model substitution or silent degradation.

5. What It Can Solve and What It Cannot Solve

Every technical solution has boundaries. An objective assessment helps avoid blind optimism.

Problems That Can Be Detected in Theory

• A low-cost model impersonating a flagship model.
• Silent degradation or context trimming by a service provider.
• A distilled imitation model pretending to be the original model.

As long as the impersonated target has published a public commitment, the impersonator cannot provide a valid proof claiming that the output came from the flagship model.

Key Prerequisites and Inherent Limitations

• Dependency on vendor cooperation
  The trust anchor comes from commitments and proof capabilities actively published by the model vendor. If an upstream party is malicious and monopolizes commitment publication and proof generation, the trust problem degenerates into trust in a single trust root.

• Non-negligible computational overhead
  Large language models have huge parameter counts and inference steps. Generating zero-knowledge proofs for them is much more expensive than inference itself. In engineering practice, techniques such as proof recursion, hardware acceleration, or approximate circuits are often needed, and practical deployment remains difficult.

• Process verification, not quality verification
  It can prove that “the answer was correctly generated by the designated model,” but it cannot determine whether the answer content is correct or good. That still requires evaluation systems.

6. Practical Recommendations for Developers and Enterprises

Before mature cryptographic verification schemes become widespread, teams can still significantly improve trustworthiness through engineering practices.

• Build a reproducible evaluation benchmark
  Prepare a fixed question bank and standard answers. Regularly run regression evaluations against the same API and use quantitative metrics to detect signs of degradation.

• Cross-validate across multiple providers
  Send the same request to multiple service providers concurrently and compare the results. Abnormal divergence can serve as a signal of model substitution.

• Maintain auditable call logs
  Persist requests, responses, timestamps, token usage, and other relevant information into logging services and object storage. Combined with monitoring and alerting, this creates a traceable audit chain and helps detect abnormal cost fluctuations early.

• Track the evolution of verifiable inference
  Prefer services that proactively provide transparency proofs and publish model commitments. This helps replace “trust” step by step with “verifiability.”

7. Conclusion

The trust crisis in large language model services is essentially an information asymmetry problem caused by the invisibility of the generation process. Zero-knowledge proofs and verifiable computation provide a technical path from “please trust me” to “you can verify it yourself.”

There are still challenges to overcome in large-model scenarios, including computational cost and dependence on trust roots. However, the direction is clear: future competition in AI services will not only be about model capability, but also about trustworthiness and verifiability. For developers, establishing auditable and evaluable verification habits early is like adding a protective guardrail to your business.