Anti-Fraud Technology for AI Responses: How Zero-Knowledge Proofs Identify Model Substitution

As large language model (LLM) services become ubiquitous, users are paying closer attention to model identity, version, and service quality. Many platforms offer multiple model endpoints, but the name displayed to the user does not necessarily correspond to the actual model processing the request. Due to cost constraints, rate limiting, failover, or operational strategies, service providers may route requests to lower-capability, cheaper models. For ordinary users, it is difficult to accurately determine which model actually fulfilled the request based solely on the response content.

This issue is fundamentally not about whether a model can introduce itself, but rather a problem of service authenticity verification. Asking a model “What model are you?” does not constitute valid evidence, because system prompts can instruct any model to impersonate a specific identity. Even if the response style, format, and reasoning process appear similar, they can only serve as empirical judgment and cannot form an auditable, verifiable proof.

A more rigorous approach is to require the model service provider to attach a cryptographic proof to every generated output. Verifiers do not need to trust the platform’s verbal promises or see all internal model parameters; they only need to use a public verification program to determine whether the result was indeed generated by a registered model. This is exactly the problem that “Verifiable AI Output” aims to solve.

1. Why Traditional Identification Methods Are Unreliable

Currently, common model identification methods fall into three main categories.

Identity Inquiry: Users directly ask the model for its name, version, or service provider. This method has the lowest implementation cost but also the lowest reliability. Model outputs are fundamentally text; system prompts can alter a model’s self-description, and proxy layers can directly modify the returned content. Therefore, identity inquiry is better suited for debugging rather than authentication.

Behavioral Testing: Users prepare a fixed set of questions and judge based on response quality, format adherence, coding ability, or long-text processing capability. This method can detect obvious model downgrades but cannot prove model identity. Different models may perform similarly on certain tasks, and the same model may exhibit significant variations due to context, sampling parameters, and service load.

Interface Feature Inspection: For example, checking return fields, request formats, tool-calling structures, or streaming output behaviors. This method can identify implementation differences in some proxy layers, but interface formats are easily counterfeited. As long as the relay layer can repackage the response, it can forge surface features.

Therefore, traditional methods mostly fall under “inference” rather than “proof.” They can help users detect anomalies but cannot provide cryptographically sound conclusions.

2. The Basic Concept of Verifiable Output

Verifiable AI Output typically involves three participants:

1. Model Service Provider: Responsible for running the model and generating responses.
2. User or Business System: Submits requests and receives results.
3. Verifier: Checks whether the result and its proof are valid.

In an ideal workflow, the provider returns not only the text but also a proof related to the request, model identity, and computation process. The verifier checks this using a public verification key, ultimately reaching a conclusion like “Proof Valid” or “Proof Invalid.”

The key here is not to expose the model’s internal parameters, but to prove that a specific computation was indeed completed as agreed. For example, the provider can prove that:

• The input request went through the specified preprocessing pipeline.
• The computation used the digest of registered model parameters.
• The output result is consistent with the input and the model computation.
• The returned content was not substituted after generation.
• The proof corresponds to the required service version and time range.

To avoid leaking user privacy, systems typically do not directly expose the complete input, internal parameters, or intermediate activation values. Instead, they use cryptographic commitments and Zero-Knowledge Proofs (ZKPs) to complete the verification.

3. Commitment Mechanisms: Lock First, Reveal Later

A commitment can be understood as a cryptographic mechanism to “lock content first, and choose whether to reveal it later.”

The provider can compute digests for model parameters, model version, request content, and output content, treating these digests as commitment values. A digest has two important properties:

• Hiding: Based solely on the digest, outsiders cannot reconstruct the original model parameters or user request.
• Binding: Once a provider publishes a commitment, it is computationally infeasible to substitute it with a different set of content later while still obtaining the same digest.

For example, this can be abstractly represented as:

model_commitment = Hash(model_manifest)
request_commitment = Hash(request_data)
output_commitment = Hash(response_data)

Here, Hash represents a cryptographic hash function, and model_manifest can contain metadata such as model version, parameter file digests, runtime configurations, and release dates. Actual systems must also consider random numbers, context windows, tool-calling results, and sampling configurations; otherwise, the proof scope may be incomplete.

A commitment alone does not equal a complete proof. It only shows that the provider fixed certain content in advance; it cannot independently prove that the model actually executed according to that commitment. Therefore, verifiable computation must be introduced.

4. How Zero-Knowledge Proofs Work

Zero-Knowledge Proofs allow a prover to demonstrate to a verifier that a certain proposition is true without revealing any sensitive information beyond the proposition itself.

In the model service scenario, the proposition can be abstracted as:
“There exists a set of valid model parameters and a computation process such that, given the input, the specified computation yields the current output, and the digest of the model parameters equals the registered commitment value.”

The verifier only needs to confirm that the proof is valid, without obtaining the full model parameters or seeing all intermediate computation steps.

Many schemes use Succinct Non-Interactive Arguments of Knowledge (SNARKs) within ZKP systems. The advantage is that the proof is relatively compact, and verification speed is typically much faster than re-executing the complete model inference. However, LLM computation involves massive matrix multiplications, non-linear functions, attention calculations, and cache operations. Directly converting the complete inference process into a proof circuit entails high engineering complexity and computational costs.

Therefore, actual designs often require a layered approach:

• Use hash commitments for model files and the runtime environment.
• Generate verifiable proofs for critical computation steps.
• Use Trusted Execution Environments (TEEs) or audit logs for parts that are not suitable for direct proof.
• Use signatures and timestamps for the entire request chain to ensure integrity.

This means that an “output with proof” does not necessarily mean that every floating-point operation was individually proven; it depends on which stages the scheme explicitly covers.

5. A Complete Verification Workflow

A relatively complete workflow can be divided into the following steps:

Step 1: Model Registration. The provider publishes a model manifest, including the model identifier, version information, parameter digest, proof rules, and verification public key. The model identifier here cannot just be editable text; it must be bound to an immutable digest.

Step 2: Request Binding. After the user submits a request, the provider normalizes the request content, system configurations, session context, and random parameters, and generates a request digest. This prevents the same proof from being copied to other requests.

Step 3: Model Computation. The provider uses the registered model to complete inference and records necessary execution information. If the service includes retrieval, tool calls, or external data, it must also clarify whether these fall within the proof scope.

Step 4: Proof Generation. The provider generates a proof based on the model commitment, request digest, and output digest. The proof can include public metadata such as version, time, request ID, and service policies.

Step 5: Return Result. The response includes not only the response text but also the proof, model digest, request digest, signature, and the public parameters required for verification.

Step 6: Client Verification. The user end or business gateway calls the verification program to check whether the signature, digest, proof, and context are consistent. Upon verification failure, the result should be marked as “unconfirmed” rather than simply judged as “definitely fraudulent,” because failure could also stem from version incompatibility, data loss, or verifier program errors.

6. Can It Completely Identify Relay Service Misuse?

The answer is: with a clear coverage scope and the genuine participation of the provider, it can significantly improve identification capabilities, but it cannot automatically solve all relay issues.

If a service provider maps a premium model name to a cheap model, and the cheap model lacks a corresponding valid proof, the verifier can detect the model commitment mismatch. This approach is particularly effective against “name misuse.”

If the provider obtains results from a legitimate model and then tampers with the text, the output digest and proof will usually become invalid. However, the system must prove the final delivered content, not just the upstream raw content; otherwise, the middle layer could still modify the results.

If the service provider merely forwards the request and retains the original proof, it can prove that “the result came from a legitimate model,” but not necessarily that the request was not logged, rewritten, or appended with additional context. Therefore, input content, system prompts, and tool calls must also be included in the binding scope.

If the platform uses dynamic routing internally—for example, switching between multiple versions based on load—the registration mechanism must allow multiple legitimate models to participate in verification and clearly specify the version actually used for each request. Otherwise, the verification result might be valid, but the user still cannot determine whether the service quality matches the purchased commitment.

7. Key Challenges in Engineering Implementation

Proof Cost: LLM inference involves massive computation. A complete proof can introduce noticeable latency and hardware overhead. Engineering solutions typically require layered proofs, batch proofs, sampling audits, or specialized acceleration hardware. It is unrealistic to assume that the verification mechanism can run at zero cost.

Floating-Point Computation Consistency: Different hardware, compilers, and parallelization strategies can produce subtle numerical differences. If the proof system requires strict reproducibility, it must unify computation representations, rounding rules, and operator implementations. Otherwise, the same model might not yield consistent results in different environments.

Privacy Protection: User requests may contain code, business data, personal information, or internal documents. The proof system should only expose necessary digests and avoid writing raw inputs into public logs. For highly sensitive scenarios, it must also incorporate access control, key rotation, and audit trails.

Proof Key Leakage Prevention: If the private key required to generate proofs is compromised, an attacker could forge seemingly valid results. Therefore, Hardware Security Modules (HSMs), tiered key management, revocation mechanisms, and versioned verification rules should be utilized.

8. How Enterprises Can Build a More Reliable Model Audit System

Enterprises should not rely solely on a single anti-fraud technology but should establish a multi-layer verification strategy:

• Save request digests, response digests, and timestamps at the service access layer.
• Establish an immutable registry for model versions, configurations, and routing rules.
• Require signatures or verifiable proofs for critical responses.
• Conduct automated regression testing for anomalous results.
• Establish clear alerts for model degradation, failover, and manual intervention.
• Implement tiered handling for verification failures, missing proofs, and version inconsistencies.
• Periodically sample and review model outputs against the capability scope defined in service contracts.

In practice, proof technology is best combined with gateways, logging services, monitoring alerts, and audit systems. The gateway is responsible for verification, the logging system for retaining evidence, the monitoring system for detecting anomalies, and the audit system for tracing the chain of responsibility. This expands the scope from “whether a single response is genuine” to “whether the entire service process is traceable.”

9. Understanding the True Value of This Technology

The value of Verifiable AI Output is not to make users blindly trust a model name, but to translate some service commitments into checkable technical facts. It can mitigate risks associated with model substitution, result tampering, and version confusion, and help legitimate providers prove they have not arbitrarily swapped models.

However, it still cannot directly prove that a model is “smarter,” that a response is “more correct,” or that a result is “more valuable.” Cryptographic proofs can only prove that a computation process complied with rules; they cannot prove the rules themselves are reasonable, nor can they replace fact-checking, quality evaluation, and security testing.

Therefore, a more mature future model service certification should cover three layers simultaneously:

Layer 1: Identity Authenticity. Confirm which registered model the result originated from.
Layer 2: Process Integrity. Confirm that the request, configuration, routing, and output have not been illegally substituted.
Layer 3: Quality and Compliance. Confirm that the model capabilities, data sources, and usage methods meet business requirements.

Only by combining these three layers can AI services gradually transition from being purchased based on “brand and reputation” to being “verifiable, auditable, and accountable.” For developers, the most important takeaway is not whether a specific project is trending, but how to truly integrate cryptographic proofs, service gateways, and engineering audits into the model invocation chain.