1. The Problem: Model Swapping and “Dumbed-Down” Inference in Relay Chains

In the large model service ecosystem, API relays and proxy layers have long played the role of the “last mile.” A user pays to call a premium model, but the relay may quietly forward the request to a cheaper model and pass the result back unchanged. A more covert tactic: the relay injects identity claims into system prompts, making any model “claim” to be the target model and thereby bypassing simple question-based verification. Such practices are not rare in the industry, and the actual inference quality users receive may fall significantly short of what they paid for.

Another problem has also surfaced: some parties mass-register fake accounts to harvest outputs from leading models at scale for distillation training, then package the distilled products as original models and offer them as services. Public reporting has put the volume of such harvested conversations at tens of millions to hundreds of millions, and the controversy continues. These phenomena point to the same technical gap: callers have no way to independently verify which model actually generated a given response.

Traditional detection methods fall roughly into three categories. First, asking the model for its identity—but system prompts can trivially override the answer. Second, comparing response style and wording characteristics—but stylistically similar models are hard to distinguish this way. Third, checking inference signatures or response field structures—these can prove “the upstream is some official service,” but cannot further distinguish between different model tiers within the same vendor’s lineup. Vendors themselves, when facing questions about whether their models have been quietly degraded, lack objective evidence they can present externally. At its root, the authenticity of AI output has long rested on trust, not proof.

2. From Trust to Proof: The Basic Idea of Verifiable Inference

Solving this requires a mechanism that lets model vendors provide a cryptographic proof of the claim “this output was generated by model X under specific parameters,” a proof that any third party can independently verify. This is precisely where verifiable computing applies to AI inference.

The core technical route typically involves zkSNARKs (zero-knowledge succinct non-interactive arguments of knowledge). Zero-knowledge proofs allow a prover to demonstrate that a computational statement is true without revealing private inputs such as model weights or intermediate inference states. Applying this to model output verification breaks down into three core steps:

Step 1: Commitment. The model vendor generates a cryptographic commitment to its model’s weights, inference configuration, or computation graph structure—essentially taking a “fingerprint” of the model. The commitment is publicly checkable and binding: the vendor cannot swap out the model afterward without being detected. Commitment schemes typically use hash functions or polynomial commitments; the former is simple and low-overhead, while the latter supports more sophisticated algebraic verification.

Step 2: Inference and Proof Generation. During each inference, the system records the computation trace and generates a proof stating that “this output was produced under the model parameters and computation logic corresponding to the commitment.” Proof generation requires converting the model’s forward computation into arithmetic circuits or constraint systems, which the proof system then compresses into a succinct proof. The computational overhead of this step is closely tied to model size and proof system efficiency, and it remains the main bottleneck for real-world deployment.

Step 3: Verification. Any third party holding the commitment value and the output can verify the proof’s validity at extremely low computational cost. Verification success means the binding between the output and the committed model holds; verification failure indicates the output was not produced by that model, or that the model does not match the commitment.

3. Engineering Breakdown and Key Design Decisions

The workflow is conceptually clear, but real-world implementation involves several critical details.

Commitment granularity directly affects verification capability. Committing only to model weights proves “the output came from this set of weights,” but cannot distinguish differences in decoding strategies under the same weights; committing to the full inference graph significantly raises proof costs. In practice, teams usually trade off between verification strength and proof overhead—for example, committing separately to weights and key inference configurations, or including sampling parameters as public inputs.

Proof system adaptation is another challenge. Large model inference involves massive matrix multiplications and nonlinear activations, and converting them into constraint forms friendly to proof systems requires specialized operator mapping and quantization strategies. Directions currently being explored include custom circuits tailored to Transformer architectures, lookup-table arguments for nonlinear layers, and moving parts of the computation outside the proof scope to reduce overhead. These optimizations directly affect proof generation latency and determine whether the approach is viable in online serving scenarios.

Verification-side service integration matters as well. The verification logic itself is lightweight, but it needs supporting infrastructure for commitment distribution, version management, proof storage, and audit interfaces. Developers can return proofs alongside the response body and have clients or independent verification services validate them; enterprise users can integrate verification modules at the gateway layer to check every upstream response. General-purpose cloud infrastructure can support this layer—for example, object storage for proof files, serverless functions for verification logic, and logging services for verification results—depending on the specific architecture.

Vendor cooperation is the precondition for the whole scheme to work. Commitments must be generated and published by the vendor at model release, and inference proofs must be generated on the serving side and returned with each response. If the upstream vendor does not cooperate, the verification chain cannot close the loop. This is the approach’s main current limitation: it provides a “vendor self-attestation + third-party verification” framework, not a universal detection method that works without vendor participation.

4. Real-World Boundaries: What It Can and Cannot Solve

In terms of technical capability, the scheme covers several typical scenarios. First, relay-layer model swapping: if the output a user receives comes with a proof matching the target model’s commitment, the swap cannot be hidden; a missing or mismatched proof signals an anomaly. Second, distilled models impersonating originals: a knockoff model cannot generate a proof corresponding to the original commitment unless it fully replicates the original model’s computation. Third, silent quality degradation by the service provider: if degradation means switching to a smaller model, the proof exposes the identity difference; if it means parameter adjustments within the same model, key configurations must be included in the commitment scope to detect it.

But the scheme has clear boundaries. For issues like “output quality fluctuating under different load levels for the same model,” cryptographic proofs can only bind “model identity and computation process”—they cannot directly prove “output quality meets expectations.” In addition, the extra latency and compute overhead of proof generation may limit applicability in low-latency scenarios, and implementation complexity means the onboarding cost for smaller vendors is nontrivial. Most importantly, the framework depends on vendors proactively disclosing and cooperating—it cannot force participation from unwilling actors.

5. Practical Takeaways for Developers

For developers and enterprises using model services, this technical direction suggests several actionable ideas.

At the architecture level, treat “output verification” as an optional capability layer in your model gateway. The gateway records proof information returned by upstream when forwarding requests; a verification module validates proofs offline or online, and results are written to audit logs. For critical business paths, you can set policies such as “alert if no proof is present” or “reject if the proof does not match,” establishing an observable trust baseline.

At the vendor evaluation level, include “does the provider support output proofs” as a selection criterion. While few vendors currently support this capability, as standardization progresses, proof generation and verification are likely to become a baseline observability metric for model services.

For self-built services, if your organization provides model capabilities externally, consider piloting the commitment–proof–verification design internally. For example, manage commitments for internal model versions, attach lightweight proofs in the inference service, and have callers verify version consistency. This helps reduce the troubleshooting cost caused by internal model version confusion.

6. Closing Thoughts

Verifying the authenticity of model outputs is fundamentally the process of converting “trust assumptions” into “verifiable claims.” Zero-knowledge proofs offer a theoretically viable path, but real-world deployment still requires continued iteration across proof efficiency, ecosystem cooperation, and standardization. For developers, understanding the principles and boundaries matters more than chasing any single tool—after all, any verification mechanism is only as effective as the willingness of every party in the chain to treat “verifiability” as a shared contract. As model serving becomes increasingly layered and relay chains increasingly complex, the shift from trust to proof is an idea worth sustained attention and hands-on experimentation.