From Weight Commitment to Inference Proof: Verifiable LLM Outputs

Where the Problem Comes From

When enterprises connect LLMs to internal systems via unified gateways forwarding to upstream inference services, callers typically only see the text output. Stylistic drift, changing rejection policies, or inconsistent knowledge boundaries can easily be explained away as “version updates” or “sampling randomness.” Manual spot-checking with fixed probes can build suspicion but rarely produces verifiable evidence: probes become outdated, evaluation criteria rely on experience, and vendors can explain away differences with reasonable engineering justifications.

Therefore, the industry is beginning to discuss a different capability: allowing third parties to confirm on commodity compute that “a specific response was indeed calculated by the committed model on a given input” without revealing the weights. This is not content safety detection or stylistic similarity scoring; it turns key computations during the inference process into publicly verifiable cryptographic objects.

The Minimum Problem Verifiable Inference Must Solve

This minimum problem can be broken down into three points.

First, the model provider makes a cryptographic commitment to a specific version of weights and publishes a commitment ID. The weights can remain private, but they cannot be swapped with another set of parameters without being detected.

Second, each (or each batch of) request is accompanied by a proof. This proof binds the “input tokens, output tokens, and the committed compute graph” together.

Third, any verifier can accept or reject the result using only the commitment ID, prompt, response, and proof, without needing training weights of the same scale or trusting the inference node’s logs.

This mechanism answers “whether the computation was executed as committed,” not “whether the answer is factually correct,” nor “whether the service provider is commercially honest.” Cryptography only closes the loop on the “computation origin” aspect.

Offline Commitment and Online Proof

The offline phase occurs only once (or when weights change). The provider maps the weights to the representation required by the proof system, generates the commitment, and publishes the public parameters and commitment ID needed for verification. The commitment must be binding: finding another set of weights that passes the same commitment should be computationally infeasible. The ID itself should be short, registrable, and integrated into the change management process to align with model versions, evaluation reports, and contract attachments.

The online phase generates proofs for specific requests. Verification inputs usually include: the commitment ID, the complete prompt (including system instructions and history, if they affect the next token), the output token sequence to be verified, and the proof bytes. Verification should complete on a CPU within an acceptable time frame so that auditing, client-side spot-checks, and dispute resolution are practical. Proof generation is much heavier than standard inference, often requiring specialized accelerators and significantly increasing latency, memory, and bandwidth usage.

In engineering, “what is being proven” must be written into the protocol, rather than just exchanging a file. At minimum, it should specify:

• Whether the compute graph corresponds to the full forward pass, or only the last few layers, or only the activated expert subnetwork;
• How the context is encoded, and whether padding, truncation, and special tokens are included in the hash;
• Whether the proof covers the entire response or just sampled positions;
• Whether the failure return is a boolean or a differential explanation with positions.

A schematic verification interface can remain implementation-agnostic:

verify(
  commitment_id,
  prompt_bytes,
  output_token_ids,
  proof_blob
) -> ACCEPT | REJECT(reason)

Callers can save this 4-tuple for post-hoc re-verification. If a gateway only stores text without token IDs and tokenizer versions, alignment may be impossible later.

Why Model Structure Affects Provability

In large parameter models, the arithmetic scale of a dense full-parameter forward pass is massive, and proof costs generally scale with effective computation. Sparse structures like Mixture of Experts (MoE) only activate a subset of parameters, allowing the proof circuit to cover only the active paths, reducing cost from “all parameters” to “parameters actually involved in this computation.” This is a key reason verifiable inference is now a viable topic, but it is not a free lunch.

Verifiers must confirm: which experts are activated, how routing scores are quantized, and whether unactivated branches are correctly excluded. These rules themselves must be part of the proven statement. If the server can arbitrarily alter routing without commitment constraints, sparsity becomes a manipulable loophole. Additionally, the actual structure of proprietary models is not always known externally. A solution working on one sparse model does not automatically cover all closed-source models. Until structure, tokenizer, and numerical format are included in the same commitment, “directionally verifiable” should not be conflated with “every response of a specific live model has been proven.”

Integer Circuits vs. Live Floating-Point Models

Zero-knowledge proof systems handle exact arithmetic. A common approach is to rewrite matrix multiplication, normalization, and key operations in attention into integer or fixed-point domains, and restrict nonlinear implementations. Live services usually run in floating-point inference engines, potentially using different kernel fusions, accumulation orders, and hardware rounding.

Thus, the object being proven is actually an “integer approximation model,” not the floating-point model on the GPU. There is no universal, automatically transferable upper bound on the logit gap between the two; empirical errors in documentation cannot replace proofs. During deployment, separate alignment tests are necessary: under the same prompt and greedy decoding, measure the token consistency rate, maximum logit difference, and whether differences concentrate on punctuation or synonymous paraphrases. These test results should be archived alongside the commitment ID, but the tests themselves are not cryptographic guarantees.

Decoding Strategies Determine the Proof Statement

Many proof prototypes default to greedy decoding: each position must equal the argmax of the integer model. This is inconsistent with the temperature sampling, top-p, and top-k commonly used online. Once sampling is enabled, the same floating-point model will naturally output different tokens, causing exact matches to fail frequently—though failure does not mean the “model was swapped.”

In practice, there are two common acceptance statements, requiring a choice or layered application in the protocol.

Exact Match. The token at each checked position must be the highest probability item of the integer model. Accept if passed; reject if any position differs. Suitable for regression testing, fixed probes, and internal tool calls requiring strong binding; not suitable for high-temperature user-facing conversations.

Bounded Gap. Calculate the logit gap of each output token relative to the optimal item, then set a clipping ceiling and an allowable cumulative cost. Accept if the gap is within budget. This tolerates approximation errors and mild randomness, but the budget itself is a policy parameter: too wide means almost no constraint, too narrow reverts to exact match. The budget, clipping constants, and whether comparing log-probabilities or unnormalized logits must be written into the public spec, otherwise verification results are incomparable.

If the service uses speculative decoding, draft models, or external safety classifiers to rewrite final tokens, the proof statement must include this post-processing, otherwise it proves the draft or intermediate result, not the text seen by the user.

Cost, Size, and Applicable Boundaries

Qualitatively, proof generation being an order of magnitude slower than direct inference is not uncommon, and proof size is significantly larger than the response text. Therefore, it is unsuitable to make every interactive token a strong real-time proof. More reasonable use cases include:

• Asynchronous proof generation for high-risk requests (contract review, financial figures, permission change suggestions);
• Daily sampled proof generation for batch evaluations and vendor reconciliation;
• Retaining only commitment IDs and log hashes for normal chats, replaying only during disputes.

Evaluate end-to-end, not just “how many tokens can be proven per second.” Queue delays, proof storage cycles, verifier CPU, and the window where commitments are invalid after weight updates all affect production readiness. Claims of being “orders of magnitude faster than early solutions” are merely research benchmarks and cannot replace your own workload measurements.

Recommended Acceptance Checklist for Integration

1. Identity Alignment. Ensure the commitment ID, tokenizer version, context template, max length, and stop conditions match the live environment.
2. Numerical Alignment. Under greedy settings, track the consistency rate between the integer model and the floating-point engine, and log failure samples rather than just average scores.
3. Decoding Alignment. If sampling is used live, the acceptance statement must be bounded gap or a separate sampling proof; do not publicize exact match pass rates as the standard.
4. Structure Alignment. Ensure sparse routing, quantization bit-width, and activation checkpoints are included in the same commitment.
5. Negative Capability. Deliberately swap a weight block, change a token, or increase temperature; verification must consistently reject. Testing only “honest path passes” is insufficient.
6. Operations. Re-commit upon weight or template changes; old proofs only apply to old IDs; retain public parameters needed for verification to avoid orphaned IDs.

What It Still Cannot Replace

Even if a proof passes, it only indicates that “the committed computation” produced those tokens. It does not prove the weights came from the claimed training process, does not prove no hidden instructions were appended outside the prompt (unless prompt bytes are fully in the statement), does not prove retrieval content wasn’t swapped (unless retrieval results are also committed), and certainly does not prove the answer complies with business policies. Security filtering, data masking, access control, and manual review remain necessary.

Conversely, a proof failure does not automatically equal fraud. Integer approximation, kernel differences, sampling, and post-processing can all create honest inconsistencies. Treating failure directly as a breach is engineeringly untenable. Contract language should distinguish between “commitment mismatch” and “explainable deviation due to approximation or sampling.”

Implementation Order

It is recommended to first close the loop in the internal evaluation pipeline: fix the tokenizer, use greedy decoding, and run a small-scale sparse or fully circuit-capable model to execute commitment, proof generation, verification, and intentional failure cases. Then discuss the threshold for gaps with floating-point services. Only when the structure is public enough to be written into a circuit, numerical differences are acceptable for the business, and proof latency fits into an async queue, should this capability be exposed to external clients. Until then, external messaging should remain “research direction and acceptance methods for verifiable inference,” not “ready to authenticate the authenticity of any flagship model’s response.”

Cryptography pushes the debate from stylistic impressions to verifiable statements, which is valuable in itself. The value depends on whether the statement covers the real service path. For the parts it doesn’t cover, you still need version management, sampled evaluations, and contract constraints; you cannot substitute them with a proof that cannot map to live floating-point and sampling.