How to Read Benchmark Results for a Chinese AI Chip Compared with NVIDIA H200
A Chinese AI chip vendor has reported performance matching or exceeding the NVIDIA H200 and H100 in selected workloads, including training, inference, and multimodal generation. This article explains the disclosed test results, identifies missing benchmark conditions, examines memory and cost claims, and provides a checklist for validating the chip with real production workloads before making a purchasing decision.
Where the Test Results Came From
A Chinese AI chip vendor demonstrated its hardware at a technology open day, comparing it with the NVIDIA H200 and H100. The demonstration covered large language model conversations, text-to-image generation, and text-to-video generation. The vendor stated that, while meeting the output-throughput requirements of online services, its chip delivered better overall performance than the comparison products in these scenarios.
A separate third-party test was conducted by Shanghai INESA in July this year. Under a prefill-decode (PD) disaggregated deployment architecture, the calculated end-to-end throughput per card was 1.05 times that of the H200. The publicly available materials do not include a report number. Readers should request the original report from the vendor or testing organization before attempting to verify the conclusion.
What Each Performance Multiple Actually Means
The vendor disclosed a wide range of performance figures, but each result is missing some test conditions. The table below summarizes the main results and the details that still need to be confirmed.
| Scenario | Vendor-reported result | Conditions that still need to be confirmed |
|---|---|---|
| Training | In an 8-GPU system running FP8 training on Qwen3-32B, effective throughput per card was 1.33 times that of the H100 | Software version, batch size, parallelization strategy, and sequence length |
| Long-context inference | Attention computation performance at a 64K context was 1.05 times that of the H200 | KV cache strategy, batch size, and memory usage |
| MoE inference | Throughput in a 64K-scale test was 1.15 times that of the H200 | Model version, number of experts, and routing configuration |
| Agent applications | Time to first token was approximately 10% to 30% lower than on the H200 with context lengths from 8K to 128K | Concurrency, number of tool-call rounds, and service-level agreement |
| Multimodal generation | Inference throughput was approximately 2% to 5% higher than that of the H200 | Model, resolution, and batch size |
| Online recommendation | Inference throughput increased by 21%, while latency for lightweight computation decreased by 26% | Model architecture, request size, and concurrency |
All figures in the table come from the vendor’s own tests. Vendors typically select configurations that present their products favorably. These figures are therefore useful for indicating technical direction, but they should not be used as the sole basis for procurement decisions.
Memory and Architecture: Verify the Numbers First
The vendor says the chip uses a proprietary Tian Shu architecture, natively supports scalar, vector, and tensor computing modes, and includes a new-generation Tensor Core. Each chip is equipped with 144 GB of high-bandwidth memory and supports FP4, FP8, FP16, and BF16 precision formats.
The supernode solution integrates 144 GPUs in a standard rack. At 144 GB per card, the theoretical aggregate memory would be approximately 20.7 TB. However, the vendor reports aggregate memory of approximately 10.1 TB—roughly half the theoretical total. The difference could result from the system configuration, reserved capacity, or differing reporting definitions, but the public materials do not explain it.
Before selecting the platform, customers should require the vendor to provide separate figures for usable memory per card and usable memory for the complete system.
Supernodes and Cost
The vendor claims that inference costs per million effective tokens are approximately 40% lower than those of leading international solutions. The source material does not define the hardware price, electricity cost, utilization rate, service-level agreement, or the meaning of “token.” It also does not specify the comparison baseline. Until these definitions are clarified, the 40% figure should be treated as a vendor claim rather than an independently validated result.
The supernode performs tensor-parallel sharding within a single rack, which can reduce communication across racks. This may provide practical value for very large models. Whether it is more cost-effective than existing solutions, however, must be determined through testing with the customer’s own workloads.
Pre-Purchase Validation Checklist
- Confirm the test model version and numerical precision, and determine whether they match the models used in your business.
- Confirm whether the figures are reported per card or per system, and convert them to the number of cards required for your deployment.
- Confirm the parallelization strategy, batch size, and sequence length. These details are especially important for long-context workloads.
- Confirm the software stack versions, including the driver, compiler, inference framework, and operator libraries. Check whether migration tools are available.
- Estimate power consumption, rack density, and data-center modification costs.
- Run load tests with real request traffic, recording throughput, P99 latency, and error rates.
- Confirm the service-level agreement, support response times, and hardware replacement procedures with the vendor.
Ecosystem and Risk
The software ecosystem will determine whether a domestic AI chip can be used reliably over the long term. Developers should examine which versions of major training and inference frameworks support the chip, as well as the migration cost for custom operators. A multi-cloud or heterogeneous-compute architecture can reduce dependence on a single vendor, but it also increases scheduling and operations complexity.
PD-disaggregated deployment and FP8 training are technology paths that have already been widely evaluated. The chip’s performance in these areas is worth monitoring. However, more public data is still needed to assess its behavior across additional model families, more complex distributed-training workloads, and a broader range of inference frameworks.
Conclusion
In the specific tests disclosed by the vendor, this Chinese AI chip delivered results close to or above those of the H200 and H100 across several metrics. This indicates that it may be competitive in selected scenarios. To turn these results into a procurement decision, organizations should first obtain the missing test conditions and then validate the hardware with their own production workloads.
