H100 GPUs establish performance leadership in MLPerf v3.0 benchmark for AI workloads.

NVIDIA H100 Tensor Core GPUs have set new records on all eight tests in the latest MLPerf training benchmarks, excelling particularly on a new generative AI test. A cluster of 3,584 H100 GPUs co-developed by startup Inflection AI and operated by CoreWeave, a cloud service provider specializing in GPU-accelerated workloads, completed a massive GPT-3-based training benchmark in less than eleven minutes. Brian Venturo, co-founder and CTO of CoreWeave, stated that "Our customers are building state-of-the-art generative AI and LLMs at scale today, thanks to our thousands of H100 GPUs on fast, low-latency InfiniBand networks." Inflection AI, co-founded in early 2022 by Mustafa and Karén Simonyan of DeepMind and Reid Hoffman, harnessed this performance to build the advanced large language model behind its personal AI product, Pi. Mustafa Suleyman, CEO of Inflection AI, noted that "Anyone can experience the power of a personal AI today based on our state-of-the-art large language model that was trained on CoreWeave's powerful network of H100 GPUs."

The H100 GPUs delivered the highest performance on every benchmark, including large language models, recommenders, computer vision, medical imaging and speech recognition, and were the only chips to run all eight tests, demonstrating the versatility of the NVIDIA AI platform. On every MLPerf test, H100 GPUs set new at-scale performance records for AI training, with near linear performance scaling on the demanding LLM test as submissions scaled from hundreds to thousands of H100 GPUs. CoreWeave delivered from the cloud similar performance to what NVIDIA achieved from an AI supercomputer running in a local data center, a testament to the low-latency networking of the NVIDIA Quantum-2 InfiniBand networking CoreWeave uses. In this round, MLPerf also updated its benchmark for recommendation systems with a larger data set and more modern AI model, and NVIDIA was the only company to submit results on the enhanced benchmark.

Nearly a dozen companies submitted results on the NVIDIA platform, with submissions from major system makers including ASUS, Dell Technologies, GIGABYTE, Lenovo, and QCT, and more than 30 submissions ran on H100 GPUs. MLPerf results are available today on H100, L4 and NVIDIA Jetson platforms across AI training, inference and HPC benchmarks, with future submissions expected on NVIDIA Grace Hopper systems. The benchmarks enjoy backing from a broad group including Arm, Baidu, Facebook AI, Google, Harvard, Intel, Microsoft, Stanford and the University of Toronto, ensuring transparency and objectivity for users evaluating AI platforms. Data centers accelerated with NVIDIA GPUs use fewer server nodes, consuming less rack space and energy, and NVIDIA powers 22 of the top 30 supercomputers on the latest Green500 list. All software used for these tests is available from the MLPerf repository, with optimizations continuously folded into containers available on NGC, NVIDIA's catalog for GPU-accelerated software, enabling virtually anyone to achieve these world-class results.