Google Cloud announces construction of its most powerful supercomputer to date for AI workloads.

Google has announced its new A3 cloud supercomputer, now available in private preview, as part of its continued effort to provide cloud infrastructure for AI purposes. According to the company's blog post, "Google Compute Engine A3 supercomputers are purpose-built to train and serve the most demanding AI models that power today's generative AI and large language model innovation." The A3 uses the Nvidia H100 GPU, the successor to the popular A100 that powered the previous A2 and is also used to power ChatGPT, which launched in November 2022 and sparked the generative AI race.

A key innovation of the A3 is that it represents the first VM where the GPUs will use Google's custom-designed 200 Gbps VPUs, which allows for ten times the network bandwidth of the previous A2 VMs. The system makes use of Google's Jupiter data center, which can scale to tens of thousands of interconnected GPUs and "allows for full-bandwidth reconfigurable optical links that can adjust the topology on demand." Google claims that the "workload bandwidth... is indistinguishable from more expensive off-the-shelf non-blocking network fabrics, resulting in a lower TCO."

The A3 delivers substantial performance improvements, providing up to 26 exaFlops of AI performance, which considerably improves the time and costs for training large ML models. For inference workloads—the real work that generative AI performs—Google claims the A3 achieves a 30x inference performance boost over the A2. The system's standout specs include eight H100s with 3.6 TB/s bisectional bandwidth between them, next-generation 4th Gen Intel Xeon Scalable processors, and 2TB of host memory via 4800 MHz DDR5 DIMMs. According to Ian Buck, vice president of hyperscale and high performance computing at NVIDIA, "Google Cloud's A3 VMs, powered by next-generation NVIDIA H100 GPUs, will accelerate training and serving of generative AI applications."

Customers can deploy the A3 on Google Kubernetes Engine (GKE) and Compute Engine, receiving support for autoscaling and workload orchestration, as well as automatic upgrades. At Google I/O 2023, the company announced that generative AI support in Vertex AI would be available to more customers, allowing for the building of ML models on fully-managed infrastructure that forgoes the need for maintenance. Google appears to be taking a B2B approach to AI infrastructure rather than releasing consumer-facing tools, and has also announced PaLM 2, its newest large language model successor, at the same conference.