AMD expands MI300 GPU portfolio with single-GPU models and eight-GPU platform featuring 1.5TB HBM3 memory.
AMD announced a comprehensive portfolio of new AI and data center products at its Data Center and AI Technology Premiere event in San Francisco, California. The company unveiled its Instinct MI300A processors featuring 3D-stacked CPU and GPU cores on the same package with HBM, alongside a new GPU-only MI300X model with eight accelerators and 1.5TB of HBM3 memory. The announcement also included 5nm EPYC Bergamo processors for cloud native applications, EPYC Genoa-X processors with up to 1.1GB of L3 cache, and EPYC Sienna processors for telco and edge deployments coming in the second half of 2023. Combined with its Alveo and Pensando networking and DPUs, AMD is positioning itself in direct competition with market leader Nvidia and Intel across AI acceleration solutions.
The Instinct MI300A is a data center APU comprising 13 chiplets, many of them 3D-stacked, with twenty-four Zen 4 CPU cores fused with a CDNA 3 graphics engine and eight stacks of HBM3 memory totaling 128GB. The chip contains 146 billion transistors, making it the largest chip AMD has pressed into production. Nine compute dies, a mix of 5nm CPUs and GPUs, are 3D-stacked atop four 6nm base dies that function as active interposers handling memory and I/O traffic. The MI300A will power the two-exaflop El Capitan supercomputer, which is slated to be the fastest in the world when it comes online later this year.
The GPU-only MI300X is optimized for large language models and comes equipped with CDNA3 GPU tiles paired with 192GB of HBM3 memory spread across 24GB HBM3 chips. AMD claims this allows the chip to run LLMs up to 80 billion parameters, which the company states is a record for a single GPU. The MI300X delivers 5.2 TB/s of memory bandwidth across eight channels and 896 GB/s of Infinity Fabric Bandwidth. The chip is forged from 12 different chiplets on a mix of 5nm (GPU) and 6nm (I/O die) nodes and contains 153 billion transistors total.
AMD conducted a demonstration of a 40 billion parameter Falcon-40B model running on a single MI300X GPU, marking what the company claims is the first time a model this large has run on a single GPU. The MI300X offers 2.4X HBM density compared to the Nvidia H100 and 1.6X HBM bandwidth compared to the H100, enabling AMD to run larger models on its chips. The cache-coherent memory architecture reduces data movement between CPU and GPU, lowering latency and improving performance and power efficiency by reducing power consumption from data movement, which often consumes more power than computation itself.
AMD also announced the AMD Instinct Platform, which combines eight MI300X GPUs onto a single server motherboard with 1.5TB of total HBM3 memory and is OCP-compliant, contrasting with Nvidia's proprietary MGX platforms. The open-sourced design is intended to speed deployment. The MI300A is sampling now, while the MI300X and 8-GPU Instinct Platform will sample in the third quarter and launch in the fourth quarter.