AMD announces the MI300 exascale accelerator for data centers, competing directly with Nvidia H100.

AMD unveiled its Instinct MI300 accelerator at CES, marking a significant enterprise announcement at the consumer-focused event. The accelerated processing unit (APU) comprises 13 chiplets, including CPU cores, GPU cores, and high bandwidth memory (HBM), totaling 146 billion transistors. This substantially exceeds Intel's anticipated Ponte Vecchio processor at around 100 billion transistors and Nvidia's Hopper H100 GPU at 80 billion transistors. The MI300 integrates 24 Zen 4 CPU cores, six CDNA chiplets (AMD's data center graphics technology), and 128MB of HBM3 memory stacked in a 3D design.

The 3D architecture delivers significant performance advantages by enabling tremendous data throughput between the CPU, GPU, and memory dies. Data flows directly to the HBM stack rather than requiring trips to DRAM, drastically reducing latency. This design also allows the CPU and GPU to work simultaneously on the same data in memory, accelerating processing capabilities.

AMD CEO Lisa Su announced the chip at the end of her 90-minute CES keynote, stating that MI300 is "the first chip that brings together a CPU, GPU, and memory into a single integrated design. What this allows us to do is share system resources for the memory and IO, and it results in a significant increase in performance and efficiency as well as [being] much easier to program." Su highlighted that the MI300 delivers eight times the AI performance and five times the performance per watt of the Instinct MI250, and noted it could reduce ChatGPT model training time from months to weeks, potentially saving millions in electricity costs.

The MI300 is currently in the labs with sampling to select customers, with a launch expected in the second half of the year. AMD's approach mirrors competitor efforts, including Intel's Falcon Shores due in 2024 and Nvidia's Grace Hopper Superchip due later this year. Additionally, AMD introduced the Alveo V70 AI inference accelerator, built on the Xilinx FPGA line AMD acquired and featuring XDNA AI engine technology. The V70 delivers 400 million AI operations per second within a 75-watt power envelope, offering 70% more street coverage for smart-city applications, 72% more hospital bed coverage for patient monitoring, and 80% more checkout lane coverage in smart retail stores compared to the competition. AMD is accepting pre-orders for the V70 cards with availability this spring.