NVIDIA, NVLink Fusion에 NVHBM 맞춤형 고대역폭 메모리 추가

NVIDIA’s NVLink Fusion program provides partners with the foundational components needed to integrate custom chips into the NVLink scale-up domain, which connects multiple processors into a single coherent system such as the Vera Rubin NVL72 rack-scale accelerator. Today, NVIDIA is expanding that toolkit with NVHBM, a custom implementation of the high-bandwidth memory that underpins nearly every AI accelerator currently in use.

According to NVIDIA, NVHBM is a custom HBM base die engineered to deliver higher bandwidth, reduced power consumption, and a smaller on-die footprint compared to standard HBM4e. Developed and validated alongside leading memory vendors, the component aims to accelerate time-to-market for custom silicon developers who would otherwise build commodity HBM integration from scratch. NVIDIA emphasizes that NVHBM is not a replacement for existing HBM; rather, it is a specialized building block exclusively available to its custom silicon partners.

Memory bandwidth remains critical for AI accelerators, and NVHBM delivers up to 30% higher bandwidth per stack than standard HBM4e. For memory-bandwidth-constrained workloads, this increase directly improves throughput, enabling higher tokens-per-second rates during AI inference.

The custom NVHBM base die also minimizes the footprint of memory-related circuitry on the primary accelerator die. Historically, the HBM memory controller was integrated directly into the main silicon package. NVHBM relocates the memory controller to the base die of the HBM stack and supplies a compact custom PHY, allowing NVLink Fusion partners to seamlessly integrate it into their designs.

This architectural shift liberates valuable package real estate, freeing up space for additional compute die area—potentially increasing compute capacity on the primary silicon die by up to 30%. Additionally, NVHBM simplifies interposer routing required to link multiple chips in advanced packaging configurations.

NVHBM also offers significant power efficiency gains over off-the-shelf HBM4e stacks. Consistent with NVIDIA’s messaging during the Vera Rubin rollout, the company maintains that “every watt that isn’t going into token production is a watt wasted.” NVIDIA states that NVHBM consumes 15% less power than commodity HBM4e. These savings can be leveraged to improve performance-per-watt metrics, reallocated to add functional units to a custom accelerator design, or utilized to achieve higher sustained performance within an identical power envelope.

Delivering higher bandwidth at lower power is a substantial advantage for AI accelerators tasked with moving large data structures, such as model weights and KV caches, particularly when these efficiencies scale across thousands of chips. The energy conserved during data transfer can be redirected toward boosting accelerator performance or supporting a greater density of accelerators within a fixed power budget.

It is important to note that these advantages apply to prospective partners integrating NVLink Fusion and NVHBM into custom designs, rather than to the Vera Rubin rack-scale systems currently in production.

NVIDIA also announced that Amazon’s Annapurna Labs will serve as the inaugural partner for NVHBM. Annapurna Vice President Nafea Bshara stated, “We look forward to this technology collaboration to benefit future AWS infrastructure designs.” Since Annapurna’s upcoming Trainium 4 AI chips already support the NVLink Fusion scale-up interface, subsequent generations are expected to incorporate NVHBM support as well.