Samsung is utilizing advanced logic fabrication processes to improve HBM performance while developing next-generation zHBM DRAM that vertically stacks memory directly onto compute dies.
Samsung has presented a comprehensive vision for evolving the HBM base die through advanced logic processes, addressing the architecture’s persistent bandwidth and power challenges. The company outlined a three-phase roadmap—spanning area reclamation and functional expansion to full 3D zHBM integration—that transforms the base die into an intelligent partner for next-generation AI systems.
During a Hot Chips presentation, Sangwook Han of Samsung’s DRAM Design Team detailed this evolution, centering the discussion on the HBM base die. The session began with a recap of HBM architecture, which comprises two core components: the C-die (Core Die), containing the core logic and DRAM cells, and the B-die (Base Die), positioned at the bottom to manage auxiliary DRAM operations. Core dies can scale up to 16 layers (16-Hi) using advanced DRAM processes such as D1a, D1b, and D1c, interconnected via multiple Through-Silicon Vias (TSVs). Each HBM chip incorporates a PHY within the base die, enabling high-speed data transfer to computing dies (XPUs like GPUs and TPUs) through a fast interconnect pathway.
As AI workloads continue to expand, higher bandwidth remains the primary driver of DRAM and HBM evolution. Current offerings reflect this demand: HBM4 delivers over 3 TB/s of bandwidth per stack, HBM4E pushes toward 4 TB/s, and HBM5 doubles HBM4’s bandwidth while supporting capacities exceeding 60 GB. However, scaling bandwidth faces inherent constraints, primarily limited TSV counts, pitch restrictions, and I/O count and speed limitations within the base die.
Advanced logic process nodes present another bottleneck, though the performance gap between the B-die and xPU system-on-chips continues to narrow. Leveraging advanced nodes offers significant advantages. Samsung confirmed its application of D1c and 4nm processes to HBM4, targeting reduced power consumption and a smaller active area. These optimizations represent the initial step toward true DRAM and advanced logic integration.
A key focus for Samsung is custom HBM, which explores embedding value-added functionalities directly onto the base die. While standard HBM base dies handle basic data routing and test paths, custom HBM leverages advanced logic processes to integrate system-on-chip-like capabilities alongside standard core die stacks. By offloading specific functions from the xPU, custom HBM reclaims valuable silicon area. It also replaces the traditional HBM PHY with a device-to-device (D2D) interface built on advanced logic nodes. This redesign yields a smaller footprint, shorter signal channels, improved energy efficiency, and additional silicon real estate for xPU expansion.
Transitioning from HBM4 to HBM5, custom HBM solutions utilizing advanced logic processes dramatically reduce the physical footprint of the PHY and D2D interface. While shorter channel lengths enhance performance, they also introduce thermal hotspot challenges. To mitigate this, Samsung introduced its Heat Path Block (HPB) technology, integrated into its custom HBM4 (cHBM4) solution. HPB reduces peak temperatures by over 35% and covers approximately 50% of the PHY area.
With thermal management addressed, the next phase involves migrating logic functions from XPUs to the custom HBM. The memory controller is a primary candidate due to its manageable thermal impact. Additionally, Samsung is evaluating a near-memory-controller (Near-MC) "SRAM-Base" cell repair scheme. This approach repurposes unused base die space for SRAM repair resources, enabling cell-level fail-address decoding and redirection within the memory controller.
In the second phase, Samsung proposes integrating a memory extension controller and PHY into the remaining base die space to address growing capacity demands. Exponentially expanding context windows in AI applications drive massive KV cache requirements, necessitating expanded memory footprints. The integrated PHY can support higher capacities through external memory solutions. Further silicon offloading to the cHBM base die includes partial computation "Processing Elements," which reduce D2D bandwidth requirements, lower power consumption, and decrease thermal overhead.
These initiatives culminate in the third phase: 3D integration. Samsung is developing zHBM, a solution that vertically stacks HBM directly atop the XPU. This tightly coupled architecture further conserves silicon area and eliminates the need for a 2.5D interposer. By leveraging distributed I/Os, zHBM minimizes data travel distances within the stack. The 3D structure replaces conventional 2D interfaces, delivering significantly higher efficiency for ultra-low-power systems. Power reductions stem from optimized I/O architectures that remove SerDes components, eliminating unnecessary power overhead.
Compared to a standard HBM4e stack, Samsung claims the zHBM solution improves power efficiency by 70%, increases DRAM bandwidth by 230%, saves up to 100W per DRAM module, and allocates 8.3% more power to the XPU (illustrated here with a GPU). The demonstrated configuration features four zHBM stacks mounted over a single XPU.
Enabling zHBM requires advanced packaging technologies. Samsung is actively developing Wafer-on-Wafer (WoW) and Hybrid Cube Bonding (HCB) techniques to achieve ultra-high I/O density, ultimately facilitating a unified SoC-DRAM design.
The evolution of the HBM base die through advanced logic processes marks a transformative shift in AI-era memory architecture. Migrating the base die to advanced nodes unlocks unprecedented flexibility beyond traditional data routing and testing. Samsung’s three-phase roadmap operationalizes this vision: first, reclaiming area and offloading functions via custom HBM; second, expanding capacity and integrating auxiliary controllers and processing elements; and third, achieving full 3D vertical integration with zHBM. Together, these advances position the base die as an active, intelligent partner to the XPU. By mastering advanced packaging and unified SoC-DRAM co-design, Samsung aims to overcome the power, area, and capacity constraints limiting current AI systems, paving the way for greater efficiency, performance, and scalability in the years ahead.
About the author: Hassan Mujtaba is a software engineer by training and a PC enthusiast by passion. He serves as Wccftech’s Senior Editor for the hardware section, specializing in deep-dive technical analysis of next-generation CPU and GPU architectures, motherboards, and cooling solutions. With years of industry experience, his work encompasses breaking news on emerging technologies, extensive hands-on reviews, and benchmarking.
Follow Wccftech on Google to receive more of our news coverage in your feeds.