Micron presented its evolving memory architectures tailored for artificial intelligence workloads at Hot Chips 2026, highlighting current packaging opportunities.

Kicking off Hot Chips 2026 Day 0, which has evolved into a significant standalone event, Micron presented its vision for evolving memory architectures tailored to artificial intelligence. The company highlighted how high-bandwidth memory (HBM) and advanced packaging have moved to the center of AI system design.

Micron’s presentation opened with the OpenAI compute-efficient frontier to contextualize why memory has become central to language model scaling. The company pointed to the power-law relationship between model size, dataset size, and training compute, arguing that all three must scale in tandem for large language model performance to improve. Memory serves as the critical resource linking these factors. Addressing the “memory wall,” Micron noted that AI accelerator TFLOPS roughly triple every two years, while the bandwidth of 2.5D-attached memory such as HBM climbs at less than double that rate over the same period. This widening gap has made memory technology a primary bottleneck limiting AI system performance.

Micron framed HBM as the essential compute-memory bridge, tracking rising cube capacity, bandwidth, and energy efficiency across generations from HBM2e through HBM5. Illustrating the physical footprint of modern accelerators, the company showed a typical GPU package exceeding 12,000 square millimeters when accounting for the base die and eight HBM4 instances. With 12-high HBM4 configurations, memory silicon can occupy more than eight times the surface area of the GPU die itself. By raising the memory bandwidth ceiling, HBM shifts operational boundaries, enabling memory-intensive AI workloads to run faster before hitting capacity limits.

The presentation detailed HBM’s architectural evolution, tracing a product roadmap from HBM1 through HBM4. Each generation increases data rates, pseudo-channels, and stack height to deliver greater bandwidth and capacity. Unlike standard ECC RDIMMs, HBM integrates directly into heterogeneous systems as a System-in-Package (SiP). Within each DRAM die, multiple independent channels operate with two pseudo-channels per channel. HBM3E features 128 banks per die, while HBM4 doubles this to 256. A base die sits between the host processor and the DRAM stack, utilizing a microbump PHY on the host side and a 3D TSV PHY connecting to the stack. These components communicate over highly dense point-to-point interposer connections, which scale from 1K I/O on HBM3E to 2K I/O on HBM4.

Micron outlined the inherent performance-versus-capacity trade-offs, noting that specific metrics regarding PCB area and power consumption were deferred to future discussions. The company also highlighted the silicon cost of achieving this speed: the combined overhead of design architecture, advanced packaging, and manufacturing complexity consumes approximately three times as much silicon to deliver equivalent HBM3E capacity compared to DDR5. To support ongoing AI innovation, Micron emphasized advancements in performance, capacity, and power efficiency. Faster I/O links and larger interposers continue to drive SiP development, incorporating memory-optimized SERDES, die-to-die PHYs, and co-packaged optics. The packaging roadmap also includes larger SiP form factors such as CoWoS-L and CoWoS-R, alongside emerging glass substrates.

Reliability and RAS challenges were addressed next, focusing on chip-package interactions. Coefficient-of-thermal-expansion mismatches between materials in heterogeneously integrated HBM devices generate significant thermo-mechanical stress—a challenge previously encountered during the development of Cerebras’ Wafer Scale Engine. ECC coverage now spans two layers: beginning with HBM3, it provides 16 meta-bits per 256-bit access at the system level, supplemented by on-die symbol-based Reed-Solomon ECC. Thermal management remains equally critical, as increased DRAM activity, taller stacks, and enhanced base die functionality generate substantial heat. Micron identified liquid cooling, hybrid bonding, and modifications to solder and side molding compounds as key innovations required to sustain bandwidth and capacity scaling.

Collectively, these developments illustrate Micron’s approach to balancing expanded HBM bandwidth and capacity against the packaging, thermal, and reliability constraints of placing massive memory arrays adjacent to compute units. The industry-wide shift toward HBM and the surging demand from AI data centers are currently driving a memory supply crunch. As Micron expands its focus beyond raw capacity into advanced packaging and system-level integration, the trade-offs among HBM bandwidth, silicon cost, thermal performance, and reliability will dictate accelerator design for the coming generations. This landscape sets the stage for further innovation, including custom HBM configurations featuring base dies that integrate dedicated memory controllers and compute elements.