Samsung Unveils Three-Stage HBM Roadmap, zHBM Targets 70% Power Cut and 230% Bandwidth Gain

At the Hot Chips technical conference, Samsung Electronics unveiled a three-stage evolution roadmap for high-bandwidth memory (HBM), with the ultimate goal being a new architecture called zHBM, in which DRAM is stacked directly vertically atop compute chips such as GPUs and TPUs, eliminating the 2.5D interposer. In a presentation by Samsung's DRAM design team, the company disclosed that, compared with standard HBM4E, the zHBM approach can achieve a 70% reduction in power consumption, a 230% increase in DRAM bandwidth, save 100W per DRAM module, and free up an additional 8.3% power headroom for the GPU.

HBM consists of a core die (C-die) and a base die (B-die). The former contains DRAM memory cells that can be vertically stacked up to 16 layers, while the latter sits at the bottom of the stack, handling DRAM control functions and communicating with the compute chip through a physical interface layer. The two are connected via through-silicon vias (TSVs). Current HBM4 stacks already exceed 3TB/s of bandwidth per stack, HBM4E advances into the 4TB/s range, and HBM5 doubles bandwidth over HBM4 while exceeding 60GB of capacity. Bandwidth scaling faces two hard constraints: the physical limits of TSV count and pitch, and the upper limits on the number and speed of physical interface I/Os within the base die.

The first stage of the roadmap focuses on freeing up silicon area for the compute chip. Samsung has applied its D1c and 4nm logic processes to the HBM4 base die to reduce power consumption and shrink the effective area. To address the hot-spot issues arising from area reduction, Samsung introduced the Heat Path Block technology, which can lower peak temperatures by more than 35% and covers 50% of the physical interface area.

The second stage is the functional expansion phase. Samsung plans to integrate memory expansion controllers and physical interfaces into the idle silicon area of the base die, expanding the system's available memory capacity through external LPDDR or HBM solutions, in response to the growing KV cache demand driven by the expanding context windows of large AI models. Additionally, Samsung proposes integrating some compute processing units on the base die to offload certain computations to the memory side, a configuration referred to as AHBM. This stage also includes integrating reliability, availability, and serviceability sensors, real-time telemetry capabilities, and on-chip self-test features.

The third stage is the ultimate zHBM form. Current mainstream AI systems use a 2.5D packaging architecture, where GPUs and HBM are placed side by side on the same interposer. zHBM verticalizes this structure by placing DRAM stacks directly atop the compute chip. Key features of zHBM described by Samsung include distributed I/O to minimize data transmission distances, a 3D structure that eliminates traditional 2D interfaces, an I/O power target of approximately 0.5 pJ/bit, a bandwidth increase of more than 2.3 times, and a system thermal headroom of 100W. The demonstration configuration shows four zHBM stacks atop a single compute chip. Samsung is developing wafer-to-wafer (WoW) and hybrid cube bonding (HCB) packaging technologies to achieve ultra-high I/O density.