SK hynix details HBM4 packaging and hybrid bonding roadmap at Hot Chips 2026
A 775-micron height limit is forcing memory makers to rethink die stacking, and the mechanics explain why AI workloads are memory-bound.
SK hynix used its presentation at the Hot Chips 2026 conference at Stanford to lay out how it is packaging high-bandwidth memory for AI accelerators and where the physical limits of die stacking now sit, ServeTheHome reported on 23 August. The talk, delivered on the opening day of the three-day event, focused on 3D stacking, thermal management and the bonding techniques that connect layers of DRAM within an HBM cube.
According to the presentation, HBM4 delivers 2,048GB/s of bandwidth per stack, double the 1,024GB/s of HBM3E, with the number of data I/Os rising from 1,024 to 2,048. A stack sits within a 12.8mm by 11mm footprint at a Z-height of 775 microns, contains more than 20,000 through-silicon vias and 16,148 base micro-bumps, and offers 48GB of capacity per cube. SK hynix put the power-efficiency gain over the previous generation at more than 40 percent. Twelve-high stacks are in production, with 16-high stacks under qualification.
The company outlined three bonding approaches. Thermo-compression bonding with non-conductive film offers high productivity and low thermal resistance but is sensitive to chip warpage. Mass reflow with molded underfill handles warpage better at the cost of higher thermal resistance. Hybrid bonding, which forms direct copper-to-copper and oxide-to-oxide connections after room-temperature placement and annealing above 200 degrees Celsius, is positioned as the route to stacks of 20 or more dies.
The presentation also traced the bandwidth progression from HBM2E at 460GB/s through HBM3 at 717GB/s and HBM3E at 1,024GB/s, and listed TSV uniformity, copper contamination, thin-die warpage and thermal resistance as the main engineering challenges ahead.
Why it matters
Accelerator compute is growing faster than the memory feeding it, so packaging has become as strategic as transistor scaling. A fixed stack height means every extra layer must come from thinner dies and tighter bonds, pushing the industry toward hybrid bonding despite its manufacturing difficulty. Decisions made here determine how much model state a single GPU can hold and how fast it can be read. For learners, it is a concrete example of the memory wall that shapes model architecture and inference cost.
Accelerator compute is growing faster than the memory feeding it, so packaging has become as strategic as transistor scaling.
- HBM2E460 GB/s
- HBM3717 GB/s
- HBM3E1,024 GB/s
- HBM42,048 GB/s
Figures: SK hynix Hot Chips 2026 presentation, as reported by ServeTheHome
What you can learn from this
- Bandwidth, not raw compute, often limits AI workloads. A matrix multiplication can only run as fast as the operands arrive from memory, and large language model inference in particular moves enormous weight matrices through the chip for every token. If memory bandwidth grows more slowly than arithmetic throughput, the arithmetic units sit idle. That is the memory wall, and it is why doubling per-stack bandwidth between HBM3E and HBM4 matters more to inference speed than many headline FLOPS figures. Techniques such as quantisation and key-value caching exist largely to reduce the bytes that must cross this boundary per token.
- HBM achieves bandwidth through width rather than clock speed. Instead of driving a few pins very fast, an HBM stack exposes thousands of data lines, 2,048 in HBM4, over a short interposer path to the processor. Wide, short and slow-per-pin interfaces use less energy per bit than long, narrow, fast ones, which is where the reported efficiency gains come from. The tradeoff is that this design requires advanced packaging with extremely fine bump pitches.
- Through-silicon vias are the vertical wiring of a stack. A TSV is a conductor etched straight through a thinned DRAM die so that signals and power can pass from one layer to the next without going around the edge. Tens of thousands of them per stack must line up across every layer, which is why uniformity is listed as a challenge. Any misalignment or void raises resistance and can cut a channel off entirely, and because a stack is tested as a unit, one bad layer can scrap the whole cube.
- Bonding method sets the ceiling on stack height. Conventional bonding joins dies with micro-bumps and a filler material, and each layer adds bump height and fill thickness. With the overall stack capped at 775 microns, more layers require thinner dies and thinner gaps, and eventually the bumps themselves become the obstacle. Hybrid bonding removes the bumps, joining copper pads directly, which is why it is the enabling step for 20-high stacks.
- Heat has to escape through the same stack it is generated in. Each DRAM die dissipates power, and the layers above it insulate it from the heatsink. Bonding materials with high thermal resistance make this worse, so a method that handles warpage well may hold heat in, and hotter DRAM must refresh more often, which costs bandwidth. Engineering a stack is therefore a balance between mechanical yield, thermal path and electrical density rather than a single optimisation.
We teach this
How to use this in practice
- Compute the arithmetic intensity of a model you use. Pick an open-weight model, note its parameter count and precision, and estimate the bytes of weights read per generated token. Divide a GPU's memory bandwidth by that figure to get a rough tokens-per-second ceiling, then repeat the calculation at 8-bit and 4-bit precision. Done means a short calculation showing why a 7-billion-parameter model at 16-bit precision is bandwidth-bound on a card you can name, and how much headroom each lower precision buys.
- Draw an HBM stack in cross-section. Sketch a base logic die, a stack of DRAM dies above it, TSVs running vertically, and the interposer connecting the stack to the processor. Label the 775-micron height, show the micro-bumps or hybrid-bond interfaces between layers, and mark where heat flows out. Done means a diagram where you can point to the reason adding a layer is hard without changing the bonding method, and explain which interface each of the three bonding approaches replaces.
- Inspect memory bandwidth on your own hardware. On a machine with a discrete GPU, run a simple bandwidth test, such as a memory-copy microbenchmark from a CUDA or ROCm sample, or a Python script using a GPU array library to time a large tensor copy. Compare the measured figure with the card's specification sheet, and try a few transfer sizes to see where throughput plateaus. Done means a note recording the measured GB/s, the rated GB/s, the percentage you achieved and the smallest transfer size at which you reached it.
- Profile one inference run for memory stalls. Load a small model with a framework profiler enabled and record where time goes across a few generated tokens. Look for kernels that report low compute utilisation but high memory traffic, and note how the picture changes between processing the prompt and generating each subsequent token. Done means you can name the top two memory-bound operations, say which phase of inference they dominate, and explain in a sentence how faster HBM would change the result.
Sources
- SK hynix HBM Packaging at Hot Chips 2026 — ServeTheHome
Our reporting is an original summary; full coverage is at the links above.
Don't just read about it — build it.
Square 1 teaches the skills behind the headlines, with every line of your work graded by AI. Find your starting point in 3 minutes.
Get your free skill report