Most limits in advanced chipmaking come from physics that is hard to explain without equations. This one comes from a ruler. The stack of memory sitting beside an AI processor cannot be taller than 775 microns, which is about eight times the thickness of a sheet of paper, and that number is now shaping the roadmap of the most valuable component in the industry.
Jaesik Lee, vice president of package engineering at SK hynix America, laid out the arithmetic at Hot Chips 2026 on August 23. Along the way he all but confirmed that hybrid bonding, the packaging transition the memory industry has been anticipating for years, will not arrive with HBM4E.
The reason for the ceiling is almost disappointingly mundane. A standard 300mm logic wafer is 775 microns thick.
When a cold plate is fitted to a GPU package, both the logic die and the memory stacks are ground back to expose bare silicon so heat can escape into the plate. If the memory cube were any taller, it would stand proud of the processor beside it and the cold plate would no longer sit flat against both.
“That’s the kind of limit that we can go up so far, because the logic wafer thickness is also 775 microns,” Lee said.
That ceiling is not fixed by nature, and JEDEC has already moved it once. The HBM4 standard raised the package thickness allowance from the 720 microns that held through HBM3E, which is precisely why the urgency around hybrid bonding faded.
Inside that budget, every extra layer of DRAM has to come from somewhere. Twelve-high HBM4 is in mass production; sixteen-high is in customer qualification at 48GB per cube, and getting there meant thinning the core dies to roughly 50 microns and halving the gap between them.
Thinning the silicon has a consequence that does not show up on a spec sheet. A stack made of thinner dies contains proportionally more oxide, and oxide conducts heat poorly compared with silicon.
At the same time, the interface is getting hotter for reasons unrelated to stacking. Pin speeds have climbed from 1 Gbps in early HBM to 8 Gbps in HBM4, concentrating more power into the same footprint.
SK hynix’s own figures put the thermal burden 2.2 times higher across the generations shown, while the number of stacked dies doubles every two generations. Thinner conductors, more insulator, more power, same square millimeters.
Today that stack is assembled with mass reflow-molded underfill, or MR-MUF, which places all the dies with pick-and-place tooling and joins them in a single reflow step. Lee described the main manufacturing challenge of sixteen-high plainly: filling gaps that have shrunk by half while controlling warpage on dies under 50 microns thick.
Hybrid bonding is supposed to be the way out. It removes the micro-bumps between layers entirely, joining flattened copper pads and oxide surfaces at room temperature and then relying on copper’s thermal expansion during a cure step to complete the bond.
The payoff is substantial. Without micro-bumps, core dies can be up to 24 percent thicker at twenty-high, thermal resistance falls roughly 35 percent against MR-MUF at that height, and bump pitch drops below 18 microns compared with the 30 microns MR-MUF uses today.
“This is a very simple process, but in reality it’s really challenging,” Lee said. “We are talking about 16 layers and 20 layers that we need to make the hybrid bonding, so it’s very different from the one-layer stacking.”
The schedule keeps slipping accordingly. Samsung publicly committed to hybrid bonding for HBM4 in May of last year while SK hynix held the copper-to-copper technique in reserve behind advanced MR-MUF, and the JEDEC thickness relaxation then removed the immediate pressure.
Industry discussion has now moved to raising the ceiling again, to somewhere between 825 and 900 microns for twenty-high stacks, which would push the crossover point out once more. Every time JEDEC has lifted the limit, MR-MUF has survived another generation.
SK hynix reportedly placed a first mass-production hybrid bonding order in March, a single inline system pairing Applied Materials and Besi tools worth around 20 billion won, or about $15 million. Counterpoint Research expects the technique to enter full-scale HBM production with HBM5 around 2029 to 2030, and stacks of twenty layers and above remain at the research stage.
Since the stack cannot get taller, the other line of attack is getting the heat out sideways. Lee spent part of his session on iHBM, SK hynix’s cooling architecture first shown in May.
The idea is to embed thermally conductive but electrically insulating blocks into the base die’s die-to-die PHY region, the interface hotspot where power density peaks, for a claimed reduction in thermal resistance of more than 30 percent.
Lee’s slides benchmarked it against Samsung’s Heat Path Block, which routes heat out through dedicated pillars, and Micron’s base-die circuit redesign, which claims more than 20 percent better energy efficiency. All three are vendor claims measured on different metrics, and none of them is close to shipping.
The SK hynix and Samsung designs are both aimed at HBM5, with neither expected in mass production before 2028. There is a further catch that says something about how the industry now works.
Because the blocks sit inside the package alongside the die-to-die interface, they have to be optimized with the customer’s own design and cannot be retrofitted to a generation already in progress. “It’s a kind of good option that we can do, but this is not something that we can apply to the generation that we already have in design,” Lee said.
The sharpest moment came during questions, from Tanj Bennett of SemiAnalysis, who challenged the premise that taller is better at all. DRAM operating at the cell level delivers on the order of 20 TB/s per square centimeter, he noted, while a twenty-high stack tops out around 4 TB/s, and HBM consumes far more manufacturing capacity than equivalent DDR5 or LPDDR.
“As you get to 20 high, the average speed of that memory is slower than DDR5,” Bennett said. “Why is it better to be using the height of the HBM stack instead of intelligently placing cheaper memory around it?”
Lee’s answer conceded more than it defended. Training workloads need both bandwidth and capacity, he said, but inference may well split the difference, keeping the KV cache in high-bandwidth memory and offloading the rest to LPDDR.
That split is not hypothetical. Nvidia’s Vera Rubin platform already pools LPDDR5X alongside HBM4 over NVLink-C2C for exactly this purpose, and the High Bandwidth Flash specification SK hynix co-developed with Sandisk extends the same tiering idea down to NAND.
The commercial stakes behind this technical caution are considerable. SK hynix holds roughly 70 percent of Nvidia’s HBM orders for the Vera Rubin generation, and every one of those cubes will be stacked with MR-MUF rather than hybrid bonding.
Zoom out from the package and the same constraint appears at every scale. TrendForce reported on August 21 that thermal design power for individual AI chips from Nvidia, AMD and Google has passed 1 kW, with rack-scale systems drawing hundreds of kilowatts.
Liquid cooling penetration among AI chips is projected to climb from around 33 percent in 2025 to 53 percent in 2026 and approach 60 percent in 2027. KAIST professor Kim Joung-ho has warned that thermal limits could constrain AI expansion outright as both logic and memory move to three-dimensional stacking.
His proposed response is worth noting for its circularity: use AI to design the packages. Kim points to an “HBM Design AI Agent” running in his own laboratory as an early example of design automation optimizing structures for heat dissipation.
Seoul National University of Science and Technology professor Kim Sung-dong made a related observation, that the industry is now prioritizing thermal management over further raw performance gains, and pointed to co-packaged optics and system-technology co-optimization as the leading approaches.
What Lee described in San Francisco is therefore less a memory problem than a preview. For thirty years the binding constraint in computing was how small a transistor could be made.
The constraint now is how much heat can be removed from a fixed volume, and 775 microns is simply the most precisely measured expression of it. The next generation of AI hardware will be shaped less by what engineers can fabricate than by what they can cool.
References
- Luke James, “Hot Chips 2026: SK hynix pushes hybrid bonding to HBM5 as AI memory hits 775-micron ceiling,” Tom’s Hardware, August 24, 2026.
- Jaesik Lee, SK hynix package engineering presentation, Hot Chips 2026, August 23, 2026.
- “Chip Packaging Experts Flag Heat as AI’s Biggest Bottleneck,” TrendForce, August 21, 2026 (citing ETNews, August 20, 2026).
- “Liquid Cooling Becomes Standard for High-End AI Infrastructure,” TrendForce, August 17, 2026.
