A new technical paper titled “3D Stacked HBM and Compute Accelerators for LLM: Optimizing Thermal Management and Power Delivery Efficiency” was published by a researcher from Georgia Institute of Technology and SK Hynix.
Abstract
“Advanced packaging is becoming essential for designing hardware accelerators for large language models (LLMs). Different architectures such as 2.5D integration of memory with logic have been proposed, however the bandwidth limits the throughput of the complete system. Recent works have proposed memory on logic systems, where high bandwidth memory (HBM) can be 3D stacked on top of logic to improve the throughput by 64× and energy efficiency by 3×. However, the high-power consumption of logic dies and the high thermal resistance of HBM can result in thermal and power delivery challenges in such heterogeneously integrated stacks. In this work, we explore various design configurations such as logic-on-memory, and memory-on-logic, and consider some hybrid configurations. Further, accurate modeling of DRAM dies is performed, and mitigation strategies are proposed to further improve the throughput by 16% for memory-on-logic, reduce the IR drop for logic-on-memory system by 640 mV, and get 4× higher throughput for a hybrid system compared to the 2.5D integrated system.”
Find the technical paper here. October 2025.
Sharda, Janak, Madison Manley, Jungyoun Kwak, Chinsung Park, Muhannad Bakir, and Shimeng Yu. “3D Stacked HBM and Compute Accelerators for LLM: Optimizing Thermal Management and Power Delivery Efficiency.” IEEE Journal on Exploratory Solid-State Computational Devices and Circuits (2025). Creative commons license.
Leave a Reply