Scaling DRAM Technology To Meet Future Demands: Challenges And Opportunities


Since the invention of the 1T1C bit cell more than 50 years ago, DRAMs have become the main memory of choice for processors in computer systems and many consumer electronics devices. As new use computing paradigms have been created, including 3D graphics, cloud computing, smart phones, and AI processing, specialized processors and DRAM memories have been developed that are optimized for these u... » read more

LLM Inference: Core Bottlenecks Imposed By Memory, Compute Capacity, Synchronization Overheads (NVIDIA)


A new technical paper titled "Efficient LLM Inference: Bandwidth, Compute, Synchronization, and Capacity are all you need" was published by NVIDIA. Abstract "This paper presents a limit study of transformer-based large language model (LLM) inference, focusing on the fundamental performance bottlenecks imposed by memory bandwidth, memory capacity, and synchronization overhead in distributed ... » read more

Five Questions To Ask When Selecting A Temporary Bonding And Debonding System


High-bandwidth memory blocks (HBM) memory, microprocessors, field-programmable gate arrays (FPGA), AI accelerators, and other devices used in advanced system-level packaging all rely on temporary bonding and debonding systems to shrink their footprint. Understanding which properties play the most crucial role in device reliability and efficient production will ensure you are maximizing your yie... » read more

Examination Of Thermal Issues Related to Hybrid Bonding of 3D-Stacked HBM


A new technical paper titled "Thermal Issues Related to Hybrid Bonding of 3D-Stacked High Bandwidth Memory: A Comprehensive Review" was published by researchers at Chungbuk National University. Abstract "High-Bandwidth Memory (HBM) enables the bandwidth required by modern AI and high-performance computing, yet its three dimensional stack traps heat and amplifies thermo mechanical stress. We... » read more

Co-Designing Data Center Architecture To Support LLMs (Intel, Georgia Tech)


A new technical paper titled "Scaling Intelligence: Designing Data Centers for Next-Gen Language Models" was published by Intel Corporation and Georgia Tech. An excerpt from the paper's abstract: "Our work provides a comprehensive co-design framework that jointly explores FLOPS, HBM bandwidth and capacity, multiple network topologies (two-tier vs. FullFlat optical), the size of the scale-ou... » read more

System-Level Approach To Reducing HBM Cost for AI inference (RPI, IBM)


A new technical paper titled "Breaking the HBM Bit Cost Barrier: Domain-Specific ECC for AI Inference Infrastructure" was published by researchers at Rensselaer Polytechnic Institute and IBM. Abstract "High-Bandwidth Memory (HBM) delivers exceptional bandwidth and energy efficiency for AI workloads, but its high cost per bit, driven in part by stringent on-die reliability requirements, pose... » read more

Physics Limits Interposer Line Lengths


Electrical interposers provide a convenient surface for mounting multiple chips within a single package, but even though interposer lines theoretically can be routed anywhere, insertion losses limit their practical length. Lines on interposers — and on silicon interposers in particular — can be exceedingly narrow. Having a small cross-section makes such lines resistive, degrading signals... » read more

HBM Roadmap: Next-Gen High-Bandwidth Memory Architectures (KAIST’s TERALAB)


A new technical paper titled "HBM Roadmap Ver 1.7 Workshop" was published by researchers at KAIST’s TERALAB. The 371-page paper provides an overview of next-generation HBM architectures based on current technology trends, as well as many technology insights. Find the technical paper here or here.  Published June 2025. Advising Professor : Prof. Joungho Kim. Fig. 1: Thermal Manag... » read more

The Best DRAMs For Artificial Intelligence


Artificial intelligence (AI) involves intense computing and tons of data. The computing may be performed by CPUs, GPUs, or dedicated accelerators, and while the data travels through DRAM on its way to the processor, the best DRAM type for this purpose depends on the type of system that is performing the training or inference. The memory challenge facing engineering teams today is how to keep... » read more

Arithmetic Intensity In Decoding: A Hardware-Efficient Perspective (Princeton University)


A new technical paper titled "Hardware-Efficient Attention for Fast Decoding" was published by researchers at Princeton University. Abstract "LLM decoding is bottlenecked for large batches and long contexts by loading the key-value (KV) cache from high-bandwidth memory, which inflates per-token latency, while the sequential nature of decoding limits parallelism. We analyze the interplay amo... » read more

← Older posts Newer posts →