HBM Issues In AI Systems


All systems face limitations, and as one limitation is removed, another is revealed that had remained hidden. It is highly likely that this game of Whac-A-Mole will play out in AI systems that employ high-bandwidth memory (HBM). Most systems are limited by memory bandwidth. Compute systems in general have maintained an increase in memory interface performance that barely matches the gains in... » read more

HBM2E Memory: A Perfect Fit For AI/ML Training


Artificial Intelligence/Machine Learning (AI/ML) growth proceeds at a lightning pace. In the past eight years, AI training capabilities have jumped by a factor of 300,000 (10X annually), driving rapid improvements in every aspect of computing hardware and software. Memory bandwidth is one such critical area of focus enabling the continued growth of AI. Introduced in 2013, High Bandwidth Memo... » read more

High-Performance Memory For AI And HPC


Frank Ferro, senior director of product management at Rambus, examines the current performance bottlenecks in high-performance computing, drilling down into power and performance for different memory options, and explains what are the best solutions for different applications and why. » read more

HBM2E and GDDR6: Memory Solutions for AI


Artificial Intelligence/Machine Learning (AI/ML) growth proceeds at a lightning pace. In the past eight years, AI training capabilities have jumped by a factor of 300,000 driving rapid improvements in every aspect of computing hardware and software. Meanwhile, AI inference is being deployed across the network edge and in a broad spectrum of IoT devices including in automotive/ADAS. Training and... » read more

What Engineers Are Reading And Watching


By Brian Bailey And Ed Sperling An important indicator of where the chip industry is heading is what engineers are reading and what videos they are watching. While some subjects remain on top, such as the level of interest in the latest manufacturing technologies, other areas come and go. The stories with the biggest traffic numbers are almost identical to last year. Readers want to know wh... » read more

HBM2E: The E Stands for Evolutionary


Samsung introduced the first memory products in March that conform to JEDEC’s HBM2E specification, but so far nothing has come to market—a reflection of just how difficult it is to manufacture this memory in volume. Samsung’s new HBM2E (sold under the Flashbolt brand name, versus the older Aquabolt and Flarebolt brands), offers 33% better performance over HBM2 thanks to doubling the de... » read more

HBM2e Offers Solid Path For AI Accelerators


Today, AI processors are so blazingly fast that they’re constantly having to wait for data from memory. Unfortunately, with the status quo, memory is just not fast enough to unleash the true performance of those new and highly advancing AI processors. In simple terms, AI processor performance is rapidly growing, and memory is not keeping up. This creates a bottleneck, or what Rambus calls the... » read more

Week In Review: Design, Low Power


Tools & IP Cadence uncorked the latest version of JasperGold formal verification platform, providing improvements to the proof-solver algorithm and orchestration by using machine learning to select and parameterize solvers to enable faster first-time proofs and optimize successive runs for regression testing. Additionally, it increases design compilation capacity by over 2x with 50% reduct... » read more

GDDR6 – HBM2 Tradeoffs


Steven Woo, Rambus fellow and distinguished inventor, talks about why designers choose one memory type over another. Applications for each were clearly delineated in the past, but the lines are starting to blur. Nevertheless, tradeoffs remain around complexity, cost, performance, and power efficiency.   Related Video Latency Under Load: HBM2 vs. GDDR6 Why data traffic and bandw... » read more