Scaling AI/ML Training Performance With HBM2E Memory


In my April SemiEngineering Low Power-High Performance blog, I wrote: “Today, AI/ML neural network training models can exceed 10 billion parameters, soon it will be over 100 billion.” “Soon” didn’t take long to arrive. At the end of May, OpenAI unveiled a new 175-billion parameter GPT-3 language model. This represented a more that 100X jump over the size of GPT-2’s 1.5 billion param... » read more

Memory Access In AI Systems


Memory access is a key consideration in AI system design. Ron Lowman, strategic marketing manager for IP at Synopsys, talks about how memory affects overall power consumption, why partitioning of on-chip and off-chip is so critical to performance and power, and how this changes from the cloud to the edge. » read more

High-Speed SerDes At 7/5nm


Manmeet Walia, senior product marketing manager at Synopsys, talks with Semiconductor Engineering about how to optimize PHYs for integration on all four corners of an SoC, as well as the PPA implications of moving large amounts of data across and around a chip. » read more

Ensuring HBM Reliability


Igor Elkanovich, CTO of GUC, and Evelyn Landman, CTO of proteanTecs, talk with Semiconductor Engineering about difficulties that crop up in advanced packaging, what’s redundant and what is not when using high-bandwidth memory, and how continuous in-circuit monitoring can identify potential problems before they happen. » read more

2.5D Architecture Answers AI Training’s Call for “All of the Above”


The impact of AI/ML grows daily impacting every industry and touching the lives of everyone. In marketing, healthcare, retail, transportation, manufacturing and more, AI/ML is a catalyst for great change. This rapid advance is powerfully illustrated by the growth in AI/ML training capabilities which have since 2012 grown by a factor of 10X every year. Today, AI/ML neural network training mod... » read more

AI Requires Tailored DRAM Solutions


For over 30 years, DRAM has continuously adapted to the needs of each new wave of hardware spanning PCs, game consoles, mobile phones and cloud servers. Each generation of hardware required DRAM to hit new benchmarks in bandwidth, latency, power or capacity. Looking ahead, the 2020s will be the decade of artificial intelligence/machine learning (AI/ML) touching every industry and applicatio... » read more

HBM Issues In AI Systems


All systems face limitations, and as one limitation is removed, another is revealed that had remained hidden. It is highly likely that this game of Whac-A-Mole will play out in AI systems that employ high-bandwidth memory (HBM). Most systems are limited by memory bandwidth. Compute systems in general have maintained an increase in memory interface performance that barely matches the gains in... » read more

HBM2E Memory: A Perfect Fit For AI/ML Training


Artificial Intelligence/Machine Learning (AI/ML) growth proceeds at a lightning pace. In the past eight years, AI training capabilities have jumped by a factor of 300,000 (10X annually), driving rapid improvements in every aspect of computing hardware and software. Memory bandwidth is one such critical area of focus enabling the continued growth of AI. Introduced in 2013, High Bandwidth Memo... » read more

High-Performance Memory For AI And HPC


Frank Ferro, senior director of product management at Rambus, examines the current performance bottlenecks in high-performance computing, drilling down into power and performance for different memory options, and explains what are the best solutions for different applications and why. » read more

HBM2E and GDDR6: Memory Solutions for AI


Artificial Intelligence/Machine Learning (AI/ML) growth proceeds at a lightning pace. In the past eight years, AI training capabilities have jumped by a factor of 300,000 driving rapid improvements in every aspect of computing hardware and software. Meanwhile, AI inference is being deployed across the network edge and in a broad spectrum of IoT devices including in automotive/ADAS. Training and... » read more

← Older posts