中文 English

Author's Latest Posts


High-Performance SerDes Enable The 5G Wireless Edge


Investment at the core of the global internet is red hot. The number of hyperscale data centers jumped to 700 worldwide at the end of 2021, and with more than 300 in the pipeline, should rise to over 1000 by 20241. In the span of five years, total hyperscale data centers will have doubled. And as the raw number shoots up, more powerful compute and networking hardware is rapidly being deployed, ... » read more

It’s Official: HBM3 Dons The Crown Of Bandwidth King


With the publishing of the HBM3 update to the High Bandwidth Memory (HBM) standard, a new king of bandwidth is crowned. The torrid performance demands of advanced workloads, with AI/ML training leading the pack, drive the need for ever faster delivery of bits. Memory bandwidth is a critical enabler of computing performance, thus the need for the accelerated evolution of the standard with HBM3 r... » read more

On the Road To Higher Memory Bandwidth


In the decade since HBM was first announced, we’ve seen two-and-a-half generations of the standard come to market. HBM’s “wide and slow” architecture debuted first at a data rate of 1 gigabit per second (Gbps) running over a 1024-bit wide interface. The product of that data rate and that interface width provided a bandwidth of 128 gigabytes per second (GB/s). In 2016, HBM2 doubled the s... » read more

GDDR6 Memory On The Leading Edge


With the accelerating growth in data traffic, it is unsurprising that the number of hyperscale data centers keeps rocketing skyward. According to analysts at the Synergy Research Group, in nine months (Q2’20 to Q1’21), 84 new hyperscale data centers came online bringing the total worldwide to 625. Hyperscaler capex set a record $150B over the last four quarters eclipsing the $121B spent in ... » read more

Accelerating AI/ML Inferencing With GDDR6 DRAM


The origins of graphics double data rate (GDDR) memory can be traced to the rise of 3D gaming on PCs and consoles. The first graphics processing units (GPU) packed single data rate (SDR) and double data rate (DDR) DRAM – the same solution used for CPU main memory. As gaming evolved, the demand for higher frame rates at ever higher resolutions drove the need for a graphics-workload specific me... » read more

HBM2E Raises The Bar For AI/ML Training


The largest AI/ML neural network training models now exceed an enormous 100 billion parameters. With the rate of growth over the last decade on a 10X annual pace, we’re headed to trillion parameter models in the not-too-distant future. Given the tremendous value that can be derived from AI/ML (it is mission critical to five of six of the top market cap companies in the world), there has been ... » read more

GDDR6 Memory For Life On The Edge


With the torrid growth in data traffic, it is unsurprising that the number of hyperscale data centers has grown apace. According to analysts at the Synergy Research Group, in July of this year there were 541 hyperscale data centers worldwide. That represents a doubling in the number since 2015. Even more striking, there are an additional 176 in the pipeline, so the breakneck growth in hyperscal... » read more

Scaling AI/ML Training Performance With HBM2E Memory


In my April SemiEngineering Low Power-High Performance blog, I wrote: “Today, AI/ML neural network training models can exceed 10 billion parameters, soon it will be over 100 billion.” “Soon” didn’t take long to arrive. At the end of May, OpenAI unveiled a new 175-billion parameter GPT-3 language model. This represented a more that 100X jump over the size of GPT-2’s 1.5 billion param... » read more

An Expanding Application Space For GDDR6 Memory


The origins of graphics double data rate (GDDR) memory can be traced to the rise of 3D gaming on PCs and consoles. The first GPUs used single data rate (SDR) and double data rate (DDR) DRAM, the same memory used for CPU main memory. The quest for higher frame rates at higher resolutions drove the need for a graphics-workload specific memory solution. The commercial success of gaming PCs and con... » read more

2.5D Architecture Answers AI Training’s Call for “All of the Above”


The impact of AI/ML grows daily impacting every industry and touching the lives of everyone. In marketing, healthcare, retail, transportation, manufacturing and more, AI/ML is a catalyst for great change. This rapid advance is powerfully illustrated by the growth in AI/ML training capabilities which have since 2012 grown by a factor of 10X every year. Today, AI/ML neural network training mod... » read more

← Older posts