Redefining XPU Memory For AI Data Centers Through Custom HBM4: Part 1

By Archana Cheruliyil - 26 Nov, 2024 - Comments: 0

This is the first of a three-part series on HBM4 and gives an overview of the HBM standard. Part 2 will provide insights on HBM implementation challenges, and part 3 will introduce the concept of a custom HBM implementation. Relentless growth in data consumption Recent advances in deep learning have had a transformative effect on artificial intelligence (AI) and the ever-increasing volume of ... » read more

HBM Options Increase As AI Demand Soars

By Laura Peters - 21 Nov, 2024 - Comments: 0

High-bandwidth memory (HBM) sales are spiking as the amount of data that needs to be processed quickly by state-of-the-art AI accelerators, graphic processing units, and high-performance computing applications continues to explode. HBM inventories are sold out, driven by massive efforts and investments in developing and improving large language models such as ChatGPT. HBM is the memory of ch... » read more

Extending The DDR5 Roadmap With MRDIMM

By Zaman Mollah - 14 Nov, 2024 - Comments: 0

Given the voracious memory bandwidth and capacity demands of Gen AI and other advanced workloads, we’ve seen a rapid progression through the generations of DDR5 memory. Multiplexed Registered DIMMs (MRDIMMs) offer a new memory module architecture capable of extending the DDR5 roadmap and expanding the capabilities of server main memory. MRDIMM reuses the lion’s share of existing DDR5 infras... » read more

GDDR7 Memory Supercharges AI Inference

By Tim Messegee - 17 Oct, 2024 - Comments: 0

GDDR7 is the state-of-the-art graphics memory solution with a performance roadmap of up to 48 Gigatransfers per second (GT/s) and memory throughput of 192 GB/s per GDDR7 memory device. The next generation of GPUs and accelerators for AI inference will use GDDR7 memory to provide the memory bandwidth needed for these demanding workloads. AI is two applications: training and inference. With tr... » read more

Research Bits: Sept. 24

By Jesse Allen - 24 Sep, 2024 - Comments: 0

Modeling negative capacitance Researchers from Lawrence Berkeley National Laboratory developed an open-source 3D simulation framework capable of modeling the atomistic origins of negative capacitance in ferroelectric thin films at the device level. When a material has negative capacitance, it can store a greater amount of electrical charge at lower voltages. The team believes the FerroX fra... » read more

HBM4 Feeds Generative AI’s Hunger For More Memory Bandwidth

By Nidish Kamath - 12 Sep, 2024 - Comments: 0

Generative AI (Gen AI), built on the exponential growth of Large Language Models (LLMs) and their kin, is one of today’s biggest drivers of computing technology. Leading-edge LLMs now exceed a trillion parameters and offer multimodal capabilities so they can take a broad range of inputs, whether they’re in the form of text, speech, images, video, code, and more, and generate an equally broa... » read more

DDR5 12.8Gbps MRDIMM IP: Powering The Future Of AI, HPC, And Data Centers

By Kos Gitchev - 12 Sep, 2024 - Comments: 0

The demand for higher-performance computing is greater than ever. Cutting-edge applications in artificial intelligence (AI), big data analytics, and databases require high-speed memory systems to handle the ever-increasing volumes and complexities of data. Advancements in cloud computing and machine virtualization are stretching the limits of current capabilities. AI applications hosted in the ... » read more

GDDR7: The Ideal Memory Solution In AI Inference

By Frank Ferro - 29 Aug, 2024 - Comments: 1

The generative AI market is experiencing rapid growth, driven by the increasing parameter size of Large Language Models (LLMs). This growth is pushing the boundaries of performance requirements for training hardware within data centers. For an in-depth look at this, consider the insights provided in "HBM3E: All About Bandwidth". Once trained, these models are deployed across a diverse range of... » read more

Memory Implications Of Gen AI In Gaming

By Tim Messegee - 08 Aug, 2024 - Comments: 0

The global gaming market across hardware, software and services is on track to exceed annual revenues of $500B in 2025.1 That’s bigger by an order of magnitude than the combination of movies and music. On the cutting edge of that enormous market is open world gaming, where the driving goal is to give players the freedom to do anything they can imagine in a coherent and immersive environment. ... » read more

Are You Ready For HBM4? A Silicon Lifecycle Management (SLM) Perspective

By Faisal Goriawalla - 06 Aug, 2024 - Comments: 0

Many factors are driving system-on-chip (SoC) developers to adopt multi-die technology, in which multiple dies are stacked in a three-dimensional (3D) configuration. Multi-die systems may make power and thermal issues more complex, and they have required major innovations in electronic design automation (EDA) implementation and test tools. These challenges are more than offset by the advantages... » read more

← Older posts

Knowledge Centers
Entities, people and technologies explored

Shift Left Is The Tip Of The Iceberg

A transformative change is underway for semiconductor design and EDA. New languages, models, and abstractions will need to be created.

by Brian Bailey

Partitioning In The Chiplet Era

Understanding how chiplets interact under different workloads is critical to ensuring signal integrity and optimal performance in heterogeneous designs.

by Ann Mutschler

3.5D: The Great Compromise

Pros and cons of a middle-ground chiplet assembly that combines 2.5D and 3D-IC.

by Ed Sperling

AI’s Role In Chip Design Widens, Drawing In New Startups

Focus is on letting engineers do much more with the same or fewer resources — and less drudgery.

by Karen Heyman

What Comes After HBM For Chiplets

The standard for high-bandwidth memory limits design freedom at many levels, but that is required for interoperability. What freedoms can be taken from other functions to make chiplets possible?

by Brian Bailey

Memory Fundamentals For Engineers

eBook: Nearly everything you need to know about memory, including detailed explanations of the different types of memory; how and where these are used today; what's changing, which memories are successful and which ones might be in the future; and the limitations of each memory type.

by The SE Staff

Why Small Fab And Assembly Houses Are Thriving

Megafabs churning out the most advanced processors are not the only game in town.

by Bryon Moyer

New AI Processors Architectures Balance Speed With Efficiency

Hot Chips 24: Large language models ratchet up pressure for sustainable computing and heterogeneous integration; data management becomes key differentiator.