Expanding Server Memory Capabilities With Multiplexed Rank DIMM (MRDIMM) Technology

The scaling of computational power within a single, packaged semiconductor component continues to rise following a Moore’s law type curve enabling new and more capable applications including machine learning (ML), generative artificial intelligence (AI), and training and deployment of large language models (LLM). On-demand lifestyle applications like language translation, direction finding, and recommendation systems are delivered with increased sophistication and scale in highly efficient cloud computing environments where the cost per unit of compute continues to fall. Recent advances in socket-level server CPU designs have utilized multi-core processors and multi-chip substrates incorporating chiplets and optimized fabrics to achieve the expected rate of performance growth.

As per socket compute density increases, the amount of directly accessible, low-latency memory bandwidth and capacity to maintain a balanced system and to adequately feed data to the multiple cores without contention, needs to scale accordingly. Memory bandwidth typically is a function of the raw DRAM component bandwidth, the efficiency of the memory channel between the CPU and DRAM, and the number of channels per socket. These typical knobs have their challenges and are reaching limits. Since the number of channels (and thus the number of pins) between the CPU and DIMM module is a critical resource, increasing the bandwidth across this interface at a rate higher than the natural DRAM IO scaling could be a solution. JEDEC recently unveiled the development of a new DIMM technology called Multiplexed Rank Dual Inline Memory Modules (MRDIMM). DIMMs typically have multiple ranks of DRAM that allow for higher capacity per DIMM. The CPU today only addresses and communicates with one rank at a time. With the MRDIMM architecture, the CPU would be able to communicate to both ranks simultaneously and the bandwidth would be effectively doubled.

For this idea to achieve reality, the double width of access to the two ranks of DRAM would need to be multiplexed onto one set of channel wires. This would be accomplished by special logic chips on the DIMM that would multiplex and demultiplex the two independent data streams and communicate with the CPU at a rate up to twice the data rate of the native DRAMs. This type of function in the industry is often called a gear box function where 2N wires operating at bit rate B are consolidated into N wires operating at a bit rate of 2B. This gear box function and higher speed 2B rate signaling would be present in new logic component specifications that are built on top of existing DDR5 component definitions for the Registering Clock Driver (RCD) and Data Buffer (DB). These new components called the Multiplexed RCD (MRCD) and Multiplexed DB (MDB)3 provide buffering, re-timing, load reduction, and multiplexing of the command/address bus and data signals, respectively

Read more here.