Scaling Server Memory Performance To Meet The Demands Of AI

AI, whether we’re talking about the number of parameters used in training or the size of large language models (LLMs), continues to grow at a breathtaking rate. For over a decade, we’ve witnessed a 10X per year scaling. It’s a growth rate that puts pressure on every aspect of the computing stack: processing, memory, networking, you name it.

The platform vendors are responding to the increasing demands of AI with the most ambitious server roadmap ever seen. For example, the just introduced 5^th Gen Intel Xeon Processor came just a year after its predecessor. The 4^th Gen Xeon used 4800 Megatransfers per second (MT/s) DDR5 memory, 5^th Gen pushes performance up another level with 5600 MT/s DDR5.

Memory is a key enabler of computing performance, so to support an accelerating server roadmap, greater bandwidth will be required in each new platform generation. DDR5 RDIMMs employ two independent 40-bit wide memory channels (32-bit data plus 8-bit ECC). Since bandwidth is the product of data rate and memory channel bit width, if we hold the memory channel width constant, we’ve got to increase the data rate to achieve higher bandwidth.

And DDR5 is designed to scale its data rate: 4800, 5600, 6400, 7200 to 8000 MT/s and above. In addition to scaling the speed of the DRAM devices, implementing higher data rate RDIMMs requires scaling the memory interface chipset, and in particular the Registering Clock Driver (RCD).

The RCD is the key control plane chip on an RDIMM, providing clocks and command/address (C/A) signals to the DRAMs. It’s like a conductor, keeping the symphony of memory operations in sync. Above and beyond that, the C/A signals from the RCD tell each DRAM the location and operation (read or write) for data.

Recently DDR5 RCD chips were announced that can hit the 4^th major speed grade of DDR5: 7200 MT/s. That represents a 50% increase in data rate and bandwidth over the current mainstream DDR5 solutions in the market running at 4800 MT/s.

Rambus was first to announce a 7200 MT/s DDR5 RCD. The Rambus DDR5 RCD delivers low latency and power, and offers optimized timing parameters for improved RDIMM margins. The RCD is the flagship of the Rambus DDR5 memory interface chipset, which also includes a Serial Presence Detect (SPD) Hub and Temperature Sensor, two more key components for server memory systems. The SPD Hub and Temperature Sensor improve DDR5 DIMM system and thermal management in order to achieve higher performance levels within the desired power envelope.

The demands on data center servers will continue their rapid rise, and memory is a critical enabler of greater server computing. As such, the leaders in the server memory ecosystem must continue to deliver high-performance chip solutions ahead of the market need.

References

• Rambus DDR5 Memory Interface Chipset
• Data Center Evolution: DDR5 DIMMs Advance Server Performance