PCIe 7.0 Takes Center Stage At PCI-SIG DevCon ’24

From the simplest building blocks like GPIOs to the most advanced high-speed interfaces, IP subsystems are the lifeblood of the chipmaking ecosystem. A key enabler for IP has been the collaboration between industry and academia in the creation of standards and protocols for interfaces. PCI-SIG drives some of the key definitions and compliance specifications and ensures the interoperability of interface IP.

HPC/AI markets continue to demand high throughput, low latency, and power efficiency. This is fueling technology advancements, ensuring the sustainability of PCIe technology for generations to come. PCIe 7.0 specifications and beyond will enable the market to scale, and we look forward to helping our customers build cutting-edge SoCs using Cadence IP solutions.

Fig. 1: Evolution of PCIe data rates. (Source: PCI-SIG)

What’s new this year at PCI-SIG DevCon?

At PCI-SIG DevCon ’24, the PCIe 7.0 standard will take center stage, and Cadence is showing off a full suite of IP subsystem solutions for PCIe 7.0 this year.

At Cadence, we believe in building a full subsystem for our testchips with eight lanes of PHY along with a full 8-lane controller. Adding a controller to our testchip significantly increases the efficiency and granularity in characterization and stress testing and enables us to demonstrate interoperability with real-world systems. We are also able to test the entire protocol stack as an 8-lane solution that encompasses many of the applications our customers use in practice. This approach significantly reduces the risks in our customers’ SoC designs.

Fig. 2: Piper – Cadence PHY IP for PCIe 7.0.

Fig. 3: Industry’s first IP subsystem for PCIe 7.0.

Which market is this for?

At a time when accelerated computing has gone mainstream, PCIe links are going to take on a role of higher importance in systems. Direct GPU-to-GPU communication is crucial for scaling out complex computational tasks across multiple graphics processing units (GPUs) or accelerators within servers or computing pods. There is a growing recognition within the industry of a need for scalable, open architecture in high-performance computing. As AI and data-intensive applications evolve, the demand for such technologies will likely increase, positioning PCIe 7.0 as a critical component in the next generation of interface IP. Here’s a recent article describing a potential use case for PCIe 7.0.

Fig. 4: Example use case for PCIe 7.0.

Why are optical links important?

It takes multiple buildings of data centers to train AI/ML models today. These buildings are increasingly being distributed across geographies, requiring optical fiber networks that are great at handling the increased bandwidth over long distances. However, these optical modules soon hit a power wall where all the budgeted power is used to drive the signal from point A to point B, and there is not enough power left to run the actual CPUs and GPUs. Such scenarios create a need for non-retimed, linear topologies. Linear Pluggable Optics (LPO) links can significantly reduce module power consumption and latency when compared to traditional Digital Signal Processing (DSP) based retimed optical solutions, which is critical for accelerating AI performance. Swapping from DSP-based solutions to LPO results in significant cost savings that help drive down expenditure due to lower power and cooling requirements, but this requires a robust high-performance ASIC to drive the optics rather than retimers/DSP.

To showcase the robustness of Cadence IP, we have demonstrated that our subsystem testchip board for PCIe 7.0 can successfully transmit and receive 128GT/s signals through a non-retimed opto-electrical link configured in an external loopback mode with multiple orders of margin to spare.

Fig. 5: Example of ASIC driving linear optics.

Compliance is key

For PCIe 6.0, the official compliance program has not started yet; this is typical for the SIG where the official compliance follows a few years after the spec is ratified to give enough time for the ecosystem to have initial products ready, and for test and equipment vendors to get their hardware/software up and running. At this time, PCIe Gen6 implementations can only be officially certified up to PCIe 5.0 level (the highest official compliance test suite that the SIG supports). We have taken our PCIe 6.0 IP subsystem solution to the SIG for multiple process nodes, and they are all listed as compliant. Due to space limitations, not all combinations could be tested at the May workshop (e.g., N3 root port) – this will be tested in the next workshop.

Also, the SIG just held an “FYI” compliance event this week to bring together the ecosystem for confidential testing (no results were reported, and data cannot be shared outside without violating the PCI-SIG NDA). We participated in the event with multiple systems and can report that our systems have done quite well. The test ecosystem is not mature yet, and a few more FYI workshops will be conducted before the official compliance for 6.0 is launched. We have collaborated with all the key test vendors for electrical and protocol testing throughout the year. As early as the middle of last year, we were able to provide test cards to all these vendors to demo PCIe 6.0 capabilities in their booths at various events. Many of them recorded these videos, and they can be found online.

• Cadence Subsystem IP for PCIe 6.0: Protocol and Electrical Testing
• Cadence Subsystem IP for CXL Protocol Test Demo
• Cadence Subsystem IP for CXL2.0/3.0 Protocol Test Demo
• Cadence Subsystem IP for PCIe 6.0: Protocol Stack Demo

Summary

Cadence showcased PCIe 7.0-ready IP at PCI-SIG Developers Conference 2023 and offers complete solutions in advanced nodes for PCIe 7.0 that will be generally available early next year. With a full suite of solutions encompassing PHYs, Controllers, Software, and Verification IP, Cadence is proud to be a member of the PCI-SIG community and is heavily invested in PCIe.