A Sea Change In Signaling With PCIe 6.0

PCI Express (PCIe) is one of those standards from the PC world, like Ethernet, that has proliferated far beyond its original application space. Thanks to its utility and economies of scale, PCIe has found a place in applications in IoT, automotive, test and measurement, medical, and more. As it has scaled, PCIe has pushed NRZ signaling to higher and higher levels reaching 32 gigatransfers per second (GT/s) with PCIe 5.0. What has been true since the introduction of PCIe 3.0 in 2010 is that each new generation has doubled the signaling rate of its predecessor.

The latest generation PCIe 6.0 delivers another doubling, bringing signaling to 64 GT/s. But doing so required some of the most fundamental changes yet seen by the standard, including saying farewell to NRZ. To achieve 64 GT/s, PCIe 6.0 adopts PAM4 signaling. PAM4 signaling (“Pulse Amplitude Modulation with four levels”) combines two bits per clock cycle for four amplitude levels (00, 01, 10, 11) vs. PCIe 5.0 and earlier generations using NRZ modulation with one bit per clock cycle and two amplitude levels (0, 1).

There’s no free lunch, and the transition to PAM4 signal encoding introduces a significantly higher Bit Error Rate (BER) vs. NRZ. This prompted the adoption of a Forward Error Correction (FEC) mechanism to mitigate the higher error rate. PCIe 6.0 adopts an FEC that is sufficiently lightweight to have minimal impact on latency. It works in conjunction with strong CRC (Cyclic Redundancy Check) to minimize the Link Retry probability (5×10^-6). This new FEC feature targets an added latency of under 2ns.

PCIe 6.0 also introduces FLIT mode, where packets are organized in Flow Control Units of fixed sizes, as opposed to variable sizes in past PCIe generations. The principal reason for introducing FLIT mode was that error correction requires working with fixed size packets; however, FLIT mode also simplifies data management at the controller level and results in higher bandwidth efficiency, lower latency, and smaller controller footprint.

With fixed-size packets, the framing of packets at the Physical Layer is no longer needed, yielding a 4-byte savings for every packet. FLIT encoding also does away with the 128B/130B encoding and DLLP (Data Link Layer Packets) overhead from previous PCIe specifications. This results in a significantly higher TLP (Transaction Layer Packet) efficiency, especially for smaller packets.

A final innovation I’d like to highlight is the introduction of a new Low Power State or L0p mode. L0p enables traffic to run on a reduced number of lanes to save power. L0p maintains at least one active lane at all times to ensure uninterrupted traffic flow. The link always trains in the highest possible width and can modulate down (and back up again) as needed by the traffic. One of the upshots as a tradeoff between power savings and complexity is that PCIe 6.0 only supports x1, x2, x4, x8 and x16 links. Support for x3, x5, x12, etc. and the x32 width have been dropped.

In summary, PCIe is everywhere in modern computing architectures, and it’s expected that PCIe 6.0 will gain quick adoption in performance-critical applications in AI/ML, HPC, cloud computing, and networking. For designs for both the upcoming generation of PCIe as well as previous generations targeting the wide spectrum of data-centric applications, Rambus offers a broad family of PHYs, controllers and complete interface subsystems for easy integration into SoCs and ASICs.

Additional resources:

• Website: Rambus PCIe 6.0 Controller
• Solution Brief: Rambus PCIe 5.0 Interface Subsystem
• Website: Rambus PCIe 5.0 PHY
• Website: Rambus PCIe 5.0 Controller