Safeguarding Data Over PCIe & CXL In Data Centers

As more devices enter the market and drive exponential growth of data in the cloud, cloud computing is going through a significant overhaul. The increasing presence of “hyperscale” cloud providers for big data and analytics, 5G for rapid IoT connectivity, and the wide use of AI for natural data processing and for extracting insights are compounding both the amount of connected data and the data vulnerability.

To keep up with the rapid data growth, designers are driving innovation in interface and storage technologies to support increased capacity and performance, as well as more acceleration and new compute architectures. High-speed interfaces like PCI Express (PCIe) 5.0/6.0 and Compute Express Link (CXL) 2.0 are proliferating:

• Faster data rates for cloud-based computing systems are setting the stage for PCIe 5.0 and PCIe 6.0, which are replacing PCIe 4.0 interfaces
• Storage/SSDs are moving to PCIe 5.0/6.0 interfaces
• Data centers that typically deal with many bandwidth-hungry devices and vast shared memory pools are moving to CXL 2.0 interfaces

How can system architects protect cloud data that contains confidential, sensitive, or critical information that can be corrupted, replaced, modified, or stolen by malicious actors? I/O interconnects need to implement security from the start of the design. With limited security, attackers might aim to profit from secrets learned, interfere with the operations of a targeted company, or obstruct a government agency. The types of hacks differ in nature and continue to evolve, like attacks from malicious peripherals delivered over PCIe links, or root access attacks to access memory of other processes to capture secrets and/or alter code execution.

In addition, industry is faced with increasing laws and regulations, such as:

• GDPR (Global Data Protection Regulation) in Europe that imposes steep fines on corporations if private user data is compromised
• Health Insurance Portability and Accountability Act (HIPAA) in the US that stipulates how Personally Identifiable Information (PII) maintained by the healthcare and healthcare insurance industries should be protected from fraud and theft
• Payment Card Industry Data Security Standard, and many others

As the attacks become more sophisticated, the security standards have to continuously adapt to better protect sensitive data and communications and ultimately protect our connected world. To this end, the PCI-SIG and CXL standards organizations added security requirements like Integrity and Data Encryption to PCIe 5.0 and CXL 2.0 specifications in late 2020. Security is expected to continue to be adopted for the next generation PCIe 6.0 and CXL 3.0 interconnects as well.

PCIe and CXL security system components

Security for PCI and CXL interfaces has two main components: 1) Authentication & Key Management, and 2) Integrity and Data Encryption (IDE), as depicted in Figure 1.

Authentication & key management
Authentication and key management include functions like authentication, attestation, measurement, identification, and key exchange, all running in a trusted execution environment / secure module.

The main reference standard for authentication and key management is the Security Protocol and Data Module (SPDM) that is managed by the Distributed Management Task Force (DMTF). SPDM defines messages, data objects and sequences for performing message exchanges between devices over various transport and physical media and enables efficient access to security capabilities and operations. The message exchanges’ description includes authentication of hardware and measurement of firmware identities.

The PCI-SIG introduced two Engineering Change Notices (ECNs) for authentication and key management:

• Component Measurement and Authentication (CMA) defines how SPDM is applied to PCIe/CXL systems
• Data Object Exchange (DOE) supports data object transport over different interconnects

Integrity and Data Encryption (IDE)
IDE provides confidentiality, integrity and replay protection for Transaction Layer Packets (TLPs) for PCIe and Flow Control UnITs (FLITs) for CXL, ensuring that data on the wire is secure from observation, tampering, deletion, insertion and replay of packets. IDE is based on the AES-GCM cryptographic algorithm and receives keys from the Authentication & Key Management security component.

• Reference standards
  • PCI-SIG: PCIe IDE ECN
  • CXL 2.0: IDE for CXL.cache/mem protocols. CXL.io protocol refers to PCIe IDE ECN.

Fig. 1: PCIe & CXL security system level view.

PCIe & CXL IDE IP solutions

When looking for PCIe and CXL solutions with security, the tradeoffs to consider are performance, latency, and area. All of this needs to be in compliance with the latest standards, of course, and backed by experts.

Things to look for include:

• Throughput full-duplex for receiver and transmitter directions
• Integration with flexible data bus widths and the same clock configurations as the controllers
• Encryption, decryption, and authentication for TLPs for PCIe and FLITs for CXL, based on the AES-GCM cryptographic algorithm with 256-bit key size
• Configurable widths for cipher and hash algorithms for area and latency optimized solutions
• Inflight key refresh for seamless changes of keys in the system
• Low-latency in-order bypass mode for non-protected traffic

Fig. 2: PCIe IDE Security Module block diagram & integration with PCIe Controller.

Figure 3 depicts a CXL 2.0 IDE security module with pre-verification.

Fig. 3: DesignWare CXL IDE Security Module block diagram & integration with DesignWare CXL Controller.

Conclusion

With the tremendous data growth in our connected world, security is essential to protect private and sensitive information in data as it transfers across systems, including over high-performance interconnects such as PCIe and CXL.

Synopsys recently announced the industry’s first security modules for protecting data in high-performance computing SoCs that use the PCIe 5.0 or CXL 2.0 protocols. The DesignWare IDE Security Module IP for PCIe 5.0 or CXL 2.0 are already being deployed with hyperscaler cloud providers. The robust IDE Security Modules are pre-validated with controller IP for PCIe or CXL, making it faster and easier for designers to protect against data tampering and physical attacks on links while complying with the latest versions of the interconnect protocols. Synopsys’ security IP solutions help prevent a wide range of evolving threats in connected devices such as theft, tampering, side channels attacks, malware and data breaches.