Securing Terabit Ethernet For AI: Where MACsec, IPsec, And UET TSS Each Fit (And Why You Need More Than One)

As AI and HPC systems scale, the network has become both a critical enabler of performance and a rapidly expanding attack surface. The shift from rack-scale compute to cluster- and data center-scale AI infrastructure means that data is no longer confined to a single chip, board, or even system. Instead, it moves continuously across hundreds, or thousands, of endpoints, often at aggregate bandwidths measured in terabits per second.

This transformation is forcing a fundamental rethink of how network security is implemented. It is no longer sufficient to rely on a single protocol or a single layer of protection. Instead, modern AI infrastructure requires a layered security architecture, where protocols such as MACsec, IPsec, and the emerging Ultra Ethernet Transport Security Sublayer (UET‑TSS), as defined by the Ultra Ethernet Consortium (UEC), each play a distinct and complementary role.

Understanding how these technologies fit together, and why none of them is sufficient on its own, is now essential for system architects designing next-generation data centers. Traditional data center networks were built around relatively stable trust boundaries. Traffic flowed between well-defined endpoints, often within a single administrative domain, and security controls could be applied at relatively clear demarcation points.

AI infrastructure breaks those assumptions.

Modern AI clusters span multiple racks, pods, and sometimes geographic regions. They support multi-tenant workloads, dynamically scheduled jobs, and large-scale distributed training and inference pipelines. Data movement is increasingly driven by RDMA-style transports that bypass host CPUs, moving data directly into accelerator or application memory.

These changes introduce new requirements. Security must scale with the network. It must operate at line rate, without adding latency or disrupting tightly synchronized workloads. And it must support fine-grained isolation, ensuring that different jobs, users, or tenants remain securely separated even when sharing the same physical fabric. No single security protocol was designed to address all these requirements simultaneously. That is why a layered approach is emerging.

MACsec remains a foundational building block for securing Ethernet-based networks. Designed to protect traffic at the link layer (Level 2), it provides confidentiality, integrity, and authentication for data moving between directly connected devices. This makes MACsec highly effective for hop-by-hop protection, particularly in environments where physical access to the network infrastructure may be a concern. As Ethernet speeds scale to 1.6T and beyond, MACsec implementations must operate at full line rate with minimal latency, ensuring that security does not become a performance bottleneck.

However, MACsec’s strengths are also its limitations. Because it operates at the link layer, it secures traffic only between adjacent nodes. Once traffic is decrypted at each hop, it becomes accessible within the next device in the path, meaning every device in the chain must be trusted to prevent data exposure. This model works well for securing individual links but does not provide end-to-end or workload-level isolation, which is increasingly required in AI environments.

IPsec addresses a different class of problems. Operating at the network layer (Level 3), it enables encryption and authentication of traffic across routing boundaries, making it well suited for inter-domain communication and secure overlays. In AI infrastructure, IPsec is often used to protect traffic between clusters, data centers, or control planes. It provides a flexible framework for building secure tunnels that can span large and heterogeneous networks.

But IPsec also faces challenges in AI environments. It was designed for general-purpose networking, not for the ultra-low-latency, high-throughput flows seen in modern AI workloads. As a result, even hardware-accelerated implementations must carefully balance throughput, latency, and power efficiency. More importantly, IPsec operates largely independently of application or job context. It can protect traffic between endpoints, but it does not inherently understand which workload that traffic belongs to, or how it should be isolated within a shared AI fabric.

The emergence of Ultra Ethernet reflects a broader industry recognition that AI and HPC workloads place fundamentally new demands on networking. As part of this effort, the Ultra Ethernet Consortium introduced a new transport protocol along with a dedicated security layer: the Transport Security Sublayer, or TSS.

Fig. 1: UET-TSS architecture, source: Ultra Ethernet Specification v1.0.2

Unlike traditional approaches, UET‑TSS is designed to align security with how data actually moves in AI systems. Instead of securing only links or network paths, it operates at the transport level, enabling protection that follows endpoints, workloads, and communication flows.

This shift is critical. AI clusters often host multiple workloads simultaneously, each with its own security requirements. By enabling the creation of distinct security domains tied to specific jobs or tenants, TSS supports a level of isolation that is difficult to achieve with MACsec or IPsec alone. Equally important, TSS is intended for hardware implementation close to the Ethernet interface, allowing it to scale to high data rates without compromising performance.

However, TSS is not a replacement for existing protocols. It is deliberately designed to complement them, addressing gaps that become visible as networks scale to AI workloads. The key insight for architects is that MACsec, IPsec, and UET‑TSS solve different problems at different layers of the network stack.

• MACsec provides strong, efficient protection for individual links, ensuring that data cannot be intercepted or tampered with as it moves between directly connected devices.
• IPsec extends that protection across broader network domains, enabling secure communication between systems that may not share the same physical or administrative boundaries.
• UET‑TSS adds a new dimension, enabling transport-aware security that aligns with the structure of AI workloads, providing isolation and protection at the level of jobs, tenants, and communication flows.

Together these protocols form a coherent and scalable security architecture. Rambus, with over three decades of state-of-the-art security expertise, offers state-of-the-art line rate solutions for UET-TSS, MACsec and IPsec. These are all part of the industry’s  leading portfolio of security IP solutions backed by world-class support from Rambus security experts.

Links:

• Rambus Secure Networking IP
• Webinar – UEC Security: Delivering High-Performance, Scalability and Secure Networking for AI and HPC