Interconnects Essential To Heterogeneous Integration

Designing and manufacturing interconnects is becoming more complex, and more critical to device reliability, as the chip industry shifts from monolithic planar dies to collections of chips and chiplets in a package.

What was once as simple as laying down a copper trace has evolved into tens of thousands of microbumps, hybrid bonds, through-silicon vias (TSVs), and even junctions for optical fiber. The main goal is still to get signals from point A to point B as quickly as possible with the least RC delay, using the lowest amount of power possible — all while ensuring those signals are intact and that they reach their destination. But making all of that work is a growing challenge.

“As data rates increase and we push the limits of what data can be pushed down a physical channel, parallel processing or nested parallel processing will be required to gain speed,” said Chris Mueth, senior manager of new markets and digital twin program manager at Keysight. “The implication here is more interconnects are required than ever before.”

This is evident with chiplets, where data needs to flow into and out of the chiplet to connect it to other components in a package. That approach is potentially more complicated, but there is significant payback in terms of power.

“Regular chips have big power-hungry drivers on their output pins that are strong enough to drive an electrical signal through the relatively large and long signal trace on a PCB,” said Marc Swinnen, director of product marketing for the semiconductor division of Ansys. “But chiplets don’t need those really big drivers because 2.5D interconnect is much smaller, and so you can save space and power with much smaller I/O drivers on each chip.”

Underlying much of this shift is the physics of packing more features into a fixed area. While digital logic will scale into the single angstrom range, shrinking wire diameters increases resistance and capacitance, while adding a bunch of new physical effects. Devices can run hotter, signals may run slower, and signal integrity becomes more difficult to maintain. Overcoming those issues requires new materials with higher electron mobility and wider pathways for critical data. It also requires a deep understanding of how a device will operate under different workloads, which can affect the overall layout of interconnects along the x, y, and z axes.

“You rip apart what used to be on a chip into something that is going broader into multiple chiplets,” said Frank Schirrmeister, vice president of solutions and business development at Arteris. “The way you communicate on chip needs to be extended to how you talk between the chiplets, but it has nothing to do with what substrate you use in between. The complexity has grown for the blocks on chip.”

The history behind today’s proliferation of interconnect schemes provides perspective on why the landscape seems so overgrown. As chips grew larger in the late ’90s, the industry became concerned about how to connect them, leading to virtual socket integration schemes and a wide variety of buses that were tailored to each situation. Because the number of blocks had become unmanageable, there were test buses, high-performance buses, peripheral buses, among others. Over time, the bus systems became too energy-hungry, which led to the development of protocols to reduce the overhead.

Arm began to straighten out the situation with the creation of the Advanced Microcontroller Bus Architecture (AMBA), an open-standard for the connection and management of blocks in SoCs. Over the past 30 years, AMBA has been revised and expanded with several secondary protocols. Recently, Arm announced the new CHI C2C specification, extending AMBA to chiplets.

Interconnect irony
There’s a certain irony to the abundance of interconnect PHY and protocols. “One of the early huge benefits of monolithic chips was that there were no interconnects,” said Swinnen. “Technically there were, but they were all made in a single process step. There is a rule that says the reliability of the system will go down as the number of interconnects in the system goes up. Despite this, there are many more connections now. Even a modest 2.5D design can easily have 500,000 bumps.”

Also, reliability complications may be inevitable, according to Andy Heinig, head of department for efficient electronics at Fraunhofer IIS’ Engineering of Adaptive Systems Division. “At a certain point the assembly technology is chaining, for example, from solder balls to copper pillars, or later to hybrid bonds. With new assembly technologies, we may see some new reliability topics. Here, the chiplet interfaces can introduce new challenges because the number of interconnects in a certain area is quite high.”

Nevertheless, well-designed interconnects are essential to realize the benefits of heterogeneous integration and chiplets. As many more signals and growing volumes of data have to shuttle through ever-more complicated layouts, interconnects can become bottlenecks due to the increases in latency caused by so many connections.

“You’re only as fast as the slowest interconnect in your design,” noted Mick Posner, vice president of product management for high-performance computing IP solutions at Synopsys. “As a result, everything has to scale at the same time. There’s an extra level that’s multi-die, where there is direct connectivity, which could have been chip-to-chip before. There’s automatic scaling when it goes to die-to-die, but even then there are package-level considerations.”

Fig.1: Are interconnects now bottlenecks? Source: Synopsys

Interconnect taxonomy and hierarchy
Interconnects themselves need to interconnect. In multi-layered ICs, thin, short local interconnects provide on-chip connections, while thicker, longer global interconnects travel between different blocks. Through silicon vias (TSVs) allow signals and power to be transmitted from one layer to the next, as described in detail by Larry Zhao, technical director at Lam Research.

The key difference between 2.5D (and in the future, 3D-IC) chiplet interconnects, and traditional PCB interconnects, is that 2.5D has much thinner, higher-density interconnects, which often are shorter, as well. New features like TSVs, micro-bumps, and hybrid bonding also complicate the interconnect picture, especially for 3D integration.

“On the upside, this means that communication between 2.5D chiplets is faster, higher bandwidth, and lower power than is possible with PCBs,” said Swinnen. “The downside is that it is more expensive than PCB technology. And many of the high-speed signals need to be designed with full electromagnetic coupling analysis, which is more complex than the simpler RC modeling you can get away with if you stay on-chip.”

However, as interconnects crowd ICs, issues like IR drop and RC delay begin to degrade performance. In response, the industry plans to deliver power through the backside of a chip, reducing the routing congestion in the upper metal layers of a device. That helps maintain signal integrity throughout the device, while also ensuring transistors receive enough power, but it adds a whole new level of complexity that has not yet been fully addressed for high-volume manufacturing.

Fig. 2: A working taxonomy of interconnects. Source: Intel

The choice of interconnect solutions is further complicated by what has been dubbed the “protocol zoo,” as standards shake out and ever-more detailed variants emerge, such as the various flavors of the coherent hub interface (CHI), which provides for communication between defined nodes.

“If you look at interconnects within an SoC, you immediately think of things like the AMBA bus,” says Synopsys’ Posner. “There’s been an evolution of that with streaming interfaces, extensions like CHI, and within that an expansion to more network on chips.”

Arteris has focused on expanding out for scalability of heterogenous, block-to-block topologies, as well as mesh topologies in which an SoC is split over multiple dies. “This is a process complicated by the protocol zoo, which is further complicated by conflicting adoption of versions,” said Schirrmeister. “Most companies working with RISC have gone to CHI, so the question is in the fine print: ‘Which version are they using?’ For example, the latest Arm cores have CHI-e interfaces, and the older Arm cores had CHI-b interfaces. You go through versioning, and you have different capabilities in different versions.”

This means communication and compatibility are critical. “You have to speak the same language on both sides,” said Schirrmeister. “You need to control who has the latest memory data and other elements between the different chiplets. For example, if you have a coherent protocol, you’d have an NoC on one side, which might speak AXI, one of the AMBA protocols. Now it’s being packed into 256-, 512-byte bitstreams, so these are serial bitstreams. On the other side, you need to unpack it again and make it AXI again.”

Simplifying interconnect protocol options
The proliferation of protocols is not likely to be pruned anytime soon, nor should it be, said Debendra Das Sharma, Intel senior fellow and co-general manager of memory and I/O technologies. “Some folks mistakenly have a view that there should be one interconnect that does it all. That is not correct. I believe the industry has rallied around the right set of interconnects — UCIe for on-package, PCIe and CXL for off-package, as well as rack/pod level, and Ethernet for networking.”

Because of that, it’s important that all of these interconnects can communicate with each other, and interoperability remains a necessary goal among designers. “To address these challenges of multiple interconnects, the industry really needs an interoperable standard for both scale up and scale out,” said Priyank Shukla, principal product manager for interface IP at Synopsys. “There’s a whole ecosystem that is trying to come together and match that performance. We see the UltraEthernet Consortium providing a back-end network, which can scale out, and AMD has open fabric and CXL technologies that can provide cache coherency. For die-to-die splitting, UCIe is the way to go. These interoperable open standards are providing the innovation to solve the interoperability problem that the industry is facing.”

Chiplets
Although there are different interconnects for different implementations, there is a clear trend toward standardization when it comes to chiplet interconnects. “Even users that own both ends of the connection are gravitating toward standards, as they want to benefit from collective work done by large standards organizations such as UCIe,” said Mayank Bhatnagar, product marketing director in the Silicon Solutions Group at Cadence. “We will never have enough engineers to design every interconnect possible, and leaning on a standard allows users to learn from the collective work of others in the field.”

At the same time, the tight supply chain of advanced packaging is causing more users to consider organic packages. “Also known as standard packages, organic packages allow shorter turnaround times, and the supported bandwidth density meets the needs of many customers who initially assume that advanced packaging will be required for their designs,” Bhatnagar said.

Still, there remains a crucial, unresolved issue as the industry moves towards chiplets. “One very important challenge with chiplet interconnects comes from the fact that nobody can test the interfaces later by probing with a needle or probe card, as it was usually done before,” Fraunhofer’s Heinig noted. “Such tests are necessary if the bring-up wasn’t successful or if some errors pop up during operations. Here, we need new solutions like on-chip monitoring and test.”

Coping with new complexities
The need for new solutions is an essential aspect of the increasing complexity that has come about with newer 2.5/3D package designs, with product development now multi-disciplinary, bringing in different specializations and different analysis tools.

“High-speed digital, RF, photonics, power electronics, ASIC design, thermal, mechanical, etc., all have to be married together cohesively and combined,” said Keysight’s Mueth. “This is one dimension of complexity, and these disciplines are often interdependent, further complicating the design process. Requirements, processes, and data have to be managed across the engineering lifecycle of design, test, and manufacturing, which adds more dimensions of complexity to the product development effort. Lastly, chiplets must perform in higher-level hierarchical systems, so there is an element of top-down design, bottom-up verification that must be considered.”

Fig. 3: The current complexity in the industry. Source: Keysight

Further Reading
UCIe-3D: SiP Architectures With Advanced 3D Packaging With Shrinking Bump Pitches (Intel)
A technical paper titled “High-performance, power-efficient three-dimensional system-in-package designs with universal chiplet interconnect express” was published by researchers at Intel.
Many More Hurdles In Heterogeneous Integration
More resources will be needed for IC-to-package design, process extendibility, and improved reliability.
AI Drives Need For Optical Interconnects In Data Centers
Old protocols are evolving as new ideas emerge and volume of data increases.