Simultaneous Bi-Directional Signaling: A Breakthrough Alternative For Multi-Die Assemblies

In designing multi-die systems-in-package, with or without chiplets, it is easy to think of the interconnect between dies as simply analogous to the interconnect between functional blocks on a single die. But this analogy can lead architects and designers into a blind alley from which it becomes impossible to meet system performance and power requirements. The reason lies in fundamental differences between on-die and inter-die interconnect. One novel solution to this problem is a piece of networking technology not normally used in interconnect: simultaneous bi-directional (SBD) signaling.

Connections between functional blocks on a die are generally implemented as masses of unidirectional connections. This makes sense given the enormous interconnect density, microscopic distances, and, hence, huge bandwidth possible in the metal layers of an IC. But between dies on a multi-die module, whether built on a conventional organic substrate or using advanced packaging technology, the environment is entirely different.

Connections between dies face a series of severe limitations. The allowable bump pitch will place a limit on the number of connections that can be made per millimeter of die edge on a given die. The requirement to place bumps for high-bandwidth channels on directly facing die edges—to minimize wire length and avoid corners—exacerbates this limitation by restricting where you can place connections. Overall routing resources on the substrate may impose further limitations. In addition, the much greater distances and higher parasitic impedances of inter-die connections compared to on-die connections limit bandwidth and make wide, source-synchronous busses impractical for all but the slowest signals.

From parallel to serial

These limitations require designers to minimize the number of wires between dies. This is usually accomplished by packing multiple signals onto a single wire using high-speed serial interconnect: converting parallel data to a serial bit stream, transmitting the stream at high speed across a single wire, receiving it at the other end of the wire and converting it back from serial to parallel. Multiple lanes can be used to achieve the necessary bandwidth for a connection between dies. Widely discussed standards for inter-die connections, including UCIe and BoW, are serial busses of this sort.

Generally these connections are unidirectional. But other alternatives exist. A unidirectional connection has a transmitter at one end and a receiver at the other and, obviously, sends data in only one direction. But in some situations, it makes sense to design a connection that has both a transmitter and a receiver on each end, and to configure the link to operate unidirectionally in one direction or the other. This can be done at system initialization, a technique called configurable bi-directional signaling. It is also possible to switch directions during operation, forming a half-duplex, or dynamically configurable connection. These are used, for instance, in memory interfaces, where the same data lines are switched back and forth between read and write modes.

Fig. 1: Typically, on-chip interconnects are unidirectional (left), but bi-directional approaches offer performance benefits.

But in connections such as those between CPU clusters or between CPUs and accelerators, data moves in both directions at the same time when the system is in operation. Supporting this dataflow pattern with unidirectional signaling would require a dedicated channel running in each direction, with all the necessary die-edge area, bumps, and substrate traces. In these cases, a link that allowed SBD signaling would halve the number of required connections for a given bandwidth, or, alternatively, double the available bandwidth for a given length of die-edge.

Having it both ways

So why isn’t SBD signaling included in most emerging multi-die interconnect standards? After all, SBD is used in high-speed networking, but has not previously been employed for inter-die connections. The main reason is that SBD introduces two major challenges that must be addressed with additional per-pin electronics.

The first challenge is interference from the transmitter sitting right next to the receiver. If the receiver is to capture a signal sent from the far end of the connection, it must somehow ignore the more powerful signal being sent by the transmitter right next to it and connected to the same wire. This is accomplished by use of an auxiliary transmitter—a second transmitter circuit, driving a nearly identical transmitted signal into a summing junction called a hybrid. The hybrid in effect subtracts the auxiliary transmitter signal from the signal on the wire, leaving at the receiver input, ideally, just the signal coming from the far end of the connection.

Fig. 2: Simultaneous Bi-directional signaling employs an auxiliary transmitter and a summing junction (called a “hybrid”) to cancel the transmit signal and recover the received data properly.

In networking applications where distances are long and transit losses great, the transmitter signal will be much, much stronger than the far-end signal. Hence the transmitter and auxiliary transmitter signals must be very precisely matched in amplitude and phase and the cancellation circuitry must be complex and extremely linear in order for the far-end signal to get through with an acceptable signal-to-noise ratio. Fortunately, in inter-die applications, the attenuation from the interconnect is much less significant, and so it is possible to achieve a good signal-to-noise ratio with relatively simpler and lower power circuitry.

There is a second challenge as well. The near-end transmitter signal will reflect off of any point in the interconnect—bump, via, even an angle in the trace—where there is a sudden change in impedance. These reflections will travel back to the input to the hybrid, and so they must be cancelled as well. As in networking applications, this is done using a multi-tap digital finite-impulse response (FIR) filter. The filter taps and weights must be adjusted using a training algorithm. In networking, this is a significant technical challenge. For example, a 10GBASE-T Ethernet needs an FIR with ~40,000 cancellation taps, taking a big toll on power. Again, the scale of the problem for inter-die connections is much less severe, and hence the required circuitry less demanding, with only a handful FIR taps.

At Eliyan we have demonstrated that our compact, low-power SBD transceiver design can achieve excellent eye openings both for challenging interconnect on conventional standard substrates and for the extremely high data rates on advanced-packaging substrates.

Costs and benefits

An obvious question at this point is the cost of this additional functionality. There would be little point in using SBD links if the added die area, edge space, and power made SBD as costly as a second unidirectional link. But in fact, available area at the die edge is determined by the bump pitch, and the auxiliary transmitter, delay line, and hybrid circuit are sufficiently compact to fit into unused area mandated by the bump pitch, even for existing advanced packaging specs. So, there is little or no cost in die area to make a channel SBD instead of unidirectional. And the power consumed by the additional circuity, while non-zero, is significantly lower than alternative methods to get higher bandwidth per channel, such as using PAM4 modulation or doubling baud rate.

It might also be asked whether SBD, with its requirements for carefully matched transmitter and auxiliary transmitter circuits, for FIR filters, and for careful layout, imposes an inappropriate workload on a chip design team. It is for just this reason that Eliyan provides its SBD interconnect technology as hard IP, eliminating the need for users to deal with the challenges of the physical design. This IP, by the way, includes the embedded microcontroller used for training the transceiver, so the integration into customer ASICs is as straightforward as with unidirectional links.

Opening up a bottleneck

When a multi-die architectural design calls for simultaneous data movement between blocks on separate dies, a simultaneous bidirectional approach can be valuable. SBD signaling can either double the aggregate bandwidth of a channel for a given number of wires, or it can halve the number of wires—and the amount of precious die-edge real estate—for a given bandwidth. Depending upon the specific limitations of the multi-die module design, either gain may be invaluable. With silicon-proven hard IP across a range of process technologies, Eliyan has made SBD signaling a practical alternative for designers of these multi-die systems.