Heterogenous Integration Creating New IP Opportunities

The design IP market has long been known for constant change and evolution, but the industry trend toward heterogenous integration and chiplets is creating some new challenges and opportunities. Companies wanting to stake out a claim in this area have to be nimble, because there will be many potential standards introduced, and they are likely to change quickly as the industry explores what is required for various forms of integration.

There are two major categories of market, each of which has very different requirements. The first is the dissaggregation of the SoC, primarily because it has exceeded the reticle limit or there is a desire to take advantage of multiple fabrication technologies. The second is the potential to be able to integrate chiplets that are purchased in a general marketplace and integrated inside a package, rather than integrating those functions on a board.

SoC disaggregation
While Moore’s Law continues in a technical sense, the economic aspects of it have started to tail off. That is leading the industry to explore other possibilities, including spreading functionality across multiple dies and then integrating those on-package.

This may lead to more scalable architectures for AI accelerators, where processing arrays are no longer limited to the size of the reticle. “Why would you split up an SoC?” asks Mick Posner, director of product marketing at Synopsys. “It could be that you’ve just met the theoretical limit of how big the die can become. It could be that you want to scale a processor unit, an AI unit. You may want to use more mature technologies for part of the design and not the other. All of these possibilities are generating a whole new set of IP opportunities. Fundamentally, there’s an extra interface in there just from connecting them.”

Those interfaces will impact power and performance. “Heterogeneous integration allows companies to integrate multiple reticle size devices, but you’re definitely going to get your highest performance and lowest power if you can do it within a single chip,” says Tony Mastroianni, advanced packaging solutions director at Siemens EDA. “But you can only go so far. Once you hit the reticle size, you hit that. If you look at an AI processor, those typically scale by having multiple chips that get connected on the board. Those board connections require a long reach SerDes. One interesting architecture is to have tiles. They may be reticle size, and you can build those arrays not on the board, but on a large interposer — organic or silicon. Some of the organic interposer technologies are allowing much higher levels of integration.”

While the tiles may be mounted on an interposer today, they could be 3D tiles in the future. “A broad heterogeneous integration ecosystem will bring Moore’s law back on track while increasing flexibility and yield,” says Benjamin Prautsch, group manager for advanced mixed-signal automation at Fraunhofer IIS’ Engineering of Adaptive Systems Division. “In this sense, IP optimization and customization do not necessarily contradict each other. But the focus will shift more to the package design level, as well as to the standardization of interfaces. The latter will likely require flexibility within well-defined limits, and this will drive new EDA approaches, especially programmatic generator approaches, in order to rapidly develop interface IP.”

While some standards are emerging for interfaces, it has become clear that more are needed. “A lot of the chiplet standards, communication protocols and the IP around that, such as UCIe, are meant for homogenous, high-bandwidth speeds and feeds for communication from die to die (D2D),” says Aakash Jani, who runs technical marketing and brand growth at Movellus. “Over time, we are going to start to see more heterogeneous designs. Not all cores are going to need that high-bandwidth communication from die to die. Some may only need lower bandwidth. We also have to be aware of the overhead in those communication protocols, because one of the biggest contributors to area and power overhead is the clock forwarding protocol. As IP companies start to go down this road, we need to start removing those barriers and make D2D communication more power-efficient, and more area-efficient. Then we can start to support these low-bandwidth, low-power, heterogeneous chiplet communication protocols.”

While the number of ways that components have been connected at the board level is limited, those restrictions already have been dismissed in the SoC world. “On any given SoC, even before heterogeneous computing with multiple dies, even for a normal monolithic chip, there are many instances of interconnect that a customer will typically do,” says Michal Siwinski, CMO for Arteris IP. “The old notion used to be 1 chip, 1 interconnect. That’s no longer the case. The average, even before partitioning into multiple chiplets, is somewhere between 5 to 7 interconnects for an SoC. Some chips might have 20 or 30 interconnects. So there’s a lot of connectivity on any chip. Heterogeneous compute with chiplets, and D2D connectivity, basically adds an extra layer of connectivity.”

Chiplet integration
A chiplet is a pre-designed, pre-manufactured bare die that can be integrated into a package. “Chiplets have been around for many years across many different applications, but we are at an inflection point,” says Jeff Defilippi, senior director of product management for Arm’s Infrastructure Line of Business. “Chiplets can increase performance by expanding beyond the reticle limit while still providing the ability to manage silicon cost. The slowing of Moore’s Law has been discussed in the industry for some time. While advanced nodes (5nm and below) offer a benefit for logic, the scaling of I/O and memory components of a system-on-a-chip (SoC) has slowed significantly, translating to higher cost for less benefit. There is continual investment across the industry in innovation and standardization, which will help increase performance, lower costs, and expand adoption. At Arm, we expect to see a range of custom and standard implementations, and we look forward to what new performance points and unique SoCs are created with chiplet technology.”

This is where the disagreement starts. Can an SoC afford the luxury of a fixed function part? “When you have prefabricated chiplets, you get locked into the design specifications for these and flexibility diminishes,” says Movellus’ Jani. “You are also forced into heavy-duty interfaces. Companies that are vertically integrated may find it a lot easier, more power-efficient, and more area-efficient to go with a simple D2D communication protocol that removes a lot of that overhead. The whole idea of UCIe was to create this standardization for the companies that are going to source these prefabricated parts and then send it to a packaging house to put it together.”

But they do provide other forms of flexibility and efficiency. “As the chiplet model become widely adopted, and more affordable, it will reinvigorate the IP business,” says Ashraf Takla, founder and CEO for Mixel. “Chiplets give us more freedom to choose the best process for our IP instead of having to port our IP to a particular process technology that is best suited for our customers ICs. If a single chiplet standard becomes the clear winner, some IPs will be ripe for productization as a chiplets. Among those are bridge chiplets.”

It is not known where that balance point will be yet. “The dream of having a marketplace of chiplets, where customers would come in and find a piece of silicon ready to go is attractive,” says Synopsys’ Posner. “It has been silicon-validated. It’s a known good die. And then, all that is required is the packaging of different blocks. The Achilles heel is the chiplet is silicon, it’s set, its function is defined, and that does not take away from needing it to be optimized. That dream of having a marketplace of available IP will struggle just because there is not a one-size-fits-all.”

There are some areas where those advantages are fairly clear. “There is the possibility of incorporating what have, to this point, been impractical functions to include in a single package,” says Scott Durrant, DesignWare IP solutions marketing manager at Synopsys. “Things like logic and memory on one die, or analog or photonic circuitry on the same die as an electrical device — those were impractical. With the chiplet concept, it becomes feasible to put those things in a single package, and it will be interesting to see where the industry takes that opportunity. We will be able to put in a single package some interesting solutions that we’ve not been able to deliver as concisely or compactly in the past.”

For companies where time to market is key, or where they need to rapidly construct chips with defined functions, chiplets could be exactly what they need. “The idea of bringing in blocks that are pre-verified can really help speed up the design,” says Simon Davidmann, founder and CEO for Imperas Software. “Some smart people could build really good bits of silicon and just license them to you, and they will just work when you just glue them together. In that way I can see the benefits, because one of the great benefits of an IP business is that it’s verified and it works. All you have to do is integration test.”

For some blocks that are delivered as hard IP blocks today, they may see little difference. “I do see a broad base that would be very interested in building systems in packages with different components available,” says Siemens’ Mastroianni. “SerDes is a good example. There may be a need for a very high bandwidth, 128 gig SerDes, but that’s an expensive piece of IP, and it’s going to dictate a very expensive technology node. If there is a need for high-speed bandwidth, and if chiplets were available, that would open up the market for this methodology to many customers.”

The industry is certainly a long way from being fully standardized today. “There is chiplet authentication, and potentially data path encryption that has to be considered,” says Posner. “A chiplet must have hierarchical testability, known good die test and repair, lifecycle management for PVT sensors and the infrastructure around that. And then there is not one type of packaging technology. There are advanced interposers, and organic substrates. What you need is a customizable solution for chiplets.”

Mastroianni suggests one possibility. “Maybe they go part way. Rather than offering silicon, they have a GDS that’s taped out and ready to go, and they’ll just license that GDS. The integrator could then do the manufacturing. That is a decision they’re going to need to make, and it will probably be volume-driven. If there is a high-volume part, they may be more attracted to the per-piece model, as opposed to an upfront license fee, or there may be a run rate on royalties. Those business models really are going to be new, and they will have to think through that.”

Models
However functionality is packaged and shipped, either from a third-party supplier or reuse within a company, a new set of models will be required to encapsulate and abstract certain aspects of the chiplet. “Customers are seeking better IP packaging and integration tools,” says Anupam Bakshi, founder and CEO for Agnisys. “Unfortunately, many commercial IP vendors are not providing sufficiently detailed models. We are working with both internal and external IP teams to develop better designs and packaging models.”

There are standards organization working on this. “Chiplet Design Exchange (CDX) is a working group under ODSA, and the charter of that group is to propose a set of standardized models to support an ecosystem, as well as workflows (figure 1),” says Mastroianni. “It defines the models that would be necessary to be delivered with those chiplets, in order to support the system integrators. The feeling is that it makes sense, but wow, that’s a lot of extra work. In addition to getting into the silicon business, you have to create these models. That needs to be factored into the ROI to determine whether it’s really worthwhile. But regardless of whether it’s an off-the-shelf chiplet, if you’re doing heterogeneous integration, you still need all those views.”

Fig. 1: Chiplet value chain. Source: ODSA

UCIe is a newly announced interconnect standard for chiplets. “If you look at what UCI did in their 1.0 spec, it is the most comprehensive spec on the market,” says Posner. “It covers the full data rates required by most of the designs. It offers multiple protocol levels for latency-optimized raw streaming, but also the addition of higher-level PCIe or CXL, or actually any other protocols sitting on top of the raw interface. There are topics that are just marked for UCIe 2.0. For example, UCI is targeted to 2.5D interposer and organic substrates. 3D is not included. Security capabilities for authenticating a die is seen as UCI 2.0. We can expect to see rapid evolution of that spec.”

There are many other standards for D2D interconnect. “D2D interfaces are largely going to be these parallel interfaces and AIB, BoW and UCIe,” says Mastroianni. “There are several protocols, HBI is another one that are available. It would be nice if we could converge on one, but there may be a need for less sophisticated and cheaper protocols for those applications that are really not pushing the performance envelope. You could use a short reach XSR or USR (extra short reach or ultra short reach) SerDes that is implementable in many process technologies. An XSR PHY is much more affordable, and you’re not dictating that three or five nanometer process be used for the custom design that you’re doing.”

Conclusion
Time will tell which standards gain traction.

“Maybe it’s UCIe, maybe CXL, maybe something else will come up,” says Arteris’ Siwinski. “But the challenge, and the opportunity, is to figure out the right set of standards for how people are going to be able to start stitching the dies together. We need to end up with a heterogeneous system that acts almost homogeneously. That is the challenge ahead of us.”