Mini-Consortia Forming Around Chiplets

Mini-consortia for chiplets are sprouting up across the industry, driven by demands for increasing customization in tight market windows and fueled by combinations of hardened IP that have been proven in silicon.

These loosely aligned partnerships are working to develop LEGO-like integration models for highly specific applications and end markets. But they all are starting small, because it’s proving difficult to create a commercial marketplace for chiplets that can work across a wide variety of use cases. It’s one thing to connect chiplets using a standardized scheme, such as the Universal Chiplet Interconnect Express (UCIe), or using bridges developed by Intel or Samsung. It’s quite another to expect them to work in heterogeneous devices under different compute loads and operating conditions.

Fig. 1: UCIe open chiplet ecosystem. Source: Fraunhofer IIS EAS/UCIe/Chiplet Summit

Understanding how different chiplets may interact with each other, and how they will behave under different use cases, is difficult to predict. Even with the best simulation tools, there is insufficient data, and probably always will be for many applications. But unless these chiplets are fully characterized in the context of other components and different use cases, there can be ensuing issues involving thermal management, various types of noise, different stresses, and inconsistent aging, all of which can affect reliability in the field.

On the other hand, chiplets have been proven to work extremely well in controlled situations. AMD and ASE have been collaborating on chiplets and graphics subsystems for the better part of a decade. Marvell has used chiplets since 2016. And Intel Foundry Services is customizing systems for its data center customers based on chiplets.

Fig. 2: AMD’s EPYC architecture integrates different process technologies. Source: Synopsys/AMD/Chiplet Summit

Yet all of these companies are relying on internally sourced chiplets to create what are essentially disaggregated SoCs, where they have total control of the system. The challenge now is to begin developing chiplets and chiplet-friendly architectures that can be sold commercially for any vendors’ devices.

“In some ways, we are in a new era,” said Bill Chen, fellow and senior technical advisor at ASE Group. “Optimization is a choice, and it’s important for us to allow those choices. We also are in an era of opening up and sharing. With chiplets, SiP, and heterogeneous integration, we started with the most difficult area, which is high-performance computing. There are two reasons. One is this is the market sector that needs it the most. And second, this is where these products are being developed, and where the technology is there to address them. Then we can move on to communications and other markets, such as wearables and medical applications.”

That’s where commercial chiplets developed by different vendors will fit in. But as companies at the leading edge of this shift will attest, sourcing chiplets from different vendors adds a whole new set of challenges. And even if the idea makes sense in theory, it never has been realized.

Small consortia are the first signs of real progress. Foundries, packaging houses, and large chipmakers are working with partners with deep domain expertise using various arrangements to ensure various essential components work together and can be contextually characterized. There are a number of different approaches on the table, from a handful of partners to more broad-based infrastructure, such as connectivity fabrics or standards coupled with stringent design rules. But for everyone involved, it’s a cautious, learn-as-you-go process.

“Samsung Electronics is building its own ecosystem,” said Kevin Yee, director of IP and ecosystem marketing at Samsung Electronics. “Even in that, there’s so much learning about things we hadn’t thought about. From a technology perspective, chiplets are here and proven. But currently, they’re all vertically integrated. So chiplets as a solution exists and is working. The challenge now is chiplets as a market or as a business. That’s what everyone is working toward, and there’s probably going to be an interim step. Everyone is going to start building their own micro-ecosystem first to make sure it works. So one company will do an I/O die, one will do the interconnect, one will do the data die. You will determine how to architect it and how to build something that will work. But the Holy Grail is when you can choose between 6 different compute dies, or 10 different I/O dies, select your memory dies, and then put it all together. We have a long way to go before that happens.”

Likewise, Palo Alto Electron (PAe), which focuses on advanced packaging and chiplet design, is spearheading its own chiplet consortium that includes Promex Industries (system integration), Thrace Systems (power dissipation analysis), Palo Alto Electron (advanced packaging and chiplet design), iTest (reliability and failure analysis), Hyperion (system-level design, interconnects and advanced packaging), and Anemoi Software (thermal solvers).

“This is a minimum viable ecosystem, and all of these companies are in the United States,” said Jawad Nasrulla, CEO of PAe. “We said, ‘Okay, we’re going to find small businesses and innovative startups that are attacking specific technical problems.’ For example, Anemoi Software is focused on thermal modeling of chiplets. Power/thermal is a critical enabling technology for doing the design. Palo Alto Electron has experience building ICs, and in the last seven years we’ve been doing it with chiplets. Each of these companies is an expert in a certain area. Now we need to learn how to work with each other.”

Dick Otte, CEO of Promex Industries, agreed. “Each one of these pieces is a multi-pronged thing. Each one has a capability, but everybody’s got less than 100% of all possible capabilities. So the real question is going to be, when we have jobs rolling through here, how many of them will there be? And how frequently will we have to go find some support or added capability? So far, it hasn’t been an issue.”

Some of the chiplets being created are modules or mini-systems themselves, rather than individual IPs like a specific I/O, where at least some of the integration work already has been done. But as the market for commercial chiplet develops, there could be many more chiplets targeted at specific jobs or functions, rather than complete subsystems. This can allow customers to add programmability and more customizability into the chiplet architectures without having to change the overall architecture, which is similar to the path Intel and Marvell have taken.

“If you look at small, midsize and large cars, each of those needs a totally different amount of electronics,” said Andy Heinig, department head for efficient electronics in Fraunhofer IIS’ Engineering of Adaptive Systems Division. “The chiplets allow you to be more flexible. But in the future, you may be able to combine chiplets together to create a larger one. You can configure your electronics this way, almost like building blocks, to get exactly what you need for a car.”

That will take time to develop. For now, many chiplet makers are approaching this at a higher level.

“Chiplets are broken up into relatively large chunks,” said Promex’s Otte. “It’s like the chip designer has designed the individual die, and then somebody has to pull all this together and do a high-level design that integrates not just the chiplets, but whatever interconnect or substrate or interposer technology that you’re utilizing, as well. The third part is this assembly services, which is our role. If this proves to be a highly successful model, and we are able to perform with quality and good economics for the customer, then it has the potential to snowball and become a major activity, and we will get into more difficult issues where we have a lot of projects going on at different states. That implies different kinds of coordination. The issue now is finding customers. Later on, it will be the delivery of parts.”

Fig. 3: Chiplet.us Alliance. Source: Semiconductor Engineering/Chiplet Summit

Rethinking old problems
For years, the general consensus across the chip industry was that business relationships would be the big hurdle to a working chiplet marketplace. As these mini-consortia efforts have shown, however, there are plenty of technical challenges, as well. And while these consortia often have a chosen “general contractor” for overseeing all aspects of the design through manufacturing flow, there are a lot of steps involved already, with many more to come as chiplets become more targeted and narrower in functionality.

“The mindset has to change when you’re doing chiplets,” said Yee. “A lot of people are still thinking this is like building an SoC. You’re really building a full system now. How do I talk to it? How do I configure the compute die? What sideband signals do I have to have? Firmware needs to be taken into consideration. Are you set up to use that firmware and boot up their compute die? There are a lot of system-level discussions now involving issues people haven’t really thought about before.”

Even choosing the right package is a challenge. “You’ve got so many different flavors,” said Michael Posner, product line senior group director for IP at Synopsys. “You would think, ‘Oh, you should be able to put each one in a box and maybe come up with a single IP that works across all of these, but that’s not the case. You’ve got different bump pitches, different performance and power, different parasitics, and power integrity issues. So rather than having a single via per node, as we traditionally do for an IP, maybe to a north-south or east-west orientation, we end up with one for advanced, one for standard, and maybe one for RDL because of the changes in technology. The number of IPs we need to develop across this whole ecosystem is exploding, and there isn’t a clear leader yet.”

Something old, something new
Not all of this is new, of course. OSATs and foundries working with advanced packaging have solved at least some of the challenges, such as how to handle chiplets, how to ensure these are known good die, as well as a variety of interconnect schemes such as hybrid bonding or microbumping. And in 2.5D implementations, HBM has largely functioned as a chiplet that works with many different configurations.

“Samsung’s packaging technology provides a significant advantage in offering complete solutions to Samsung Foundry,” said Yee. “What the team has learned from its leadership in memory packaging can be applied to foundry. HBM is a great example of memory leadership with multi-die in a package enabling the chiplets for foundry. As we move to chiplets, you can’t separate process and design from packaging. They will go hand in hand. When people think about chiplets, they assume you will be able to run connections directly one to the other. In general, that will work. In reality, what about routing jogs or offsets? How much margin do you have? With our test vehicle, we are conducting tests to determine realistic routing paths to ensure high signal quality.”

There also are proven ways of connecting chiplets that work, such as UCIe, Bunch of Wires (BoW), silicon interposers, bridges, and even hybrid bonding. In the future, it’s likely that more than one of these approaches will be used in complex designs, opening the door for more innovations.

For example, Eliyan, a startup that develops chiplet interconnects, has focused on eliminating the interposer from UCIe-compliant designs by building a physical interconnect layer (PHY) on both sides of a chiplet. “That gets rid of any complexity for manufacturing, thermal management, and allows us to bank all the things we’ve learned from the old MCM (multi-chip module) days,” said Patrick Soheili, co-founder and head of business and corporate development at Eliyan. “We have plans to build a bunch of chiplet stuff, using our technology as a basis. So we would have our technology on one end, and some other stuff on the other side, and connect two, three, or four things together. Maybe they’re HBM devices, maybe they’re other I/O controllers.”

The number of possible schemes for connecting devices together is growing rapidly. Last fall, TSMC introduced its 3D Fabric Alliance to connect different layers and devices in a 3D package. “We have EDA, IP, and design services, and we’re also adding memory partners, OSATs that help us assemble these devices, and the substrates, which become extremely important in 3D, as well,” said Dan Kochpatcharin, director in TSMC’s Design Infrastructure Management Division. “These devices can be 10 centimeters high, and the substrates can be 20 or more layers. So we need to make sure we align their roadmaps with our roadmaps so we can interface with them, and maybe there will be different materials that work together. And then you have to think about testing the whole system, which is not easy. So we’re working with Advantest and Teradyne, and also the EDA vendors. And IP is important in tests because we need to design for reliability.”

Fig. 4: TSMC’s 3DFabric ecosystem model. Source: TSMC

And that’s just the beginning of some of the generalized integration schemes emerging. There will be many more before this marketplace gets sorted out, and there will be an increasing number of proof points about what works and what doesn’t, as well as some new issues that have never been considered. For example, uneven aging in chiplets can cause all sorts of reliability problems that have never been tackled before, particularly in markets where devices are expected to remain functional for years. So cost savings in one area may be offset by cost increases in another as the economics of chiplets and packaging evolve, and costs that customers pay today may become less attractive as the chiplet model evolves.

“We’re seeing a more frequent customer decision to accept costs for TIMs (thermal interface materials just to allow their device to work,” said Nathan Whitchurch, senior staff engineer at Amkor Technology. “A device that works is probably cheaper than a device that doesn’t work. And things that used to be exotic are becoming less so, like the sintered silver category where you end up with a very hard, high-thermal conductivity matrix of a silver alloy between lid and die. Another would be softer metal atoms, like indium-based materials. And there are other things like graphite pads, which have some engineering challenges that are too difficult to overcome.”

Conclusion
Chiplets are a logical next step as the cost of shrinking and cramming everything onto a single SoC becomes uneconomical for most chipmakers. That has much of the industry thinking about next steps, and being able to standardize at least some of the components in a package to create a customized solution is a logical way of achieving what is essentially mass customization.

If this approach is successful, it potentially can shift how devices go to market while allowing for much more customization at a significantly lower price point. So the big performance gains in new architectures would be available in more niche markets, but without the heavy penalty of developing an ASIC or SoC from scratch. And what works for 80% of the market may still have huge benefits for the other 20% if some custom chiplets can be added into the architecture. But there are a lot of details to iron out first, and the chip industry is in the process of figuring out those details. These mini-consortia are a first step toward figuring out where the problems are, what can be standardized, and what role domain expertise will play in this process.

Related Reading
The Race Toward Mixed-Foundry Chiplets
The challenges of assembling chiplets from different foundries are just beginning to emerge.
Who Benefits From Chiplets, And When
Challenges involving reliability, integration and chiplet availability will take time to iron out.