Chiplets For The Masses

Chiplets are a compelling technology, but so far they are available only to a select few players in the industry. That’s changing, and the industry has taken little steps to get there, but timing for when you will be able to buy a chiplet to integrate into your system remains uncertain.

While new fabrication nodes continue to be developed, scaling is coming to an end, be it for physical or economic reasons. Many industry segments are demanding higher rates of transistor count increases than can be supplied by these new fabrication nodes. You cannot grow dies without compromising yield, and the cost of developing a chip at 3nm is a luxury few can afford. For industry segments that do not have quantities in the mega-millions, this leaves them in a bind, and chiplets provide a logical solution.

The notion of integrating chips inside a package is not new. “People want to simplify their design, or make it smaller, or use less power, than a PCB,” says John Park, product management group director for IC packaging and cross-platform solutions at Cadence. “They take the die out of the individual packages and put them on a single substrate — usually laminate, sometimes ceramic — and they work at that bare-die level to build a smaller, lower-powered PCB. We call that a multi-chip module (MCM), or system in package (SiP), and we have been in this market since the late ’80s.”

There are several terms that are used within the industry, often in ways that confuse the issues. “A SiP can be defined simply as two or more ASIC components integrated into a single package,” says Tony Mastroianni, advanced packaging solutions director for Siemens EDA. “There are many approaches to implement a SiP, including MCM, 2.5D and 3D packaging technologies. An MCM approach integrates multiple standard ASIC components mounted and interconnected on the package substrate. A 2.5D approach combines ASIC components mounted on a silicon or organic interposer, and includes die-to-die connections between two or more die through the interposer. A 3D approach allows ASIC components to be stacked and interconnected in the Z dimension.”

So is this a departure from Moore’s Law, or an extension of it? “We are following Gordon Moore’s advice, even today,” says Jose Alvarez, senior director in the CTO Office for the Programmable Solutions Group at Intel. “In 1965, he wrote a very short paper, four pages, on what has become known as Moore’s Law. On the third page he said, ‘It may prove to be more economical to build large systems out of smaller functions which are separately packaged and interconnected.’ And lo and behold, here’s where we are with the advanced packaging technology that we have today. And so, in a way, it’s just a continuation of what Gordon asked us to do.”

Fig. 1: Migration from MCM/SiP to Chiplets. Source: Cadence

What’s different with chiplets is they are purpose-built for being integrated within a package. DARPA kicked this off with their CHIPS program because the defense industrial base has lower volumes and can’t recoup the NRE of designing at 5 nanometers. Their notion of a chiplet is a physically realized block of IP with a wrapper around it.

But that is not where we are today. “DARPA’s vision is the right thing, and it is a matter of getting the education level set with design teams globally,” says Rob Mains, executive director for the CHIPS Alliance. “They need to understand the benefits and the industry needs to provide a level of assurance that it will yield effective results.”

Marc Swinnen, director of product marketing at Ansys, agrees. “It’s a sound technical idea, and there are organizations trying to make it happen. Groups like ODSA have various subcommittees working on getting chiplets to the point of sufficient standardization so the commercial market can effectively engage.”

The key is standards. “It’s a very evolving ecosystem,” says Manmeet Walia, senior product manager for high-speed SerDes at Synopsys. “It is a fragmented ecosystem. The whole concept was coined by the economics of the problem, mostly by DARPA, but that’s not where we initially saw the market develop. It was the physics of the problem, where dies were getting really large. You needed to scale the compute power, and in order to do that you needed to split the dies.”

The market segment driver is anything involving compute. “The key driver has really been high performance computing,” says Kenneth Larsen, director of product marketing for Synopsys. “This is where chiplet-based design seems to be growing, and it has the opportunity to become the next new abstraction. Today they are not based on the standards.”

You only have to look at pictures of chips to see this is being used successfully. “I look at the pictures of Intel’s new chips, and it turns out there are eight compute tiles that could be called chiplets, put together with some strips in the middle that contain cache and interconnect tiles,” says Michael Frank, fellow and system architect at Arteris IP. “And it is all sitting on a silicon substrate. There are clearly places where it is worth the money, and worth the efforts. But this paradigm has to be built on standards. It needs to cover the electrical properties, communications, physical attributes, etc. You cannot build different chiplets for every company. No matter how you look at it, it is still a chip and you have to go through all the steps you normally would for a tape-out.”

If the wrinkles can be worked out, the technology is applicable to many other areas. “Some parts of some designs may be suitable for older nodes and some for newer nodes,” says Synopsys’ Larsen. “Some of the value of chiplets will come from being able to design your IP in the optimal technology. Or you can maintain fixed interfaces while improving the PPA, or the costs of the overall product by maintaining part of the design, while you are migrating another part onto a newer node so you can get increased compute density.”

With the growing ubiquity of devices being connected, 5G chiplets may be an enabler. “I do believe that this will create an opportunity area for smaller companies, particularly for an IoT type of device,” says CHIPS Alliance’s Mains. “If you are a startup, you can combine your novel technology with some type of 5G chiplet and put that into a package.”

Industry status
Where is the industry today? “In most instances, a single company owns both sides of the link,” says Synopsys’ Walia. “They do not care about an industry-wide standard. Nvidia has what they call NVLink, AMD has their Infinity fabric, Qualcomm has Qlink, Intel has AIB, etc. They all came up with their own proprietary links. Now, as the ecosystem is evolving, there is a need for standards — and a lot are in progress.”

Standards are not all that it takes. “The big gap is the commercialization of chiplets,” says Cadence’s Park. “We already have hard and soft IP, and this would be the third flavor — a chiplet, and you would be able to procure this physical device and put it on your interposer, or laminate, or stack it, or do whatever. The packaging technology is a little bit independent of this. Chiplet viability is more about logical partitioning. The missing piece is companies that provide IP. Will they switch into this business model where they actually build these things and store them in warehouses? The answer is probably no. The concept of a business model for who will provide the warehouse to store all these chiplets, who will fabricate them, who is going to distribute them, a cost model that makes sense, hasn’t yet been established.”

Perhaps that is jumping ahead too far. “As an IP vendor, we are set up for selling a separate PHY IP for the chiplet interface,” says Wendy Wu, director of product marketing for Cadence’s IP Group. “We can foresee a future where we would be selling a complete design of the chiplet. That could be a PCIe chiplet that has the PCIe SerDes on one side and has the die-to-die (D2D) PHY on the other side. There may be a controller on there. Today we have these IPs as separate products, but we have been looking into putting this together as a unified design for a chiplet. We are not in a position to manufacture this chiplet. I could see IC companies who are building standard products today, packaged products, potentially manufacturing chiplets and being a supplier of chiplets on the market. They need to see if the market becomes big enough.”

The challenges can be separated. “Design standardization challenges can be summarized as functionality, footprint, and signoff,” says Rob Aitken, fellow and director of technology for Arm’s Research and Development Group. According to Aitken, the breakdown is as follows:

• Functionality. What the chiplet does is clearly key, but the chiplet’s relationship to the overall system architecture is important,” said Aitken. “Are different chiplets drop-in replacements for another device (as they are in memory), or are they performing similar tasks but with different software interfaces, clock frequencies, power domains, heat dissipation, etc.? In either case, clear specifications, models, and verification are going to be critical to the successful development of both the chiplets and 3D assemblies that include them.
• Footprint. “HBM standards dictate a specific arrangement of pins and functionality. A standardized logic chiplet will need the same thing, with standardization defined from the physical layer (what voltages happen when) up through the protocol associated with the connection points. The challenges that face the hard IP model (aspect ratio, pin location, test etc.) have analogs in chiplets. Even though chiplets allow for connectivity across an area, ‘beachfront’ (bits per second per millimeter along the die edge) will remain important for interface performance, because of the way chiplet die are likely to be floor-planned. While there are protocols and pin standards that support 3D, there is not yet a full logic chip footprint standard.
• Signoff. While there has been, and continues to be, a lot of work to ease the complexities of adding chiplets to the tapeout process, there is not yet a generally agreed upon solution including for how best to divide functionality and responsibility of the yield of the final assembled object, sharing power, heat, etc. across chiplet boundaries between vendors.

The only way to work out some of these issues is by doing it, and finding out where the friction points are. “Chiplets are commercially viable today, even when coming from multiple silicon suppliers,” says Intel’s Alvarez. “The standardization of the AIB interfaces has been critical in enabling this emerging ecosystem — and it is emerging. It’s not completely developed, but it is emerging and moving in the right direction.”

Fig. 2: A diverse ecosystem of chiplets based on AIB. Source: Intel

Intel provides evidence (see figure 2) of an ecosystem that is forming to work out the issues. “The idea is really to have a much more agile and flexible way of building semiconductors today, which by the way is the reason that DARPA is interested in this,” adds Alvarez. “You see a number of chiplets that are already developed, are sampling, are in production, some of them we’ve already powered up, and others that are in progress. It comes from a number of technologies and foundries, and so that’s really the idea that we have for this ecosystem — to be technology and foundry agnostic.”

Chicken and egg problem
The development of a new ecosystem presents a chicken and egg problem. What comes first, a portfolio of IP that people can integrate into their design, or systems companies demanding the availability of chiplets? “It will be a slow evolution,” says Park. “You will see chiplets start to trickle out. It will be a slow progression, and with the end of Moore’s Law scaling being close, at what point do people totally abandon the concept of a monolithic SoC and go to these multi-chiplet designs?”

Perhaps an intermediate step is logical. “No-one knows for sure, but one credible scenario is that initial chiplet systems will be built with standard dies,” says Ansys’ Swinnen. “They won’t be strictly considered chiplets, but they will be built like we see chiplets — bare die connected directly through a close connection layer. If you have such a system, and it becomes popular enough, then you could see it re-engineered as a chiplet. They cut down on the I/O drivers and increase the bandwidth to local neighbors. It would be a hybrid system, so the other chips are still standard builds, but with at least one chiplet on it. And then this might get the ball rolling.”

For an ecosystem to develop, the market has to be big enough. “Something like HBM memory had a big enough market and the requirements are unified,” says Wu. “People are talking about full package optics. There may be an application for an optics chiplet. There is a standard for it — XSR for the interface — and there are enough people trying to define the optical interface. That’s a pretty bounded application with a big enough market. It could definitely evolve into an open-market business model.”

Conclusion
Through proprietary systems, the viability and value of chiplets has been demonstrated beyond a doubt. The next step is tricker, because there are both technical and business issues that need to be worked out. Initial steps make it clear that the industry, the U.S. government, and standards bodies are all up for the challenge because this will become the way in which Moore’s Law is extended into the future. In fact, the whole industry is relying on this, even though today it is providing a competitive advantage for a few.

Coming next month: The status of standards, tools and ecosystems for chiplets.

Related

Designing 2.5D Systems
Connecting dies using an interposer requires new and modified processes, as well as organizational changes.
Many Chiplet Challenges Ahead
Assembling systems from physical IP is gaining mindshare, but there are technical, business and logistical issues that need to be resolved before this will work.
Waiting For Chiplet Standards
An ecosystem is required to make chiplets a viable strategy for long-term success, and ecosystems are built around standards. Those standards are beginning to emerge today.