Changes that could push this packaging approach into the mainstream, and the challenges ahead.
Several companies are implementing the chiplet model as a means to develop next-generation 3D-like chip designs, but this methodology still has a long way to go before it becomes mainstream for the rest of the industry.
It takes several pieces to bring up a 3D chip design using the chiplet model. A few large players have the pieces, though most are proprietary. Others are missing some key components, making it difficult to develop these designs. But several entities are moving to bring the pieces into the market, which could make chiplets more accessible for customers at some point in the future.
Today, companies, government agencies, and R&D organizations are throwing their weight behind chiplets, which are emerging as an alternative methodology to develop advanced system-level designs. With this methodology, a vendor may have a menu of modular dies, or chiplets, in a library. Chiplets can have different functions and process nodes. Customers can mix-and-match the chiplets, and then assemble them in an existing advanced package or a new architecture. The goal is to speed up time to market and reduce the cost.
This concept isn’t new. Over the years, AMD, DARPA, Intel, Marvell and others have developed chiplet-enabled designs. Today, Intel, AMD and others are developing the next wave of chiplet-based products. “You will see an increasing number of chiplet designs next year,” said Jan Vardaman, president of TechSearch International.
Chiplets may have some advantages over the traditional approach to advance a complex chip design. Traditionally, to advance a design, vendors would integrate several functions on a system-on-a-chip (SoC). Then, at each generation, they would cram more functions on the SoC. But this approach is becoming more difficult and expensive at each generation.
While some will continue to follow this path, others are looking at alternatives like advanced packaging. Another way to advance a chip design is to assemble complex dies in a package. Advanced packaging comes in different forms, and chiplets are one of many approaches. But chiplets promise to enable new architectures that mimic today’s SoCs at lower costs.
Still, this approach is in its infancy, and the technology has a long way to go before it becomes a mainstream offering for a broad array of customers. Today, only a few companies have the in-house capabilities to develop these products. Most don’t have the pieces in place or the expertise. That makes it difficult, if not prohibitive, to develop chiplet-like designs.
Basically, it takes several pieces to bring up chiplet-based products, including design capabilities, dies, die-to-die interconnects, and a manufacturing strategy. Many of these pieces are emerging in the open market, but they are scattered. Other pieces are in R&D.
But there are some new developments for the chiplet approach:
There are several other efforts, as well. Even so, vendors face various design, integration, manufacturing, and supply chain challenges with chiplets.
Why chiplets?
For years, IC vendors have relied on scaling to advance chip designs, packing more transistors onto an SoC at each new node. The most advanced chips today have billions of transistors. But chip scaling is becoming more difficult, and the price, performance, and power benefits of scaling are shrinking faster than the transistors.
Beyond 3nm, finFETs will run out of steam. Starting at the 3nm node in 2022, the industry is moving to a new transistor type called gate-all-around.
“Gate-all-around, or GAA transistors, are a modified transistor structure where the gate contacts the channel from all sides and enables continued scaling,” said Nerissa Draeger, director of university engagements at Lam Research. “Nanosheets may be simple in concept, but they present new challenges for manufacturing.”
Design costs are another issue. The average cost to design a 28nm chip is $40 million, said Handel Jones, CEO of IBS. In comparison, it costs $217 million to design a 7nm chip and $416 million for a 5nm device, Jones said. A 3nm design will run $590 million, he added.
For years, the industry has recognized these problematic trends. Advanced packaging based on heterogeneous integration is another way to achieve the benefits previously available only through scaling SoCs. The industry has developed various advanced package types for several applications, but generally this approach has been confined to high-end applications due to cost.
That’s beginning to change, though. “The industry is investing more and more on advanced packaging and has been diligently working to improve system-level interconnection density, reduce power consumption, achieve a smaller form factor, and lower cost by scaling the package-level pitch and integrating more functions into a single package,” said Xiao Liu, senior program manager at Brewer Science.
Advanced packaging solves several challenges in today’s systems. For example, in systems, data moves back and forth between a separate processor and the memory devices. But at times this exchange adds latency and increases energy consumption. One way to solve the problem is to bring the memory and processor closer together and integrate them in a package.
“There’s a need for higher memory bandwidth at lower power,” said Dave Hiner, senior director of advanced product development at Amkor, in a recent presentation. “This is where you see co-packaging of memory, either within the package or on chip.”
In another example, vendors tend to integrate all functions on a single chip. But at each generation, that approach is becoming more difficult and costly.
There is a solution. “One approach is breaking this SoC up into its functional blocks, and then re-packaging them or re-constituting those functional blocks into chiplets. And those chiplets are essentially placed in a package next to each other,” Hiner said.
The chiplets approach is appealing. “The trend we are seeing is that more and more customers want to figure out a way to integrate different pieces together. They want to mix-and-match different functional dies together,” said Kevin Zhang, senior vice president at TSMC.
Chiplets are more of a methodology than a type of packaging. Customers can leverage the chiplet model and integrate dies in existing advanced package types, such as fan-out and 2.5D. There also are options to stack logic on logic, or logic on memory, in a 3D-IC.
So what’s the best solution? “The question is what are the devices we are targeting,” said C.P. Hung, vice president of R&D at ASE, in a panel at the recent IEEE Electronic Components and Technology Conference (ECTC). “With multiple chips, you have to look at I/O density. We can handle that in flip-chip. If that’s not enough, we can consider fan-out. If we have multiple memories that need to be integrated, we may need to use 2.5D.”
Fig. 1: Different options for high-performance compute packaging, interposer-based 2.5D vs. Fan-Out Chip on Substrate (FOCoS). Source: ASE
Chiplet apps, challenges
Chiplet-based designs aren’t required for all products. In fact, it’s overkill for many applications. But for select applications, the chiplet approach provides flexibility, enabling a variety of designs. In one example, Intel is developing Ponte Vecchio, a GPU that incorporates 47 tiles or chiplets. Two are based on 10nm finFETs. In total, the design boasts 100 billion transistors.
In another example, AMD is developing 3D V-Cache, a cache chiplet stacked on a processor. Both devices are based on TSMC’s 7nm process.
Fig. 2: AMD’s 3D V-Cache stacks the cache on a processor. Source: AMD
A chiplet design even could be implemented using devices at mature nodes. “You could have technologies like RF mmWave and optical interconnects. They are going to use technologies that are different than FPGAs or CPUs. And you could have other technologies, including GaN,” said David Kehlet, a research scientist at Intel.
Developing a chiplet-based design isn’t simple. According to Cisco, here are the main challenges:
“Generally, for any technology to become more mainstream or mature, you need a significant driver,” said Jie Xue, vice president of technology and quality at Cisco, at ECTC. “A driver needs to come from someone with high volumes, so they drive the investments, drive the development, and drive the cost.”
At a starting point, designers will need to look at a multitude of issues. At ECTC, Bryan Black, a senior fellow at AMD, outlined the design considerations and challenges with chiplets:
Designers also should look at other issues upfront that deal with the manufacturing process. “For example, having the proper substrate and/or interposer for chiplet-based designs is critical,” said Rosie Medina, vice president of sales and marketing at QP Technologies. “Additionally, customers need to consider the time and cost to design and fab the substrates and/or interposers.”
In other words, along with the design considerations, it also makes sense to formulate a manufacturing strategy upfront. Ideally, the separate design and manufacturing teams should work hand-in-hand. A design should not only work in the field, but must be production-worthy.
Selecting a manufacturing partner is critical, and there are several options here: 1) manufacturing the package in-house; 2) work with a foundry; 3) go with an OSAT; and 4) work with a combination of vendors.
Each option is viable. Vendor selection depends on capabilities, manufacturing scale and cost. Customers tend to work with their vendors who they trust.
Finding dies, interconnects
Developing a design around chiplets is only half the battle. To bring a chiplet-based design into production, vendors require several pieces, such as intellectual-property (IP) cores, known-good die (KGD), and die-to-die interconnects.
A KGD is a bare die. In chiplets, the goal is to assemble good dies in the package. IP cores involve the building blocks to develop chips, such as I/O, processor cores and libraries.
Where can you find IP cores and dies for chiplets? There are several options, including developing your own technology, going to a foundry and/or OSAT, and contacting a fabless ASIC design house.
AMD, Intel and a few others have the resources to develop their own chiplets and IP. Developing in-house dies/IP takes time and money, but there some advantages. IC vendors have the critical data about in-house dies and how they interact with others.
“If every die in the package is custom-designed for a particular product, like we do at AMD, then certain things like power delivery, interconnect, clocking, cache hierarchy, and everything else are handled by a single design team at the same time, which makes it much easier to develop,” AMD’s Black said.
But even the larger companies can’t afford to develop all IP in-house. They may want to source third-party IP to save time and money.
That can be a major challenge. For example, a vendor may want to use a die from another company. But that company may not want to share the inner workings of a chip, which is essential to fully characterize it. And even if they are willing to share the data, the die will still need to undergo a verification and testing process.
“The question is, do we develop the dies internally or do we find them externally? This will dictate how the dies interact with the architecture, and how we put them together. This is going to impact how they interact at the physical level,” Black said. “Where we are going to be challenged in the five-year time frame is the heterogenous piece with respect to different organizations. How are we going to share dies from one company to another company to make more complex devices? If the dies are from several different sources, then what happens is we can end up with endless configurations of all sorts of challenges.”
Most companies don’t have the pieces in-house. The challenge is to locate the necessary pieces, which takes time and resources. So perhaps working with a foundry and/or OSAT make more sense.
Several foundries and OSATs are putting their chiplet strategies in place, but not all vendors are alike. TSMC, for one, provides a turnkey approach. The foundry giant has a large catalog of certified in-house and third-party IP cores. Customers have the option to utilize any of these IP blocks to develop traditional chips.
TSMC says many of the same IP cores and dies used for traditional chips can be leveraged to develop a chiplet-based design. It also has the manufacturing capabilities.
“The business model is not different than our wafer business,” TSMC’s Zhang said. “We work with customers to identify the right chiplets and integration schemes. When we stack different chips together, each chip comes from our customers. They are all customer-specific designed IP. They choose the fabric they want to integrate. We provide the solution to help customers to integrate different chiplets together with different advanced integration techniques.”
Other foundries may have similar or different strategies. The OSATs are also working on their chiplet strategies. For now, though, the strategy among most vendors resembles the current packaging flow. As before, foundries manufacture chips for customers. From there, they send the finished chips to the OSATs, which handle the packaging assembly requirements.
Some foundries provide various packaging components, such as interposers. They will even provide the TSV manufacturing process for customers. But the bulk of the packaging work is handled by OSATs.
Eventually, OSATs, foundries and others want to take chiplets to the next level. Many are working with the ODSA, an industry organization seeking to bring chiplets to the masses. ODSA is working on several technologies, including standard die-to-die interfaces, reference designs and workflows. All of this is leading toward the advent of the Chiplet Design Exchange (CDX), which is an open market to buy and sell certified chiplets from different vendors.
“We are working on a CDX whitepaper that will provide guidelines to the industry on building models for chiplets,” said Jawad Nasrullah from the ODSA. “The modeling uniformity is a key to develop a market for the trading of components.”
But an open exchange for chiplets won’t appear for at least two or three years. It will take time and resources to bring this into fruition.
Meanwhile, for chiplets, vendors require a die-to-die interconnect/interface technology, which joins one die to another in a package. To implement a die-to-die interconnect, vendors design a tiny IP block on each die. The block consists of a common physical interface with circuitry. That way, dies with common interfaces can be connected, enabling them to communicate with each other.
The first wave of chiplet-based designs incorporated die-to-die interconnects with proprietary interfaces for a company’s own devices. But to broaden the adoption of chiplets, the industry requires interconnects with open interfaces, enabling different dies to communicate with each other.
That’s a major stumbling block for chiplets. So far, Intel has developed one of the few open interfaces that’s available on the market. The technology, called the Advanced Interface Bus (AIB), is an interface scheme that transports data between chiplets.
Vendors want more than one die-to-die interconnect scheme. Other technologies are in R&D, but it’s unclear when they will be ready. These include:
Design and manufacturing issues
Ultimately, customers want to design and build products. For this, vendors must select a package type or architecture for a given design, as well as the appropriate chiplets and die-to-die interconnects. This isn’t a simple task. There are a multitude of different and confusing options here.
Once those decisions are made, it’s time for the design phase. Using EDA tools, vendors generally follow the traditional design steps, such as design entry, substrate/interposer routing and layout, and verification.
Some have the EDA design tools and expertise in-house. Others may have these tools, but lack the design expertise.
In response, ODSA is hammering out a set of design guidelines for developing chiplet-based products. Called the “CDX Work Flow Whitepaper and Design Guide,” the document describes various modeling techniques required for chiplets and how to implement them.
During the design phase, a vendor must model the behavioral properties of the desired chiplets. It’s also important to model the mechanical, power dissipation and thermal properties. And it’s imperative to understand the properties of the design before it moves into production. Otherwise, problems could surface.
Take die-to-die interconnects, for example. “In the context of high-speed, low-latency interconnects between dies or chiplets, there are multiple challenges,” said Michael Liu, director of technical marketing at JCET. “This includes, but is not limited to, power consumption while achieving ultra-high bandwidth.”
There are other design issues. And, of course, the design must be production-worthy. Each package type, such as 2.5D/3D, fan-out and others, has its own manufacturing flow.
Momentum is building for copper hybrid bonding, a manufacturing process that enables next-generation 2.5D packages, 3D DRAMs and 3D-ICs. It is also ideal for chiplets. Targeted for 10μm pitches and below, hybrid bonding connects dies in packages using tiny copper-to-copper connections. Still in R&D for packaging, hybrid bonding provides more interconnect density than the existing methods.
This isn’t a simple process. Hybrid bonding requires almost zero defects in the process. Product reliability is still a question.
At ECTC, Xperi presented the first results on the reliability and thermal performance of a 5-die stack module test vehicle with a 35μm pitch. “The enhanced reliability performance in hybrid bonded parts offers substantial advantages over the incumbent technology,” said Abul Nuruzzaman, vice president of product marketing at Xperi, a provider of IP to enable hybrid bonding for customers. “Hybrid bonded parts are well suited for high-temperature and/or corrosive environments such as automotive.”
Fig. 3: Xperi’s die-to-wafer hybrid bonding flow. Source: Xperi
Conclusion
Clearly, chiplets are complex with different pieces. Putting them together is the hard part.
At some point, the solutions may all come together. There is a multitude of applications that could use chiplets.
Related
Advanced Packaging’s Next Wave
A long list of options is propelling multi-chip packages to the forefront of design, while creating a dizzying number of options and tradeoffs.
Emerging Apps And Challenges For Packaging
Heterogeneous integration is reshaping some markets, but not all applications require it.
Bumps Vs. Hybrid Bonding For Advanced Packaging
New interconnects offer speed improvements, but tradeoffs include higher cost, complexity, and new manufacturing challenges.
Fan-Out Packaging Options Grow
Once viewed as a low-cost IC packaging option, fan-out is going mainstream and upstream.
Making Chip Packaging More Reliable
Challenges rise for both advanced packages and for new innovations in older technologies.
 |
 |
 |
 |
 |
 |
 |  |
I learned a lot from reading this article. Thanks.
I would add that it is really important to include system designers in architecting your long term chiplet based design strategy. Think of these designs as the next wave of miniaturization in architecture. Internally, these “chips” will start to act more like subsystems rather than like devices or components we are familiar with. Collections of these will behave more like a heterogeneous distributed system. New design considerations enter that are workload dependent. Folks should feel free to reach out to me if interested or see my MPSoC’17 keynote or DAC’19 paper.
Thermal issues would get progressively more critical as more and more hot Processor Chiplets ( say several hot custom AI engines and fragmented GPUs ) are stacked under a very hot CPU. Adding even a dense array ( say at a pitch of 10 um ) of Cu – Cu thermal bumps that can extract the heat out to both the top and bottom ends of the stack where Heat Sinks / Exchangers may be located, would not be adequate to maintain junction temperature or thermal stresses. Unless solved, Heat Extraction from a Chiplet stack can become a show stopper. Pl. cover this aspect in your next stab at Chiplets and Adv. Pkg.
Dr. Dev Gupta
Chair, Packaging Integration
IEEE IRDS ( International Roadmap for Devices & Semiconductors )
Great Article! Learned a lot.