3D-IC Ecosystem Starts To Take Form

The adoption of chiplets is inevitable, but exactly when a mass migration toward this design approach will begin is yet to be determined.

Nevertheless, some of the biggest technological and business-related barriers are being addressed. And while a chiplet-based design remains beyond the economic reach of many companies today, that is starting to change. Early signs of an emerging ecosystem already are showing up.

Not long ago, the chip industry believed that the only companies that use chiplets were those with no other choice. There was a long list of reasons why this approach didn’t make sense, including cost, performance, and added design complexity. The trend of consolidation and integration onto a monolithic die seemed to be the path into the future.

That has since proved to be a faulty assumption. One by one, chiplet-related issues are being resolved or shown to be overstated. More people are stating a case for why a growing number of companies should consider disaggregation.

“With silicon and glass interposers, the metal layers are lower-capacitance and have better signal integrity,” says Robert Patti, president of NHanced. “The thicker copper wires on an interposer have better signal propagation than you can get on a chip. If I take my SoC and cut it into 20 chiplets and re-assemble it on an interposer, it will not increase performance by an order of magnitude. But it will outperform what it was as an SoC. You can communicate across the chip faster with lower signal loss and use fewer repeaters than you did before. 2.5D chiplets are a necessary stepping stone. From there, the step to 3D is pretty short because you’re starting to sift out these logical protocol issues. The physical interface going into 3D is really a put. Once you’ve agreed that you have some standardization of physical interfaces at 2.5D, going from 2.5D to 3D is a pretty trivial exercise.”

It also will become easier over time as more companies adopt this approach, and the benefits will become more pronounced.

One of the biggest advantages of soft IP was that a company no longer had to be an expert in everything. “If I am making an SoC, I have people doing analog design,” says Mayank Bhatnagar, product marketing director of SSG at Cadence. “They may be good, but when my SoC is done, they move to some other project and they work on something else. They do not end up becoming experts for one given design because they are doing different things over time. Now, imagine if there was a dedicated company, or a smallish company, which focused on doing a particular design. They get to be very good doing the same design over and over again, using different process nodes or different speed grades. The quality will be better, not because the exact same IP is being implemented. The quality will be better because the IP itself is getting optimized over and over again by the same people. The IP gets optimized when it is done by a dedicated team who know they have a significant market, so they can invest in hiring the best talent to do it. And the reason they have a significant market is they are using this marketplace with standard interfaces, so they know they can target most of the users of their IP.”

As with any technical advance, there are multiple ways in which things can move forward. “When we talk about chiplets and reuse, the general notion is that these are hardened entities,” says Abhijeet Chakraborty, vice president of engineering at Synopsys. “They are completely placed and routed. You have a GDS. It’s all done. You just have to plug it in, connect everything, and you go. We will see where the industry evolves, but there is certainly another possibility because that usage is going to be more limited. There will always be a need for parametrization. You can’t take a die that is fully baked from the shelf — even if you are using standard interfaces — and just plug it in, because you need a few more tweaks. Parameterization is important. You may find soft chiplets, which can be hardened by that chiplet vendor. The chiplet vendor provides you the soft chiplet. It takes your changes into consideration, then presses a few buttons, and boom, it goes through the process. Out comes a hardened chiplet for you. It’s still a lot easier than putting the entire burden on the end user, so the chiplet vendor could still take on that responsibility.”

Chiplet evolution
The introduction of soft IP in the ’90s was not easy. The path to adoption was difficult, and it lacked standards. Many people doubted its success because of the inefficiencies it would create.

“I see no reason in principle why the adoption of chiplets can’t be a carbon copy of our evolution from IP for the SoC,” says Marc Swinnen, director of product marketing at Ansys. “There are mechanical and thermal models that need to be created, and it is more complex, but I don’t see why we can’t solve it. It is just a question of getting the work done and agreeing on the standards and trying them out. When you look back at the early days of the IP market, the first instances of IP were internal libraries. It was companies using their own IP, and they had their own internal standards. It was all very proprietary. It took a while for commercial IP to pick up, and for people to trust blocks from the outside. The same is going to happen with chiplets.”

The chiplet industry is still at those early stages. “I’m starting to see customers who are developing their own sets of chiplets,” says Nhanced’s Patti. “They are not compatible with anyone else. That usage is going to continue to grow because companies have latched on to the value of doing their own ASICs as a chiplet. It allows them to have more leverage. People are going down that road when it comes to HPC, compute complexes, accelerators, even memory subsystems that tie into that. You have a pocket that is very focused on leveraging UCIe, but because of the overhead required, and the fact that you have to be in a reasonably small node to really get effective use of UCIe, it’s going to be somewhat of a niche market rather than a broad market.”

Over time, more standardization will appear. “We’re going to see a trend toward more standard custom design, where you can have predefined footprints for whatever is on top,” says Mark Kuemerle, vice president of technology and CTO of custom solutions with Marvell. “The base die is the hardest thing. The top die is easy because it just drinks power and requires pipes to send data. The base die is where all the fun is because you have to adapt that. You can imagine where you have a standardized footprint for a given type of thing, and then you apply that as a template to your base die design, to your bottom die design, and that enables you to design to it.”

Of course, it is not quite that simple. “It is an order of magnitude more complex than the soft IPs that we’ve come to know and love for all these years,” says Synopsys’ Chakraborty. “You’re talking about the complexity of the chiplet itself, but also about test standards, interfaces, reliability, and security — all those kinds of things that have to come into play. This has to be designed with different technology rules and parameters, and each has different requirements and challenges. Power delivery is very important and a critical problem, and chiplets have to address that, including how they lend themselves to receiving the power from the system. We are talking about TSVs and bump power for supporting the power demands. All of those have to be handled. They all have to come together.”

Fledgling ecosystem
There are other challenges, as well. “There is definitely a need for it,” says Cadence’s Bhatnagar. “There is a demand for it. But there is no supply for it. There are companies that are beginning to develop chiplets and to sell them as commodities, but it’s at a very initial phase. The biggest roadblock is the investment required. Creating any chiplet takes a long time, costs quite a lot of money, and you want to be comfortable with the market space that you are able to target. Given that the standards are evolving, it is unclear how large the market is for a given chiplet, no matter how good it is. If you can’t capture any market, if nobody can use your chiplet, then developing it is of no value.”

There are some organizations that can help get things moving. “The chiplet ecosystem is evolving with initiatives like the Open Compute Project’s Chiplet Marketplace,” says Rozalia Beica, field CTO for Rapidus Design Solutions. “This supports the adoption of chiplet-based designs by providing a centralized resource for designers and manufacturers.”

Still, to capture a seat requires the ability to accept a chiplet and a defined benefit for doing it. “There are companies in the market today that are offering chiplets,” says Pratyush Kamal, director of central engineering solutions at Siemens EDA. “They are up against the big verticals, and for them to have a successful play, they need to take the industry toward an open chiplet economy. The vertical companies that are designing SoCs currently have five different organizations building these cores. If you can eliminate the need for one organization, they could save 20% in non-recurring expenses — the expense that goes into designing and taping out a chip. You can remove that expense from your cadence. There is a big opportunity in that sense.”

Arm, for example, has begun offering pre-integrated hardened IP modules to speed up the adoption of chiplets. “The cost of validation is ballooning,” says Christopher Rumpf, senior director of automotive at Arm. “The way around that is to use compute subsystems. The workload and the software is where differentiation is happening. So we will continue to sell IP products, but now we will be assembling them into a larger subsystem using CSS (Neoverse Compute Subsystems), which is a standardized compute platform. That has a lower cost of porting and validation, and it’s a precursor to the chiplet world.”

Some chiplets make more sense than others. “We’re going to gradually get agglomeration,” says Patti. “The folks who are building the vertical chiplets are going to want to talk to each other, and they will want FPGA chiplets. I expect we will start to see FPGA chiplets of varying size. What we really need are relatively tiny FPGA devices to glue together between these non-co-engineered chiplets from 30 different companies that are doing their own protocol invention.”

DRAM has started to show the kind of ecosystem required, particularly with high-bandwidth memory (HBM). “The ecosystem, ease of reuse, and standardization have to be solved,” says Chakraborty. “There is a cost associated with it. There are challenges with assembly. They have a lot of questions today. They are not clear how to do it. They’re not clear how reliable it’s going to be. They worry about thermal, for example, if they go to stacked die. Is it worth taking all those risks when I have an alternative path forward to do this? Once the industry helps by greasing this path — making it a lot lower risk, where the user can be knowledgeable upfront about the risk and challenges and they can make an informed decision — then we feel that more adoption will extend beyond the HPC AI customers. It’s inevitable that it is going to happen.”

Extending 2.5D to 3D
When performance is the ultimate goal, it makes adoption of chiplets more difficult — especially if it involves 3D integration. “We have never managed to re-use a chiplet because they’re so intertwined,” says Marvell’s Kuemerle. “We have to design multiple die in lockstep with each other so that we can match everything up. How do we get power up to the top die? It’s got to go through the bottom die, and the details of how that’s done is really unique — and not just to each die, but even to different terrains on a given die. If I know I’ve got a hot spot above me that needs to drink a lot of current, I must make sure I have enough ladders for that current to get up to it, and that mucks up my whole floor plan on the base die. I have to work around that, and so all the challenges we were having with 2.5D that caused us to make these custom-crafted, exactly matched chiplets, are now doubled if we move to 3D integration. I don’t just have to think about how they interface together along an edge. I have to think about how they interface together across these two dimensions.”

While standards are being worked on, they are seldom adhered to. “The broad market is problematic because there isn’t one specification,” says Patti. “I’m fond of Bunch of Wires in a broad sense, but it’s going to be a mixed set of standards to glue things together. We’re going to live with glue logic to put together chiplets. It’s going to be a long time until a de facto standard shows up in the market. We’re a little bit stuck, but it’s the people who have to do it who are going to drive it. As it relates to 3D, it’s even bit messier because 3D has the physical aspect, beyond the physical protocols. If you are doing things in 3D today, they are custom-engineering the interfaces in the stack. And in many cases, it’s memory on logic, and it’s custom memory on a custom piece of logic. It’s very fine-grain, and it doesn’t look like anybody’s standard — nor will it ever look like anybody’s standard.”

It will take the right combination of company and product to make it successful. “There are certain systems that are based on performance parameters, and they’re constantly pushing them from one revision to the next, or from a different architecture to a new one,” says Chakraborty. “The only way to achieve those objectives is to redesign everything together with a full system perspective. But there will be many applications where you don’t need that. You can take advantage of reuse, because that allows you to not only reduce your cost, but also achieve your end result, your completed solution, much faster and in a more predictable way. For those applications that lend themselves to modularization and reuse, it comes back to the question, ‘What is needed to enable that ecosystem?'”

With each application, more will be learned, and the industry will move forward. “Custom HBM is the closest thing I can think of that is moving the industry closer to a reusable 3D design point,” says Kuemerle. “It is a very isolated and well-known problem. If we put in the hooks to talk to that 3D stack of DRAM die that are on top, we can treat it like an IP block. We have other stuff that is aligned with that DRAM stack, like power delivery, that we have to plan for in a base die. But when we look at that, we can define all the stuff that’s required to talk to that top stack, or whatever is on top, and we can work around that and do whatever we want with that space.”

Related Reading
3D-IC For The Masses
Advanced assemblies have enabled an unprecedented rate of advancement in the data center, especially for neural processing, but can it expand beyond that?
Chiplets Add New Power Issues
Well-understood challenges become much more complicated when SoCs are disaggregated.
What Exactly Is Multi-Physics?
The chip industry’s new buzzword comes with lots of implications and some vague definitions.