Making Connections In 3D Heterogeneous Integration

Activity around 3D heterogeneous integration (3DHI) is heating up, driven by growing support from governments, the need to add more features and compute elements into systems, and a widespread recognition that there are better paths forward than packing everything into a single SoC at the same process node.

The leading edge of chip design has changed dramatically over the last few years. International rivalries are fueling investments in more advanced chips designed, manufactured, and packaged on-shore or in friendly locations. The Microsystems Technology Office at DARPA just announced its Next-Generation Microelectronics Manufacturing (NGMM) program to advance the state-of-the-art 3DHI microelectronics through the formation of a domestic open-access prototyping and pilot line center that is accessible to users in academia, government, and industry. Those familiar with the program say it is spurring new partnerships across the semiconductor ecosystem.

And that’s just one facet of a complex, multi-tiered shift that ultimately will affect the entire chip industry. Design teams across the globe are just starting to make sense of this new future of semiconductor design, which will require a deep understanding of the tradeoffs and growing list of options available in a 3D heterogeneous design.

“The reason that you’re trying to get into 3DHI is because you have some PPA goals, or cost advantages, that you want to have,” said Shekhar Kapoor, senior director of product management at Synopsys. “The big focus is what is the most optimal design. It starts right up front. It’s easy to say, ‘I’m going to disaggregate.’ But how do you decide that? And if you do, the whole IC design paradigm quickly shifts to a system design paradigm. At that level, with software/hardware design, what workloads are you going to be dealing with? And what chiplets do you need? There is no marketplace for chiplets out there. Even if you use sub-systems that are available from somewhere else, how does that fit into the overall scope you have in mind?”

Fig. 1: Multi-die system architecture pathfinding. Source: Synopsys

Cost remains at the top of everyone’s list, but in a system context there is more flexibility. “Cost always is important,” said John Park, product management group director in the Custom IC & PCB Group at Cadence. “But in the past, packaging was all about the lowest cost. People are now willing to pay a little bit more for packaging, especially for higher-volume products.”

3DHI also opens the door to much more flexibility in the form factor, which in turn can have a big impact on the overall design.

“If you’re putting something in a smart phone, for instance, you need a thinner, lighter profile,” Park noted. “If it’s something that goes on a rack, maybe you don’t need the same sort of profile. If there’s going to be a memory bandwidth issue, you have to figure out the best memory, and the best DRAM strategy for putting this together. There are lots of different options, and packaging is still the Wild West. Some would argue there are too many options for doing packaging, but there are lots of different products. If you’re putting something in a satellite, that’s got to be rad hard, and it’s got to last 50 years and be more reliable. It depends on what you’re designing, what your end product is, and then from that typically, cost will come out, and form factor, size, weight, and power, all these other things that come along with that.”

Keith Felton, product marketing manager at Siemens EDA, breaks this down into two areas. “First is the mechanical integration between devices,” he said. “Will they be interfacing with a substrate, package or another die/chiplet? Are microbumps going to be needed, or will it require direct or hybrid bonding? The package platform technology will dictate some of this as will the die/chiplets you are interconnecting.”

The second area is the communication protocols — UCIe, BoW, AIB, etc. “If HBM is involved, you will need a silicon interposer or bridge as the mechanical interface between the PHY, and that will partially dictate the mechanical integration technology,” Felton noted. “If your architecture requires direct die-to-die integration (chip-on-wafer or wafer-on-wafer), that most likely will require direct or hybrid bonding. The integration will impact the die/chiplets design and pin-out, so using a design tool that supports parameterized, self-generating pin regions allows the designer/architect to explore multiple integration scenarios with minimal effort or cost.”

Architecture choices matter
Unlike in the past, when many design decisions were sequential and siloed, 3DHI requires a much broader understanding of how all the pieces will fit together. Design engineers must become systems engineers, and they must deal with a variety of issues that require working with engineers with different expertise, such as mechanical engineering or software expertise.

“If you break it down into too many small pieces, not only is the integration of packaging really tough to do, but also those interconnects come with overhead — extra power overhead, and a bit of latency that gets added,” said Stephen Slater, product manager for high-speed digital simulation technology at Keysight. “You have to pick and choose it for the functions that matter.”

Chipmakers that develop large ASICs know what their yield is, and they know what they’re targeting, Slater noted. What’s changed is they now must work out which parts of it they’re going to break up, and where they’re willing to sacrifice power required for the additional PHYs and complex advanced packaging.

“One thing that is clear from [industry] is that there is no other way forward,” he said. “There really is only this way. There are some who are shooting at the wall already. They are already at the giant reticle size. They can’t go larger. This is the way forward. There are others out there for whom the turning point will be when, in their future ASICs, they worry about what the yield is going to be. That’s when they start to consider breaking it up.”

But 3DHI adds a whole new axis of complexity. How will data flow through this system, which may be a combination of 2D and 3D elements developed using different process technologies? There are many more things to go wrong, from mechanical problems to deadlocks in the networks-on-chip.

“Those deadlocks are dreaded little bugs that can prevent packets from flowing and can bring entire systems to a halt, said Guillaume Boillet, senior director of product management and strategic marketing at Arteris, who noted those deadlocks come in two flavors — topology and protocol deadlocks. “While the topology deadlock characteristics are typically basic across chiplets, it only takes one port to make a ring possible. It is not unlikely it emerges as faulty if the interpretation of the specification by two distinct vendors diverges. On the protocol deadlock side, there is an effort to simplify the nature of the data flow across chiplets. Still, the nascent specifications remain complex, text-based, and thus open to interpretation. As we have seen in the past, every time protocols are bridged, there is a risk a corner case is not covered.”

On a macro level, Cadence’s Park believes everyone has the same goals, more or less, for 3DHI. But prioritization can be tricky. “If it’s something where you want to save cost, cost becomes it. If it’s something where you want to have the highest performance, then performance overrides it. Thermal comes into play. If you’re in something that can be liquid cooled in a server farm, go full out for performance. If you’re putting something in a smartphone, it better not heat up so much that you burn your hand when you touch the phone.”

Fig. 2: The needs of IC and system designers are converging. Source: Cadence

Tradeoffs in packaging become more confusing as the number of options climbs. “In the olden days with semiconductors every couple of years we got a new node, and that was that,” Park said. “With packaging, it’s constantly evolving — especially now that the foundries have entered the world. So we get probably half a dozen to a dozen new packaging technologies per year. It’s an ever-evolving thing as the foundries, and now the OSATs, are adding to their portfolio of packaging options. It used to be pretty limited. It was mainly laminate-based substrates, but now with foundries, and with 3D stacking, you have all these different options. With interposer, do I go glass? Do I go silicon? Do I go thin-film RDL interposers? Thin-film laminate? And each of these packaging tiers can add cost and complexity, depending on what you’re planning to do. If I could give you the number of options today, it would change tomorrow because someone would come out with another new way. You have the foundries, you have the OSATs, and then these packaging technology companies — someone like a Deca that comes out with a new way of packaging things.”

Packaging has a direct impact on technical specifications of the interfaces between devices in a 3DHI system.

“If you’re just talking about the physical pin pitch, if I have a super-dense, high-bandwidth type of design, I typically need to go on a silicon type of substrate,” Park explained. “That means if I go to a silicon substrate (aka advanced package), I can go to a bump pitch of 45 microns or 35 microns. If I stay on laminate, I’m at a 125 micron pitch. So it’s the packaging that determines the density of the pins, and that of course affects other things, like bandwidth.”

Other considerations that come into play here include the communication interface, which refers to UCIe, BoW, AIB and the like.

“With these communication interfaces, it’s the same thing,” Park said. “UCIe has a standard packaging, which would be a 130 or so micron pitch, and advanced packaging, which would be more like a 40 micron pitch. And then it gets even more complex because you’ve got bridges, and the part of the die that interfaces on the bridge can have a very small pitch, whereas the part that sits on the rest of the laminate has to have a bigger pitch. So you see these multi pitch dies and chiplets that are starting to come out on the packaging technology that they’re sitting on. That’s how it’s determined. With laminate we can only go so small, and with silicon we can go smaller so we can put pins closer together.”

Exploring the options
Simulation is essential to understand all of these details. Keysight’s Slater noted that for high-speed digital, the high speed interconnect to UCIe must be thoroughly simulated. The same is true for RF microwave.

“In RF microwave, we have seen certain companies talk about how the future might look with RF chiplets,” Slater said. “While this will likely come as a later development, one thing that you do see in that RF microwave market already is that in a module you’ve got different die, different substrate technologies that are being used assembled together. So heterogenous integration already has been happening in RF module design for years. There are now new ways of being able to stack the individual substrate technologies in a much more streamlined way. Once upon a time, you had to do it in a hierarchy. You had to integrate one onto another, and then integrate that third technology. And now we can just combine all of the individual technologies into one, so it’s flattening that design hierarchy and making it much easier to design. Now that you’re talking about all these different interfaces, and all these different layers, connecting from one technology to the next, you need to be able to follow the signal and analyze how the signal path is going to behave across all of those boundaries.”

Workforce issues
With integration come workforce concerns. Today, chip, package, verification and test engineers are typically in silos of responsibility and expertise. That needs to change.

“The sequential ‘over-the-wall’ approach does not lend itself to inter-discipline co-design and co-optimization,” says Siemens’ Felton. “Designers and design teams involved in HI design need to be able to look at the impact of integration and architectural choices much earlier, and make decisions and tradeoffs before the design gets into detailed implementation. This is often called ‘shift-left,’ and requires a design methodology change to what imec defines as system technology co-optimization (STCO). Of course, workforce education/enablement is a key for the future workforce, and activities are underway to address this through U.S. government programs such as NGMM, Cornerstone, etc.”

One of the foundations of a successful chip company is the longevity and cohesiveness of the design teams. This is particularly important in advanced packaging.

“If I go to an aerospace and defense account, they’ve been there, that’s what they know, and that’s how they do things,” said Park. “You go to a large commercial company, it’s the same. They’ve always been there. That’s how they do things. I don’t often see someone that’s left the world of aerospace and defense and decided to join this company building chips for cell phones. People typically stay in their focused areas. The expertise of designing these things is not just one person. But now, with this convergence of the system and the IC, it means more people are needed. You bring in the signal integrity expert to see, ‘If you move this chiplet off the interposer onto the laminate, is that still going to give me the bandwidth that I want?’ Or, you bring the thermal person in to say, ‘If I stack these two things together, is that going to work from a thermal perspective?’ It’s now a big team effort. You can’t separate the die design and the package designers. They all come together for the packaging, layout, thermal, signal integrity, electromagnetics together with design and verification engineers, place and route, sign-off — all those different people need to come together.”

Foundries’ role in 3DHI
Foundational to 3DHI are the semiconductor foundries as enablers of the entire project.

“You can’t do heterogeneous separation without the foundries,” Park said. “They’re enablers of all that’s going on, from the 3D stacking to the heterogeneous integration, to the ultra-high density RDLs. That’s because the foundries have recognized that the whole world is going multi die, multi chiplet — that’s just the future. So instead of just designing these wafers, they now need to do the back end and integrate all those wafers. That means they are now competing with the OSATs, but competition between the OSATs and foundries is always good.”

TSMC’s 3DBlox and Samsung’s CODE, for example, are huge enablers, especially for people new to package design.

“Here’s this set of files. Read those into your tool and it sets up the whole 3D floor plan of your design,” Park said. “And by the way, that 3D floorplan it sets up, we guarantee it can be manufactured that way. That’s a huge enabler the foundries are bringing. Foundries bring that formal structure of doing design versus the packaging world, which has been very informal. To get the yields, to get the price down, the foundries bring this sense of formality to packaging. Some of that is starting to spill into packaging, so we can start to have a more formal process for designing with OSATs, which is all good for the industry.”

Siemens’ Felton noted that foundries, OSATs, and substrate suppliers can play a valuable role in reducing the barriers to HI adoption. “The first step is to help their customer ensure sign-off and in-process compliance with DRC/PDKs. The next area is providing design tool enablement, such as design templates, technology tiles, assembly specifications using standards. On top of that some OSATs and substrate suppliers are offering value-add services to their customers, such as degassing and impedance matching, which offloads required effort and skills that their customers may be struggling with.”

Chiplet issues
3DHI is still considered by many to be the Wild West, and chiplets are a growing part of that still-untamed world.

“When we’re talking about chiplets from multiple suppliers — chiplets as AMD does them or as Intel does, that’s a different story, said Paul Karazuba, vice president of marketing at Expedera. “That’s just a manufacturing design choice. That’s not nearly the supply chain issues that you face with disparate suppliers of chiplets.”

While progress is being made on commercial chiplets, the interconnect standards are still not there.

“UCIe made a lot of noise a year ago,” Karazuba said. “I’m not necessarily seeing that same amount of noise today. And that might just be a factor of the fact that people are developing it. UCIe offered a lot of promise, and does offer a lot of promise. We need to get that done.”

But there are also still business issues that come with chiplets.

“Who warranties a device? How is that all going to be handled? Are there going to be universal standards for reliance? Are there going to be universal standards for how we test these chiplets? Realistically, that’s probably going to be set by the name on the outside of the multi-chip module, because we need to come up with a better name for an assembled chiplet,” Karazuba said. “The name on the outside of the box is probably going to be the person who determines all this, but how these all interact with each other is still an unsolved issue. For something as simple as, ‘How are you actually going to physically connect all of these,’ the interconnect on packaged monolithic silicon today is still very different from place to place, node to node, etc. So there’s a lot of work that still has to be done. But the one thing the semiconductor industry has been good at for 70 years is figuring out incredibly complex problems and making it seem so simple when it is so not.”

Related Reading
Everyone’s A System Designer With Heterogeneous Integration
Engineers are encountering more unknowns, working with different people and tools, and focusing on new types of tradeoffs.
Heterogeneous Integration Finding Its Footing
Definitions, applications, and tools are still evolving, but success stories are becoming more common.
Chiplets: 2023 (EBook)
What chiplets are, what they are being used for today, and what they will be used for in the future.