Rethinking What Goes On A Chip

There are hints across the chip industry that chipmakers are beginning to reexamine one of the basic concepts of chip design.

For more than 50 years, progress in semiconductors was measured by the ability to double the density of transistors on a piece of silicon. While that approach continues to be useful, the power and performance benefits have been dwindling for the past couple of nodes. In addition, the cost per square millimeter of silicon is rising is rising due to basic physics. Even the rollout of EUV can’t alter that equation.

Shrinking features will continue to be a path forward for those who can afford the rising costs, although even that is being combined with some sort of advanced packaging to improve performance, reduce power and boost yield. In the context of a vertically integrated company developing a smart phone or an AI chip, those are still considered acceptable costs. But only smart phone chips are produced in sufficient volume to achieve massive economies of scale, and even that segment is beginning to slow down due to market saturation and more localized competition, particularly at the low end of the market.

AI, meanwhile, fits into a number of different verticals and chips are developed in smaller batches of unique designs that today are not replicable for other applications or even other companies in the same application space. This is why the GPU has done so well in the training market. It’s far from the most efficient choice, but it’s inexpensive and it works well enough in large arrays.

This is where a realignment begins to make sense. With the smart phone market flattening, there may be a shrinking number of vendors shipping a billion units of advanced-node SoCs. And with AI chips, the emphasis in many cases is on highly customized architectures, and those are different from one cloud provider to another, and from one industrial application to the next.

None of this lessens the need for increasing compute and memory density. Any AI/ML/DL application can benefit from more of both of those, as long as the chip is architected to move data through without any bottlenecks. In fact, those chips may be able to achieve several orders of magnitude performance improvements per watt and maybe more, according to multiple industry sources. The challenge will be getting all of that onto a single die, because many of these designs are full reticle size.

But rather than relying on shrinking features to open up more real estate, design teams are beginning to explore what they can remove from the die without impacting performance or cost. There are certainly tradeoffs in performance by moving some features off-chip, such as a SerDes or analog IP, but if that area is used for more processing and memory, then chipmakers can still achieve significant performance improvements. The key is prioritizing data at a system level, and working backward in designing the silicon.

This is a completely different starting point for architecting a chip, because it begins with a fixed amount of space. From there, the focus shifts to making tradeoffs at I/O level, and then developing the hardware and software architectures around that. In effect, this is a new way of approaching hardware-software co-design, and it has big implications for the entire chip design ecosystem. The real focus shifts from just moving data, to segmenting which data gets moved furthest.

What makes this idea intriguing is that it doesn’t require any significant changes in how chips are laid out, verified, debugged or tested. The same kinds of external and in-circuit analyses still apply, and the same materials and techniques that have been proven at a single process node continue to be viable. But it dramatically changes the ROI and time to market for high-performance chips.