AI Drives A New Wave For Semiconductors

In Cadence’s recent earnings call, Lip-Bu Tan, our CEO, talked about the five waves that are hitting us simultaneously.

Here’s what he said:

First of all, I’m excited about this industry, because it’s very unusual to have five major waves happening at the same time. You have the AI machine learning wave and you have 5G is starting to deploy and then you have the hyperscale guy, the really massively scaled infrastructure. And then we have autonomous driving and then the whole digital transformation of the industry group. And then as I mentioned earlier, clearly some of these big system companies and service providers, they are quietly building up the silicon capability. They’re also reaching out to us to really expand beyond that to the system analysis space. And so I think we are excited about the opportunity in front of us.

Normally in the semiconductor industry, there is one major trend driving the business. In the very beginning, a lot came from the military. Then, when computers proliferated into businesses, it was building the components used to create computers. This was still the era before the microprocessor, when a chip would only hold a few gates. Many computers were built with TTL logic, which maxed out at something like the 4-bit ALU, the 74181. See my post Carry: Electronics for more details on that. Once the PC took off, the semiconductor industry rode that wave until the early 2000s. Then cellphones became a big driver, especially after 2007 when the smartphone explosion began. Even though PCs are no longer on the growth path they once were, it is still a 250M unit market, and there is a lot of silicon inside one. Smartphones are an order of magnitude more, at over 1.5B units per year.

I know it is obvious, but Cadence gets most heavily involved with companies during (and even before) the design phase. It would be wrong to say we don’t care about production volumes, we want our customers to be successful. But we don’t (mostly) directly participate in production volumes. Whereas a foundry is the other way round—they don’t care about much else. If a modern foundry is anything like an ASIC company used to be, it doesn’t make any money on designs that don’t enter production or have much lower production volumes than planned (and priced). What this means is that Cadence cares about a market a year or more before it is in high-volume production.

Lip-Bu called out five markets. Let’s take a look at the first one.

AI and machine learning

AI and machine learning (and various other names) is in a mode that is hard to describe. In one sense, it is experiencing fast growth, and in another sense, it is too early. The sense in which it is experiencing growth is in design starts. There are literally hundreds, if not thousands, of fabless AI chip startups, and dozens of programs in established semiconductor companies to create AI chips or embed AI technology in other parts of the product line. The sense in which it is too soon is that most of these chips are being designed and are not in volume production. Right now, the big winner in terms of production silicon is NVIDIA, whose GPUs are heavily used in some servers to do AI training.

The growth of AI projects is driven by a couple of things, I think. First, neural network technology started to “work” over the last decade. Although neural networks have been known about and researched since the 1950s, nobody could work out how to program them. Then Yann LeCu, Geoff Hinton, and Yoshua Bengio worked out how to use gradient descent to tune the neural network weights. They won the Turing Award in 2019 for this work, which you can read about in my not-cleverly-titled Geoff Hinton, Yann LeCun, and Yoshua Bengio Win 2019 Turing Award. Another big consideration was that this training approach requires huge amounts of compute power, which was simply unavailable until recently. The combination of cloud data centers with GPU accelerators changed that and made training large neural networks feasible.

As AI algorithms moved out into edge devices such as smartphones and smart speakers, there was a desire to do more of the inference at the edge. The power and delay inherent in uploading all the raw data to the cloud was a problem, as are privacy issues. But inference at the edge requires orders of magnitude less power than a data center server or big GPU. In turn, that has led to a proliferation of edge inference chip projects. It reminds me of the mid-1980s when I was at VLSI Technology and we had a couple of dozen companies working with us to design ASICs to create PCs with special capabilities—and each one of those dozens of companies had a business plan to be 20% of the PC market. I think pretty much every one of them failed, since it turned out that people didn’t want a PC with a lot of differentiation, they wanted a boring standard PC that would run MS-DOS and, later, Windows. Most differentiation came in two areas other than the electronics: portability (Compaq) and business model (Dell in particular). When integration levels reached the point that everything except the memory and the processor in a PC could be put on a chip, it turned out to be VLSI technology ourselves who were successful in the PC chipset market (along with a couple of competitors). Indeed, at one point, Intel was OEMing our chipset in a bundle with their processor. Then, as we knew would happen eventually, Intel took over that market for itself.

I suspect that a similar market dynamic may play out in deep learning since the cloud data center providers are also building their own in-house AI chips. For example, Google’s TPU (see my post Inside Google’s TPU) and Amazon/AWS’s Nitro (see my post The AWS Nitro Project). So in the same way as there turned out to be a very limited market for mobile processors, there may turn out to be a limited market for AI processors: the system companies will optimize their own solutions.