Chip Industry In Rapid Transition

Wally Rhines, CEO Emeritus at Mentor, a Siemens Business, sat down with Semiconductor Engineering to talk about global economics, AI, the growing emphasis on customization, and the impact of security and higher abstraction levels. What follows are excerpts of that conversation.

SE: Where do you see the biggest changes happening across the chip industry?

Rhines: 2018 was a hot year for fabless semiconductor startups, particularly the Chinese-funded ones. The leading indicators from semiconductor executives are very negative, probably influenced by the concerns about trade friction and problems. EDA, meanwhile, has had a remarkable year and is doing extremely well because the design activity is so great. But the semiconductor industry is showing some concern because of the inevitable slowing of growth from memory moving back to more traditional learning-curve pricing combined with uncertainty about trade and the world economy.

SE: AI seems to have come out of nowhere to the point where everyone seems to be doing AI chips or using AI internally. How does that affect design?

Rhines: We’re moving into a customization period where general-purpose processors don’t have optimum performance and power dissipation for certain types of algorithms, and the demand for unique performance capabilities is stimulating custom chips. Those custom chips range from edge nodes, or the Internet of Things, all the way up to big data centers, where clever algorithms are being implemented in silicon. There is a fundamental push beyond von Neumann architectures to a lot of special-purpose architectures for special-purpose applications.

SE: Alongside of that, the starting point for some of these designs is changing. It’s now about data movement into the chip, through the chip, and then out of the chip, right?

Rhines: As long as you are doing machine learning algorithmically, which is supervised machine learning, you come up with clever algorithms that may run on general-purpose machines. Or maybe you get a special-purpose machine to run that algorithm. That’s the sequence of events that traditionally occurred. Now you have deep learning, which involves the machines discovering things about the data that provide information. So you need to process big quantities of data, but instead of having a fixed algorithm that looks for specific things, now the data itself becomes the starting point. And you look at volumes of data and notice there are abnormalities or outliers. That’s a different approach to computer analysis.

SE: But now you’re potentially putting together lots of different processors all over a large chip with small memories next to those processing elements. This works great for speed, but the challenge is keeping the processing elements busy enough.

Rhines: That becomes a problem every few generations. Are you limited by how fast you can pump data onto or off of a chip, or are you limited to the speed at which you can process the data? What’ you’re referring to has come up with special-purpose chips. Nvidia has talked about this. Wave Computing has a chip oriented this way. It almost goes back to the discussion about data-flow computing we had 30 years ago. You don’t want processors to sit around waiting for data. You want local memory so you can feed the local processor. But you don’t have a perfect balance for the different kinds of processors, so you have to come up with clever architectures that can get the throughput. Another element of this is that in the past, when you were doing the highest-performance processor, you would make longer and longer pipelines. We’ve hit the limits on the pipelines. It’s the same regarding performance for predictive branching, where you do speculative execution, and if you took the wrong branch you would go back. As long as the wrong choice was a rare event, that improves the performance. But now we’ve gotten to the point where the pipelines are so long that speculative execution produces wrong decisions too many times, and it ends up not speeding up. So now you’ve topped out putting big pipes together to get parallelism that way, and you’re looking for other ways to get that parallelism. There are a lot of ways to do that, but they require special architectures. There is a lot of innate parallelism and image processing, which is why Nvidia is able to do these super AI processors that have an order of magnitude better performance. They’re doing all the pixel processing in parallel. But there are other kinds of pattern recognition where you may want to do it in a different way to achieve the maximum parallelism or the maximum throughput by buffering with local memory with various types of processors inside the chip.

SE: Do the tools developed in EDA address all of these issues? Are new tools required? If so, where are the opportunities going forward?

Rhines: If you’re designing a special-purpose chip that’s supposed to process a certain type of data structure or perform a particular application, then you need to be able to simulate or emulate more and more data. It keeps us on the curve of increasing the performance and capacity of simulation and emulation. There doesn’t seem to be an end in sight. The bigger the emulator, the more people will utilize it. When we first introduced the latest generation of our emulator and announced we could handle 10 billion gates, there was a lot of fear both inside and outside of Mentor that there wouldn’t be any customers. Now there is no fear at all. We have many people lined up to get applications at the 10 billion level and looking toward the incremental to 15 billion that we’ve promised. That’s one of the challenges on the design side. The other involves the cost of design. We’ve said it’s getting so great that only a few people can afford it because a single mask set to build prototypes will be $10 million or more. New custom chip designs would be only for the big guys. And now, all of a sudden, we have dozens of companies developing custom chips.

SE: How can they afford that?

Rhines: One reason is high-level synthesis. Most of the change came in the datapath, and by going to high-level synthesis you could increase the amount of simulation by several orders of magnitude to test out new architectures. That was an enormous boon. If you look at who’s doing all of these leading-edge algorithmic chips, the leaders tend to be companies that weren’t leaders before. There are system companies—Google, Facebook, Amazon—and new people coming into automotive. They don’t have as much legacy know-how with the traditional ‘start by writing lines of Verilog.’ They’re more open to writing their algorithms in C++ and then synthesize them. That’s one of the contributors. Another contributor is the ability to buy shared emulation so that you don’t have to be a big company to afford access to an emulator. There are others. With AI-driven simulation, we’re looking at least a half-order of magnitude to an order of magnitude speed-up just from the ability to do the right simulation rather than simulating everything. Then there are the advantages of the Portable Stimulus, allowing you to minimize the redundant simulation.

SE: And those events are very much related, right? There was an inflection that allowed other companies to begin developing their own chips, and there are new people with no preconceived notions about what is the best way to get there.

Rhines: Yes, and that’s a necessary condition. Whenever you make a fundamental change in abstraction, it’s really hard for people to change. Having new people come into the business with a clean sheet of paper allows you to move a lot faster than if you have years and years of legacy. When we went to RTL from schematic capture, it was a very slow process. But the new startups and new college graduates were very quick to adopt it.

SE: With the big companies moving into designing their own hardware, college graduates who thought those were the top jobs are now working on system designs, where the challenge problems are in hardware. This is a big shift.

Rhines: Yes, and who would have thought there would be such a rapid turnaround? Everybody wanted to work in social networking a few years ago. Now you have all these exciting problems to develop the next great computer, and you have them at all levels–from quantum computing to edge node computers for automotive pattern recognition. It looks as if the money invested in new fabless startups in 2018 will be the largest in the last 20 years. We hit that peak in 2000 in the dot-com boom, and it looks as if we’re going to break through that number in 2018.

SE: This is all good news for EDA, right? These companies never bought tools before, at least on this scale.

Rhines: Right, but if there are 150 companies developing driverless cars, and another 300 companies developing electric cars, then all of those are going to buy EDA tools. However, not all of those are going to be successful in what they’re developing. So it’s great for EDA that they all have to be tools, but they don’t all end up building lots of chips.

SE: What’s happening in terms of security? Are new chips being designed with security in mind?

Rhines: It’s one more thing to worry about in chip design. Some of these companies are developing chips with that in mind, and there are new technologies around to guard against security problems. But there are probably are a lot who think the security problems are in layers above them, either in the operating system and the application software. They ultimately need to worry about that. The tools we have developed are a little slow in taking off, but there is more press about it. The DARPA program emphasizes the need to improve security. And there are more and more companies popping up to provide services associated with security. The processor people like Arm and RISC-V are advertising their ability to add security features that have gone far beyond what TrustZone initially offered. Every few years there are a few new things to worry about. But you still have to grow the gates per unit area and the processing power per dollar, despite having to add on all these new worries.

SE: Is verification a viable approach to security?

Rhines: There are lots of people offering people to do that analysis. But can you ever do enough simulation to find all the vulnerabilities in a chip? The quick answer is, no. Security of a chip is something like compatibility. You always can show that something is incompatible. You never can guarantee that something is compatible. The same is true here. You always can show there is a vulnerability, but you never can guarantee there are other vulnerabilities. It becomes one of an asymptotic approach, where you’ver verified so much that if there is a problem it’s going to be really rare and hard to get to. Beyond just the simulation, the insertion of features into a chip that can do analysis to minimize the possibility of a buried Trojan is, in fact, a key part. The other key part is on the incoming IP. How do you verify the IP you’ve got does not have Trojans in it? And who do you trust? Do you trust your IP vendor? Do you trust some kind of simulation that goes through rigorous verification of that IP? The answer is that nothing gives you 100% certainty, but there are tests you can run. You can put intelligent features into the chip that look at the data flow and look for unusual sequences, and then put up a flag when an unusual sequence is executed and possibly block the execution when a suspicious circumstance occurs. Products we’ve tested in the past merely flag unusual things going on in the system, and then it’s up to the users to look into it.

SE: Another related trend is that a lot of designs at advanced nodes are highly customized. Does that make them more vulnerable or less vulnerable?

Rhines: Customization, in some sense, is the enemy of verification. If everyone is using the same chip or architecture, then you have thousands of conditions under which it is tested, day and night. If a bug does turn up, it’s likely to be found because so many people are using it in so many different ways. If you customize, it doesn’t get the diversity of testing. So presumably you need to spend more time worrying about the one-off outlier security features on the custom chip because you won’t have the benefit of as many people re-verifying for you as it goes into field applications.

SE: Let’s swap topics. How are macro issues like trade and the economy affecting the industry?

Rhines: It’s causing a lot of anxiety. A GSA survey that’s done every month for bookings, revenue, inventories and hiring is becoming increasingly negative. November was the lowest in memory, so people in the semiconductor industry clearly are worried. There is a lot of concern the trade issues could blow up. It’s not in anyone’s interest to restrict the free flow of semiconductors around the world, and there hasn’t been anything that has done that. But if that’s not enough, when you look at the world economy the U.S. has been pretty much by itself in the recovery. Other economies are weak, and China is getting weaker. China is very important consumer and producer of semiconductor-based products. And then there has been an enormous boom in memory pricing. That will equalize over time as we bring on more capacity, particularly in China, so you can expect prices to fall. That’s negative for memory producers, but positive for others. There are a lot of dynamics.

SE: How much of an impact will China have in the short-term and long-term?

Rhines: Even though China will not achieve total self-sufficiency in semiconductors, it certainly is going to gain in terms of supply its own needs in memory and non-memory. That will cut into the revenue of suppliers around the world, who have depended on China as a customer for both internal consumption and for production of end products sold around the world. In both 5G and AI, China has a clear advantage over the rest of the world. Those are two very hot areas. For AI, a large share of the applications require having lots of data on lots of people. China has more people and more data per person, or at least the ability to acquire more data per person, than other regions of the world. They don’t have the same restrictions on collecting data on people, and they have more than 1 billion people to collect it from. Whether you’re doing a medical study for early indicators of a cancerous tumor, or a more efficient algorithm for facial recognition, a bigger database gives you an advantage. With 5G, that’s only economic where there are large concentrations of people. It doesn’t work well if you’re geographically dispersed. China has the largest, most densely concentrated population, and they have a government pumping in lots of money, and more than half of the largest producers of cell phones and handsets. Clearly, they have some advantages the European and American companies have not had to compete against in the past, so there is some concern on the Western side.