The Evolving Role Of AI In Verification

Experts At The Table: The pressure on verification engineers to ensure the functional correctness of devices has increased exponentially as chips have gotten more complex and evolved into SoC, 3D-ICs, multi-die chiplets and beyond. Semiconductor Engineering sat down with a panel of experts, which included Josh Rensch, director of application engineering at Arteris; Matt Graham, senior group director verification software product management at Cadence; Vijay Chobisa, senior director for product management for the Veloce hardware-assisted verification platform at Siemens EDA; and Frank Schirrmeister, executive director, strategic programs, System Solutions at Synopsys. What follows are excerpts of that discussion. Click here for part one of this discussion. Part two is here.

L-R: Arteris’ Rensch; Cadence’s Graham; Siemens’ Chobisa; Synopsys’ Schirrmeister.

SE: With AI, do we need verification experts with a generalist perspective who understand how you can apply AI to make you more productive? That isn’t something AI can tell you, particularly when it comes to knowledge of the verification flow, the tools, how they all work together, how interdependent they are.

Graham: We’re in the phase of AI where you need fairly deep expertise, because we’re still at the point of, ‘I don’t really trust it,’ or, ‘It doesn’t quite work.’ When I first started my career, somebody had wired up a breadboard with discrete components because they weren’t sure the FPGA was going to do what we thought it was going to do. Nobody does that anymore, but we’re that phase of AI, for verification or whatever, where somebody really smart has to look at the output and say AI is actually doing what it’s supposed to be doing. Hopefully we get this right before we all retire.

Schirrmeister: We’re approaching the era of snicker testing being so important, such that you look at it and you intuitively feel this can be right. It reminds me of my physics teacher who was upset with me because I was able, for a particular problem, to go through the motions of all the math and the process and everything, but my result was like three orders of magnitude off because I had some made some wrong assumption. He said, ‘Frank, you didn’t understand the basic problem. You followed the process, and I’m really upset that I give you an A minus, but your input was wrong. You clearly didn’t understand the snicker test at the end because you were three orders of magnitude off. I’m upset at you personally, but here’s your A minus.’ That’s how it is with AI. It may give you completely wrong results at the end. You need a basic knowledge of how to use it and how the results survive the snicker test, where I’m not laughing about the results because they are so obviously wrong. How do we get that into people? I don’t know. It’s an education problem. It’s a learning problem. When I was at an event the other day with students, I was asked, ‘What is the one thing you recommend us to do?’ I recommend you do what we in Germany call looking over the border of your plate. Figure out the guy next to you, the adjacency, and get a basic understanding what this guy does. What is EMI? What is thermal? How does what I do in my element, my verification, impact design and verification? How does that impact power? How does that impact thermal, and so forth? This interdisciplinarity exchange helps you get to know the generics of everything a little bit. With AI, that balance will be readjusted, because you can give it some of the more mundane tasks. But you still need to be able to identify whether that result is okay. If one thing was off on the slide, the whole slide would automatically be invalidated. So that’s the snicker test. Does it validate? In other words, you need to up level when it comes to messaging, but you need to really take a higher view and try to understand that within that realm of formal versus simulation, versus emulation, versus prototyping, is that actually in itself correct? That’s where the broadness comes to. But then, if you are going into an AMS-type simulation problem, you’ll bring in an AMS expert and you’re wondering whether that is a valid approach.

Rensch: There are a number of these ‘doesn’t smell right’ problems with technical papers. Most engineers are afraid of looking stupid, so if they don’t understand something, they won’t question it. I don’t have this fear, so I will ask questions if a paper does not make sense.

Schirrmeister: That’s something we need to instill when it comes to this balance between the specialists who can explain everything, how the electronics turn and why, versus the generalist, who understands what it means in the system context. Asking the right question is probably the key.

SE: Twenty years ago, no verification person had to think about power or thermal. Now they do. We have to add in safety and security, too, which are now verification problems — and maybe  specialist problems at the moment. But over time, they’ve got to become generalists. So isn’t there a constant change that goes on between a generalist and a specialist?

Schirrmeister: Yes, exactly. And then the tools change. AI takes care of some of the mundane tasks, so now your role evolves and you go to a different level. It’s very challenging for the next generation, especially at the speed and pace of development of all those technologies, to figure out where to specialize, because you don’t want to specialize in something that suddenly becomes a mundane task that’s being sorted out. But none of the areas like thermal, or EMI, or verification is at risk of being rationalized away, given the complexity of what we’re dealing

SE: Where is this headed in the short term? What do engineering teams need to know about here? Chiplets are coming up. If they’re not already dealing with chiplets or multi-die design, they will be, in addition to 3D-IC. There’s a lot happening all at once. How do they wrap their brains around this?

Schirrmeister: You need a whole lot more lunch-and-learns within the company.

Rensch: They need more EDA people to come and visit them.

Schirrmeister: In my realm we do cross education very specifically, so I’m very proactive because I’m very curious about these other things. I’ll just set up a meeting and learn. A lot of that is required. And from a university perspective, this will have to impact the curriculum. When I went to Technical University of Berlin, two years were spent beating out people who really didn’t like university, but it also gave me a broad understanding on everything. Then for the next two years, you specialized. That broad understanding needs to be brought back. And we do this all the time. I have people here who build AI accelerators and all that stuff in the processor domain. We do verification of this. That exchange needs to be actively fostered.

Graham: There’s that broad understanding for me that needs to also move up slightly or expand. When I came out of school, the VLSI course in my final year of university went on and on about stuck-at faults, and that was the big thing that you needed to know about when designing VLSI. And never mind that we don’t call it VLSI anymore. Now, I personally have been invited in to give guest talks to undergrads and master’s students in engineering about what is verification? What is EDA? And why is it that these students are already graduating with more knowledge of how chips are made and verified than I did? That helps push down, but hopefully it doesn’t push too much out the bottom of that, to where there’s a lack of understanding of something more fundamental. But that means we now can build on top of that more quickly, and differentiate, or expand into our specific area with that broad understanding.

SE: There’s another dimension here we haven’t touched on. We’ve talked about the verification problems of complex designs, but verification itself has become a lot more complex. It’s not just simulation now. It’s emulation in all its forms. It’s prototyping, it’s formal, it’s UVM, it’s PSS. It seems as though there’s a level of complexity that’s building in the verification chain itself that also requires a set of generalists and specialists in order to make it work.

Graham: I would have a question for my learned colleagues. Is that different in verification than it is in implementation, for example, or for a synthesis person? Don’t they have to know more about power and EMI and all this other stuff?

Rensch: Design hasn’t really changed much.

Schirrmeister: That’s a verification dude for you.

Rensch: A state machine is a state machine is a state machine. I’m not diminishing them. They have to do a lot more designing. They’ve got to do a lot more connectivity. And my company does all the interconnect for people. There’s a lot more of that. But for the actual technology — SystemVerilog, for instance — there are more constructs, but the EDA tools don’t support them, so a lot of people don’t adopt them right away.

Schirrmeister: EDA is in a very fortunate position, because every time you change the technology node underneath, everything in the tools that takes you from RTL to GDS-II breaks and needs to be redesigned. To Matt’s point, these guys are also very specialized. In my mind, there are experts for emulation, prototyping, virtual prototyping, SysML and everything. Those are experts. We try to get them to talk to each other, because they often are focused on different problems from a scope perspective — IP subsystem, system, system in context. To Matt’s question, the same is true. There are static timing experts who deal with that. There are power experts. There are signal integrity experts and so forth. And the totem pole as we often call it — this flow from top to bottom, RTL in, GDS-II out — evolves and breaks every time a new technology node comes out. So they have the same problem. Josh, you thought design is the design input to verification. Matt asked, ‘Is everything down into implementation?’ That’s the same problem we have with simulation, emulation, prototyping, virtual.

Rensch: I would agree with that. I’m just talking about the actual rank and file designers. They’re not going away, but they’re not growing. I got into verification not to beat up other people. I got into verification to make people better, because I want people to be the best. I come at it from a positive standpoint, which is why it’s really hard for old engineers when I tell them they have a bug.

Schirrmeister: It’s about improving the process.

Rensch: Yes, and as I tell the people who don’t know our industry, I’m like a book editor. Do you think Stephen King ever publishes a book without an editor? No. Verification engineers are your book editors. They’re the ones who make sure that you use periods the right way, or you don’t have 120-word sentences.

Chobisa: There’s another important aspect here. If you are doing the derivative of the chip, then you have a reference from the last chip, but the number of companies that are new to chip design have no references. They were true systems companies buying chips from chip companies. Now, those system houses are chip design houses, as well. And some of them are doing chip design for the first time, so for them there is no reference. So if you are doing verification and validation with no reference, that makes your task very challenging. With no reference, how would the team know how much real capacity they need? Many companies get into the design development process thinking they know the capacity of their chip, and they buy tools for emulation and prototyping to match that projection. The problem is that ‘usable’ capacity is different than ‘spec’ capacity. By ‘spec’ capacity I mean what the vendor claims about the capacity of the hardware platform. As you know, simulations are very, very important because block-level, IP-level, sub-system-level verification happens there. You cannot live without simulation, but for the system-level verification and validation, such as running long workloads, doing power performance analysis, doing function safety and fault scenarios to ensure your system is safe and secure — that verification and validation happens at system level. That’s why the emulation and the prototyping platforms are critical, because they have the capacity, they have the speed to enable our customer to achieve this and to reach their goals in the time given to them. The design needs to be sturdy. It needs to be predictable. It needs to have safety and security. It needs to be within a defined power budget, and all those things factored into it. And then, at the end, once you have that design in place, that design can be targeted to applications. So ultimately, the customization is to address these key KPIs and application and software-level testing.