Disturbance In Verification

When writing my recent story about agentic verification, there was one quote from Abhi Kolpekwa, senior vice-president and general manager at Siemens EDA, that really struck a chord. He was talking about the additional token costs that would be consumed when a verification engineer starts asking the agents to do what was considered to be part of their job.

“Consider the total cost of ownership,” he said. “This is where you’re able to get the maximum out of your processes by applying the minimum of your hardware.”

This is not the first time that verification teams have had to completely reformulate their methodology and work out if new technology is cost-effective. When I first started in my career, test generation was one of the big tasks for the industry. While this used early logic simulators, it was targeting manufacturing test — how to efficiently and effectively develop a minimum number of vectors necessary to obtain maximum coverage of the device under test.

Every additional vector meant a longer time on the tester, and that directly added to costs. Every missing vector meant the possibility of escapes, which impacted your credibility. The designs of the time were tiny in comparison to the ones we have today, and all of them were developed as gate-level schematics. I can remember rooms with large printouts of the schematics on the wall, pins inserted on the outputs and by the gates, and colored strings run to highlight paths and activations. They knew they needed a better method, but back-tracing algorithms were only just emerging, simulators were slow, and modeling languages had not yet been standardized.

I was involved in one of the first simulators that targeted the emerging semiconductor industry. It had both a logic capability and the first register transfer level language, which was only an SED script away from Verilog. While it revolutionized one aspect of the industry, it still relied on vectors that had to be hand-created. It also provided a fault simulator and some rudimentary test generation capabilities. Those algorithms were only effective for a few levels of logic, and so they never became an industry norm.

Directed test, which was a set of input vectors and a set of output vectors against which the design outputs would be compared, was usually generated by hand. Each time the design changed, all of the tests had to be manually updated. Timing or behavior might have changed. Every test that failed after a design change had to be manually reworked. It took a lot of time.

About five years after that, Sun Microsystems released code for a random test pattern generator and a startup (I think they were called Silicon Solutions – please correct me if I am wrong) productized it and started a massive retooling of the verification industry. It traded off time of the verification engineer for additional compute time. This technology slowly evolved into SystemVerilog and UVM. Now, verification engineers spend their time creating a set of rules for how vectors are generated, a model to predict the output of the design, and checkers to ensure the correct things happen. Because it was not known what each automatically generated test would do, a coverage model had to be developed to keep track of how much was actually being exercised.

In my opinion, UVM and SystemVerilog were highly inefficient, and the technology did not evolve in the way that it should have. Vector constraints were only combinatorial in nature rather than sequential, so many useless tests were created. The generator also ran in real time with the simulator, so while it always meant alignment between what was created and what was checked and reported, it also meant that every simulation was slowed down. In the early days, it was usually worse than a 2:1 ratio.

However, the ROI worked out. At the time, there was lots of chatter about how many verification engineers would lose their jobs, but none did. Design sizes grew so rapidly that there was always more verification to be done. In fact, verification engineers became even more sought after because they were now specialists. You had to know how to use additional languages, libraries, and methodologies.

We are now in the early days of the next change. Verification agents automate additional parts of the role of verification. Again, it is transferring human resources into additional compute resources, and the ROI for that has to be evaluated. With the meteoric rise of AI and the compute resources required to feed it, many companies that produce the components of the data centers are awash with cash, and getting to market faster is what keeps them ahead of the competition. I doubt if many of them are complaining about the added compute cost, especially since many of them are running on their own hardware.

But what about the larger industry? Many are saying that if they do not embrace agentic solutions rapidly, they will fall behind and be replaced. Possibly.

What about the verification engineer? While I will write about this more extensively over the coming months, some of what made a verification engineer a specialist may be weakening. If they now just ask agents to do their bidding in English and are no longer expected to write code, then what barrier is there to their jobs? Experience still counts – perhaps even more so, but how do you get the necessary experience? What is the path for an engineer to get to the necessary level that they can effectively use the new solutions? Will the new solutions be able to capture the knowledge of the experienced verification engineers, and if they do, can anyone now perform the task?

The future is never certain, but what is clear is that the role of the verification engineer has never seen this big a disruption in my lifetime, and that effectively means over the lifetime of the industry.