Verification Methodologies Struggle To Keep Up With AI

Engineers are flooded with new capabilities. The problem now is how best to deploy them.

popularity

Key Takeaways: 

  • The rapid development of AI has resulted in new capabilities being provided to verification teams, beyond their ability to rationally insert them into accepted methodologies. 
  • There is a lot of uncertainty about who will benefit the most from this technology. Is AI a junior engineer replacement or an enhancer? 
  • The biggest benefits will come when AI helps engineers understand a larger context than is possible today. 

Advances in EDA tools traditionally were driven by specific needs, but the focus now is on using AI in any way possible to get to market faster and with less reliance on humans. The problem is there is no playbook, so each company must forge its own path — and hope it doesn’t fall behind.  

In the past, verification methodologies were developed to best utilize the available languages, models, and tools. They all evolved together. When new demands were placed on the verification methodology, they would normally be addressed by adding new capabilities into tools. This was the quickest way to determine if those new capabilities would solve the problem. It was done cooperatively between an EDA vendor and a design or semiconductor company that had the problem. Invariably, additional customers would then need that capability, and it would become cemented into a methodology, and if necessary, codified into language extensions.  

This flow appears to have been tossed aside with the rapid development of AI. New capabilities are being developed at breakneck speed, even though in most cases the underlying tools are not changing. Technologies are not being developed out of a specific need, but because they provide a capability that may be useful. Nobody has time to really think about methodology. It is more a matter of how fast AI can be shoehorned into every aspect of work. 

“It is incredible how voracious the consumption of AI is,” says Bradley Geden, director of product marketing at Synopsys. “At the executive level, everybody is nervous about missing out. If their competition is suddenly able to tape out a chip in 6 months versus 18 months, then they lose. There’s definitely an arms race to adopt AI.”  

That means deploying AI in every aspect of the development cycle. Paul Graykowski, director of product marketing at Cadence, poses the questions on the tip of every one of his customers’ tongues: “What more can I do? What can I do better, quicker, faster or higher quality?” 

“Everybody I have worked with as a customer talks about schedules being so tight,” Graykowski said. “There is so much more always being added to the plate. ‘You have to do this, this, and that.’ Their customers can be really demanding. They are facing many pressures, such as how they improve quality, do more, and do it better. In many companies, they can’t hire, or they have limited budgets. Today, most of them are attempting to use AI to get more done, to ensure they can get to market at the right time. If they can supplement their staff with virtual engineers and drive it in the right direction, they may have more control over their schedule, so they could forecast it better.” 

While much of this is hard to quantify, it doesn’t stop people from trying. “Early customer lessons show that agentic workflows can turn a 150-person verification team into the equivalent output of a 400+ person team,” says William Wang, CEO of ChipAgents. “This is enabling smaller teams to compete at a much higher level, while allowing large organizations to move materially faster.” 

While going faster is clearly a goal, it cannot be to the detriment of quality. “They are interested in saving time to do more,” says Abhi Kolpekwar, senior vice-president and general manager at Siemens EDA. “If it used to take 10 resources to reach verification closure. But now, because of AI and agents, there could be resource savings. Can they now do more verification? The important thing is they want to do more with the people they have. And they’re seeing the value. The third area is speed. They all want faster coverage closure. They do not want to give up quality, but quality is something they don’t want AI to decide. They want the verification engineers who have 30 or 40 years of experience to be the auditors of quality.” 

 It is all about efficiency today. “Once adoption becomes stabilized, they can start seeing what the next natural step is going to be,” says Cadence’s Graykowski. “How do we do it even better? That extends to areas such as looking to improve PPA. Will that enable us to improve our time to market? Can we have fewer bugs?” 

How those gains are ultimately used is still uncertain. “AI-assisted verification can help teams explore more scenarios, detect ambiguity earlier, and improve consistency in system-level coverage,” says Andy Nightinghale, vice president of product management and marketing at Arteris. “The near-term outcome is likely to be better products rather than simply more products. Over time, increased productivity also may enable more derivative designs and faster product cycles, but only if teams trust the outputs and integrate agents into real engineering workflows rather than treating them as isolated assistants.” 

Who does it help? 
This creates an interesting dynamic within the industry. When an EDA vendor is asked about how a certain feature or capability is expected to be used, there are no clear answers. It is sometimes not clear who will use it. 

“There is the idea that you are bringing a junior engineer on board to help with these things when you start to deploy agents,” says Graykowski. “It is an assistant to a senior engineer. The senior engineer is in the loop, giving out these tasks, having the tools do some work around a problem. But if you don’t have the necessary design experience, the quality of what you get back may suffer, or you may just be doing it inefficiently.” 

But it can help human junior engineers. “For new hires, they do not have to learn the tools,” says Siemens’ Kolpekwar. “They do not have to learn the setups. They are passing those commands to agents, and agents read the context and set things up. The familiarity with the tool, and then the application of formal assertions and properties, has lowered significantly — by multiple times — the entry barrier that fresh graduates used to have to basically assimilate into a commercial world and become productive.” 

But the jury is still out on this. “Initially, we thought that it would enable a junior engineer to bring up a formal bench and that they would become an expert from day one,” says Ramesh Narayanaswamy, member of technical staff at Synopsys. “While senior engineers are perfectly capable of doing this, they see it as helping them take care of some drudge so that they can get to the core thing, such as finding that hard-to-prove assertion. This is where they bring their special skills. The senior engineer uses it to give them more bandwidth and do certain things they might not have done.” 

Still, there are potential benefits at both ends of the skill spectrum. “The gains accrue to both ends of the experience spectrum,” says ChipAgents’ Wang. “Senior engineers effectively gain a virtual AI agent team that amplifies their expertise and decision-making, while junior engineers become force multipliers who can execute at a much higher level and ramp significantly faster.” 

Ultimately, we may see two classes of AI assistants. “Just having your documentation in RAG and being able to have a chatbot provides significant help to junior staff,” says Graykowski. “They can use that in debug to provide guidance on tracing this signal. Who drove this signal? You can ask the tool, and you can learn that information. The tool can guide you. A lot of tools now have full blown APIs associated with them, and you can ask complex things. An agent will write the script for you, give you the script, or execute the script. You can save a lot of time. That is a great way for a junior engineer to benefit from just using an assistant inside of the tool.” 

Others believe the biggest benefit is for senior verification engineers. “With the way that we are framing our tool, we are saying the AI agent is a very junior verification engineer,” says Hamid Shojaei, distinguished engineer at Cadence. “That means the agent needs to be mentored by a senior verification engineer. They need to review the code. They need to give the agent feedback and work with it until they get the results. We can give maybe 10 junior verification engineers to one senior verification engineer, and they can go much faster.” 

The deployment of AI may indeed overwhelm a junior engineer. “The capabilities of a general-purpose AI tool are meant for senior people,” says Shelly Henry, founder and CEO of Moores Lab AI. “If you give it to a junior person, they could be completely lost, and they wouldn’t be able to do much with it. But if you give that to a senior person, they will make the best use of it and come up with test cases and test benches very fast. The question is how we enable junior engineers to be more productive, because you don’t have that many experts in every company. You can only expect 10% of the engineering people to be experts. We still have 90% who are junior engineers. How do we make them super productive? That is the million-dollar question. A senior engineer knows exactly how to navigate to that goal, but a junior engineer could get lost.”  

It is when you push down to the next level of detail that the picture becomes a little fuzzier. In truth, nobody yet knows the best way to deploy AI to consistently and reliably provide defined benefits, and the impact that it will have on how the teams are organized and the methodologies evolve. 

Larger scope
One of the central concepts that came out of the previous story in this series is that a mental model has to be created. “The mental model takes in all kinds of inputs — test code, your RTL, your specs, and then you work with it, talk to it, and you build this model,” says Graykowski. Based on that, it can generate test plans, it can generate coverage points, it can generate UVM code, it can do all those types of things.” 

This is what senior engineers currently rely on. It is their expertise. “Initially, experienced engineers are likely to benefit most because they understand design intent, verification strategy, and how to identify incorrect or misleading results,” says Arteris’ Nightingale. “They can effectively guide and constrain the agents. Junior engineers will also benefit through workflow assistance and knowledge acceleration, but there remains a risk of over-trusting plausible-looking outputs. Verification still requires engineering judgment because silicon does not tolerate ambiguity, even when AI occasionally does.” 

The big question is how you can capture the skillset of the senior engineer so that a junior engineer could do the same task. “There are things that senior engineers have as methodologies, or experience about the way code should be rewritten if you want better performance, or make it more verifiable,” says Synopsys’ Narayanaswamy. “Those senior engineer skills can also be provided as a first layer of mentoring, as a skill, where you could say it can do a first-level code review. It would not be as good as doing it in person, but you get that first 60% out of the way. You are boosting the skill of the junior engineer. The model will eventually be that a junior engineer needs to step up to thinking like a senior engineer. He doesn’t have reports, but agents.” 

Debug associates itself with experience. “If you have an experienced verification engineer, they usually know how to read through the signatures, take your signature and map it back to the potential root causes and go from there,” says Kolpekwar. “That takes a lot of baking time into one’s career to become effective as a debug verification engineer. This is why we continue to spend more than 40% or 50% of the total time debugging. This is where the experiences may be mixed, and they’re mixed for a variety of reasons.” 

This is an aspect of it that few people are talking about. Who sets up and trains these agents? “For somebody to be very effective in debug, they really have to understand and train their agents into a customer ecosystem so that they have very high-quality contextual intelligence to read the system, understand the failure signature, and do some root cause analysis,” says Kolpekwar. “With a simplistic setup, agents may not have been able to do very good job. Some complex setups, where there are three scripting layers above the tools, before it comes to the user the agents are going to take some time to basically bake in that system, and this is where the responses could be valid.”  

But there is a big gap. “For a junior engineer, you need an objective metric,” says Narayanaswamy. “The one thing that junior engineers do not have, and which they are supposed to be learning from the senior engineer over time, is judgment.” 

Perhaps students will start to learn new skills in college that would enable them to come up to speed faster. “If you don’t know how to program, what you get out of it’s not very good,” says Graykowski. “But if you know how to break down a problem the right way, it’s those prompts. Knowing what to ask the tool to get you to that next step is super key. That’s where the level of your engineer’s experience really matters.” 

The required skill set change could be large. “For the future, junior verification engineers need to learn new skills,” says Cadence’s Shojaei. “They need to know how to write a good spec for these agents so that they can work. They need to learn about signoff, instead of writing code and understanding UVM details, and so on. They need to know how to give feedback to the agent, how to review the results from agent, and finally, do the signoff. That’s a new skill that they should start learning from college, rather than spending so much time on Verilog and UVM. The coding part is going to be replaced sooner or later.”  

Editor’s Note: The final part of this series will explore the evolution that is being forced upon engineers, tools, methodologies and the EDA companies by AI. 


Related Article
Creating Agentic EDA Methodologies
Current approaches involve multiple tools, vendors, designs, data formats, and abstractions. Can agents really use them all?



Leave a Reply


(Note: This name will be displayed publicly)