Toward Agentic Verification

Key Takeaways:

• Agentic verification provides flow orchestration for common repetitive tasks.
• Capabilities will expand when tools can learn from a larger context, including the specification.
• Design houses need to fully understand the costs and benefits and plan accordingly.

Agentic verification is more than a buzzword. It is a pivotal moment in the evolution of verification methodologies, and the transformation is just beginning. The steps being made today are just laying the groundwork for bigger changes to come.

For decades, it has been possible to design more than can be fully verified. Verification tools and methodologies have been evolving, but not fast enough to close the gap. And recent results from the Siemens/Wilson Research study suggest that the situation is getting worse.

Advancements in AI are prompting many people to question whether AI will help close the gap. Others are questioning whether AI can deal with rising complexity and how it will impact engineering jobs. Longer-term, the tools themselves will need to be assessed for suitability in the age of AI.

What is clear, however, is that verification is about to undergo a significant transformation. Being left behind is not an option for EDA or for design houses.

What is agentic verification?
If we boil down agentic AI’s role, it is quite simple. “Agentic verification is an orchestration of a flow, a verification flow by agents, and it’s implementation by the EDA tools and technologies,” says Abhi Kolpekwar, senior vice-president and general manager at Siemens EDA. “You cannot have one without the other. If you look at the life of a verification engineer, there are a lot of repetitive, mechanical tasks involved in executing the verification plan, and then doing coverage closure. The agents will be very smart about running an engine, collecting the data, analyzing that data, recommending the next steps, and things like that.”

But the next level down gets a little more complex. “In the past, people might have built TCL scripts and Python scripts that did this stuff,” says Ramesh Narayanaswamy, member of technical staff at Synopsys. “Those were more or less deterministic. With agentic solutions, you could have an orchestrator that can reason and adjust its path dynamically. You ask it to do a certain task based on context, and it can adapt, as opposed to a more rigid workflow.”

Consider an example. “Maybe I have a UVM error and ask, ‘Why did this happen?'” says Paul Graykowski, director of product marketing at Cadence. “It analyzes the error message, explains the error message, and then the tool automatically goes back and maybe figures out what signal was driving what and why that happened. The tool explores all of those angles in conjunction with the chatbot to be able to come back and formulate a response that says, ‘This signal went to this. It should have gone to this based on the spec saying this.’ There are all kinds of things that you could tie into it, where it could help you get really close to the error, or maybe the actual error itself, and even suggest the fix.”

A key piece of this is design understanding. “The first step for design verification is to understand the design, the functionality of the design, otherwise you cannot verify it,” says Hamid Shojaei, distinguished engineer at Cadence. “If you think about how verification engineers work, they spend a lot of time understanding the design. If you don’t do a good job over there, if an agent cannot understand the design in a reliable and consistent way, you can imagine that the quality of the test plan will be low. Test bench quality will be low.”

Pieces of this are beginning to come together. “Trying to understand the RISC-V spec is incredibly difficult,” says Dave Kelf, CEO for Breker Verification Systems. “If you look at a specific piece of functionality that you want to verify, get AI to tell you where it is in the spec. That was extremely successful and made figuring out the spec much easier. We realized that, especially in system verification, there appeared to be a need to provide a back-end for these AI tools, where the AI tools might be able to read a spec, generate a verification plan, and even a high-level graph, along the lines of portable stimulus (PSS). Tools can then do all the back-end work. Take that high-level spec and generate the test, especially in the system space.”

The important thing here is that verification will start to draw in a lot more information. “A coverage tool will need to build a mental model,” says Cadence’s Graykowski. “The mental model takes in all kinds of inputs — test code, your RTL, your specs, and then you work with it, talk to it, and you build this model. Based on that, it can generate test plans, it can generate coverage points, it can generate UVM code. It can do all those types of things. It comes at it from a different angle. Here’s my test plan. Here’s what I should be covering. Let me generate the test sequences to go after that. I’m not saying you will get 100% overnight by using these technologies, but there is a lot of automation that’s getting us closer.”

Everything has to become bi-directional. “When you get a failure, you need to back-annotate that to the test and the original area in the verification plan that it was testing,” says Breker’s Kelf. “That’s where the AI debugging parts of this flow come in. It needs to be able to back-annotate to the spec, to the verification plan. Then you can figure out what’s going on. You need to find all the places in the spec where the issues you’re finding were different from what the spec says, and that’s something AI is really good at. We can take that general back-annotation to this level and help with debugging like that.”

Once everything is semantically tied together, a lot of things can be optimized. “When you update the RTL, AI can understand the changes,” says Cadence’s Shojaei. “It will tell you which test needs to be updated, how they need to be updated, and they will do that for you. AI can also run the tool and get feedback, make sure the test is passing or failing, and even debug it for you. But you need key technologies to help, ones that can analyze code or waveforms. Without these, LLMs will guess, and there will be a lot of hallucinations. With the right analysis tools that help AI understand what changed — and based on that, start reasoning — they can come up with what needs to be updated.”

Still, AI can make mistakes. “Perhaps you have used AI to create a model from the specification,” says Shelly Henry, founder and CEO of Moores Lab AI. “It is when you run the test cases that you realize it has made a mistake. It did not have the context of the FSDB and log file and the data path when it created it. Now it has that context and it can go back and analyze everything — the RTL, the test case, the PSS — in the full context. Then it might realize that in the model, the original intent was not captured correctly, so it will suggest a fix.”

There are limitations, especially when it concerns analog. “If you look at the foundational models, which are used for the RTL stuff, they scraped it off the internet,” says Alexander Petr, senior director at Keysight EDA. “All of that knowledge is available on the internet. GitHub repositories will explain to you what the HDL is. There is documentation that talks about this. All the checks and verifications you have to do, they are all widely available on the Internet. But go and look for a high-class ball filter or power amplifier — that IP does not exist on the Internet. In addition, analog is not just about behavior or timing. When you see the problem, you may not know how to fix it because it’s a multi-domain, multi-physics problem.”

The gains and costs
Most of the gains today are more limited in scope. “The real gains of agentic verification are already clear,” says William Wang, CEO of ChipAgents. “It automates the lowest-leverage, most time-consuming work, writing test vectors, setting up UVM testbenches, triaging failures, and debugging, so engineers can focus on architecture and corner-case reasoning.”

But there could be so much more. “The value of AI assistants seems clear in guiding effective tool usage and best practices, but this is a more controlled domain than generative AI (GenAI) for verification,” says Stefan Birman, partner at AMIQ Consulting. “Users are experimenting with generation of tests and testbenches, but are concerned about accuracy, the apparently non-deterministic nature of AI, and the unpredictability of costs.”

In a contained example, the results could be amazing. “I was looking at an AXI-to-APB bridge,” says Moores Lab’s Henry. “It’s a small IP, but not a trivial one. Using our platform, I could complete the verification of that IP, writing all the test cases without even writing one line of code. Everything is AI-generated code, and I just guide it to do that. I could create all the test cases, complete test bench with monitors, drivers, scoreboards, reference model, everything implemented, find bugs in the design, fix those bugs. And I could look at the coverage, create exclusion files, close coverage, get all tests to pass in under 48 hours. I was blown away. In the traditional way, this is a two-month job. That is the capability that we are unlocking here with AI.”

But in some cases, these kinds of gains will only be seen after extensive in-house training — especially when analog is involved. “Here’s a tool that will allow you to do your job, but it’s not like our old tools, where you got a tool and it worked perfectly and it could do the job, and you got the results,” says Keysight’s Petr. “This solution is different. When you use it for the first time, it will be slower, most likely the quality and the outcome will be less good, and you have to revisit it. You do this 10 times. It gets better, but it’s still not where you want it to be. You do it 100 times. It gets better. You do it a million times, and at some point, it will switch and become better than you. It will take a long time to get to this point, and you need to be willing to walk that path. I don’t see that sales problem on the digital side, because they can clearly show that the solution they ship first time around is able to do the job.”

There have been stories about AI racking up enormous bills while doing some of these tasks. “In the software space, to a degree, the number of tokens you consume is almost like a badge of honor,” says Synopsys’ Narayanaswamy. “You need to give it budgets. Otherwise, it may run forever. You may say, ‘Try 10 times, and if you fail, tell me how you failed.’ The tradeoff to look at is if the loaded cost of this engineer is $300k or $400k, and if they were consuming $2k in tokens, which is a pretty high number, that’s a fraction of this person’s loaded cost. So long as you’re getting effective work out of it, it’s probably worth paying for, but it’s probably a budget surprise because you didn’t plan for it.”

All costs have to be accounted for. “We will have to do the budget for how many GPU cycles you need,” says Siemens’ Kolpeckwar. “The headwinds will be different. They will be non-human headwinds, more budget and cost. The path is really bumpy. There are legalities with IP exposure, with the LLMs’ reliability. There is an aspect of computational cost associated with this. It’s all over the place, but I’m very hopeful that agentic verification will cut down a lot more of that complexity. Consider the total cost of ownership. This is where you’re able to get the maximum out of your processes by applying the minimum of your hardware.”

AI infrastructure, inference, security, and data preparation all carry high costs. “The value proposition must be carefully measured,” says Andy Nightinghale, vice president of product management and marketing at Arteris. “The strongest business case comes when agents reduce expensive engineering iteration by shortening debug cycles, improving coverage closure, and identifying integration issues earlier. The important metric is not how much code or how many tests are generated, but whether verification teams can reach signoff confidence faster and with fewer silicon escapes.”

It comes down to the ROI. “While AI infrastructure has a cost, the ROI is overwhelmingly positive in the semiconductor context,” says ChipAgents’ Wang. “When companies, like Nvidia, are projecting trillion-dollar-scale data center revenues, even a single day of tapeout delay can translate into billions in opportunity cost, making acceleration disproportionately valuable.”

It is not a one-size-fits-all solution today. “Some things are just better done by yourself, and they’re quicker,” says Cadence’s Graykowski. “For other tasks an LLM can work wonders, but there is a tradeoff in how much time it takes to do its thing. If you ask a question with the context being an entire SoC, it may take hours for it to come back with an answer — and that’s if it even comes back with an answer, because there’s so much context. That’s the other problem, the models can only have so much context in their memory, so you have to look at ways to handle that and be more efficient.”

There also needs to be an attitude shift. “I have done some demos running our tool, and a customer may come back and say, one of the assertions was wrong,” says Cadence’s Shojaei. “You generated one hundred assertions and one of them is wrong. This shows that the engineer needs to be in the driving seat. You need to review the result, go step by step until you get the best result. If you just leave it with the agent, it is risky, and reviewing the final result will be more challenging if you don’t give feedback while the agent is working. Setting the right expectation is another very important point that chip designers should be aware of, and at this point, they should not trust AI 100%.”

[Editor’s note: Future articles will look at how these technologies are being used, who benefits the most, and the likely evolution of tools, methodologies, the EDA industry, as well as the impact on verification engineers.]

Related Articles
Creating Agentic EDA Methodologies
Current approaches involve multiple tools, vendors, designs, data formats, and abstractions. Can agents really use them all?
AI Growing Impact On Chip Design And EDA Tools
Demand for faster design and more automation grows from key customers.
Building AI Without Guardrails
The semiconductor ecosystem is wrestling with fragmented standards, IP exposure, and the urgent need for runtime assurance.
Using AI To Monitor Dashboards In Chips And Systems
AI agents can be used to identify potential issues during operation and react before it’s too late.
Designing Chips In The Context Of Rapidly Evolving AI
Long‑running agents, tool-calling LLMs, and multimodal chaos are rewriting edge compute rules, and making chip design more challenging.