The Assertion Conundrum

It is well documented and widely agreed that assertions can provide a tremendous benefit to design and verification teams by reducing and even eliminating debug – but their use is still not ubiquitous.

Part of the reason is that assertions cannot be picked up casually, noted David Larson, director of verification at Synapse Design. “This is because assertion syntax is not straightforward and assertions behave very differently than behavioral code. Behavioral code comprises of a series of instructions that are executed one after the other, while assertions contain no instructions at all but instead specify intended signal behavior with its own set of symbols. They are also active all the time.”

Further, he said, assertions are hard to learn, hard to write and hard to debug. They are certainly valuable and worth learning, but most engineers are not willing to invest the time to learn something so esoteric. Within the SystemVerilog family of features, assertions are the ugly cousin.

Hemendra Talesara, verification technologist at Synapse Design pointed out that while the use of is increasing, they still find many projects where they are not used at all. “This is especially distressing when one realizes the immense productivity gain by using them. Consider that debugging is the number one bottleneck in all verification tasks—assertions can completely eliminate debug and diagnosis time in many circumstances. They are like a watchdog looking for any non-compliance of behavior by the design. They flag the problem right at its source and they catch problems that otherwise would have been missed. Assertions help with checks during integration of a block and at next level of hierarchy there by minimizing the need for integration support. Written in declarative style, assertions provide additional documentation. Designer’s time and effort savings are many times more than it takes to write them.”

So, why doesn’t everyone use them?

He suggested a couple of simple technical reasons. “For anyone who has not used assertions in the past, the syntax and application may appear to be complex. Secondly, they do not always know where to start. While it’s possible to write more complex assertions and checkers for complex protocols, just getting started with simple assertions alone can make one’s life very easy.”

Assertions are part of SystemVerilog language, Talesara said, which means that no additional tools are required for using assertions in simulation-based verification. Some training is required. However, once an engineer is familiar with writing assertions, the process is no more complex than writing any other code. For many simple assertions, it could be as easy as writing a comment in the code. One can also develop a library of simple assertions to be used as part of methodology. Most vendor tools come with a library of pre-defined assertions requiring that the engineer only learn how to instantiate it.

Just to demonstrate its simplicity, he offered two examples of assertions. The first one captures a very simple behavior while the second one is just a slightly more complex. “Once you get used to symbols and syntax, it’s quite easy to write. It’s declarative and closer to specification, and takes very little code to express it than if you have to write it in Verilog.”

“An engineer’s reason for avoiding using assertions is generally a lack of familiarity and/or time to learn how to create and use assertions. Management’s reason for not insisting on it is that they have not fully grasped the enormous cost-benefit. Teams need to understand that getting started is easy. Pareto’s Principle (80/20 rule) applies. One need not go overboard trying to implement complex assertions. Simple assertions with simple guidelines are easy to learn, easy to apply and have big returns,” Talesara added.

The evolution of assertions
Pete Hardee, director of product management at Cadence, recalled that when assertions were first conceived and started to get introduced – first PSL and later SystemVerilog assertions – the idea was that designers would use them. “Designers would write assertions because they have the advantage of self-documenting the design and it would be an easy way for designers to effectively complete the unit tests on their blocks before handing them off for integration and further verification. That’s the piece that a lot of people struggled with because that use model didn’t really catch on.”

“Then you’ve got the verification team that took the lead for the creation of assertions,” he continued. “That’s when you started to see some formal verification experts appear and most of the rest of the verification engineers moving on from directed test to constrained random but basically sticking with testbenches and simulation.”

Propelled by Cadence, verification apps began coming to the floor, Hardee said, which are “all about how to automate verification for a particular known and well-characterized verification problem. That became applied to things like connectivity verification. Top-level connectivity is tedious to do with simulation and there you can easily write assertions; the assertions can be created automatically for you in the verification app from a high-level connectivity spec. That’s the kind of way that we’ve been applying formal verification to get around this problem that people really don’t like writing assertions, or it’s only some subset of experts who will really write a lot of assertions.”

Michael Sanie, senior director of verification marketing at Synopsys takes issue with the argument used against assertions that they are difficult. “A lot of the difficulty has gone away. It is a standard part of SystemVerilog. It’s not that difficult, technically speaking. It would be more difficult to write a property for a formal check, which is the same thing but in a different way. If you define a property for a model-checking tool, the formal tool can then debug, and find the bugs. You would have to do the same effort but maybe a lot more if you write a model or a property rather than an assertion.”

He observed that many engineering managers want their teams to write assertions. “They want to allocate some time and take some effort to develop assertions because it relieves pressure later on. But if you zone it down to the guy developing the RTL, it’s not his favorite thing to do. It’s as simple as that…Assertion writing is like taking vitamins — it doesn’t taste good, but it’s not that bad. You can still swallow it. But it really takes care of a lot of headaches later. It’s a good practice.”

Sanie noted that while there is technology to give ideas of what the assertions would be, the problem with those technologies is that it gives a whole bunch of ideas but it gives a lot of false positives. “There are some technologies — we’re working on some — that brings some formal knowledge. It tries to understand the design better so when it provides those candidates [for assertions] they are all good. For the most part right now there is nothing people can use to really help them, and the biggest problem is that it’s post-factum. You can’t take that away. If you can do it while you’re doing the design it’s better.”

To really move things forward, he stressed, there will be a combination of technology and methodology, which is taken very lightly. “The way that assertions play into and improve formal verification — there’s a lot of opportunity to improve the design through that methodology. The right methodology not only has to enable and make it easier for designers to write assertions but also it has to provide formal technology to be used as early as possible by designers because it’s the same kind of thinking. It’s different tool usage. One is about writing assertions, one is about using them, but the idea of having a methodology forces design teams to start looking for bugs as early as possible. That would be something that would really change the way people do verification. In fact, that is happening as we speak. People are realizing they need to do more formal. Assertions may still be a pain, but they still at least focus on it, they write it, they have code reviews and do more static and formal checking up front because the whole point is to catch these bugs as early as possible. The earlier you find a bug and fix it, the cheaper it is.”

Harry Foster, chief verification scientist at Mentor Graphics has his own take on why so many engineers may be hesitant to embrace assertions.

“There’s still skepticism in terms of the ROI,” he said. “A lot of people perceive there is a lot of effort but haven’t tried it. Those who try it often do it an ad hoc manner and yield very few results.”

The second reason is that there’s a lack of understanding of the process. “Who is the stakeholder? Who writes the assertions? Are they accounted for in the test plan or not? These types of basic things people don’t know because if you think about it, EDA companies typically promote tools and languages. They don’t promote the process and a lot of people are confused about how to do this,” Foster continued.

The third aspect of assertion trepidation, he said, is that a lot of organizations lack building skills necessary to take advantage of it. “This is fundamental because it goes beyond teaching someone an assertion language: it goes to where do you effectively put the assertions, etc. There have been multiple peer-reviewed industry studies — not EDA — at respected conferences that found teams that added assertions to the design actually had less bugs. And that is significant because if you think about it, anything you can do to prevent bugs versus finding them improves productivity.”

This goes back to a lot of teams lack the investment — going back to the fact that they are unclear about the ROI so they don’t build the necessary skills within the organization.

“Assertions make the engineer think about the design in a different way than the way they actually implement the design and that thought process uncovers bugs that they wouldn’t think about normally,” he added.

Coincidentally, Mentor Graphics released a tool this week that Foster said automatically generates assertions by analyzing simulation traces for those who don’t want to write them.

Easing the path to assertion adoption
Barring a universal mandate from management, isn’t there anything that would make adopting/learning to write assertions easier?

Synapse’s Larson pointed out that one of the difficulties for adopting assertions is that verification can be done without assertions. “Procedural code can do anything assertions can do, although with less elegance and power, so few teams are seeing the need to rush into joining the assertion bandwagon.”

Also, he said, assertions have the same hurdle to cross that constrained-random, coverage-driven verification had to cross, and that is an industry-wide awakening to how they can be used to reach the finishing line of full verification much more quickly and with less effort.

Furthermore, it will be easier for assertions to be adopted if the difficulties of using them are addressed. One approach is to make them easier to understand and debug. A powerful assertion debugger can go a long way to address that difficulty. Cadence, Mentor Graphics and Synopsys all have assertion debuggers in their debugging tools, Larson concluded.

Additional Resources
Free courses on assertion-based verification and other topics.