More Intelligent Verification

By Jon McDonald
If I have flipped a coin 1,000 times and it’s come up heads every time what are the odds that I will get another head on the next flip? The odds on the next flip are still 50/50. The fact that I observe a uniquely unlikely situation has no impact on my current probability, and frankly any sequence of prior results would be equally unlikely. I find some of the same thinking applies when discussing unit-level verification relative to system-level verification.

I recently read a piece by Russ Klein, on finding a hardware problem through software execution. The problem was astronomically unlikely to occur when looking at the hardware without the system context. This resonated with a number of conversations I’ve had with customers recently. When you look at the state space of an element, how do you direct the verification process without knowing how the element is going to be used in the system? Looking at the system, what do I define as a working system? A system may be completely acceptable with a set of elements that may not be acceptable when looking at individual unit level tests, but the way the elements are used in the system results in an acceptable even efficient system.

The point of system-level design is understanding the function and performance of the system. This may be a bit simplistic, but I believe it is important to understand the function of the system in a mode of operation as close to the end system use case as possible, before we worry about exhaustive verification of the system, or the elements of the system.

The state space for most modern complex systems is enormous. Very few designs can be exhaustively verified. The best starting point for testing is to use the system in the way it is intended to be used. The state space may be huge, the odds of being in any given state minuscule, but the application of the system function will drive the system through specific state sequences. The fact that an application drives the system through a sequence of incredibly unlikely states is not really unlikely at all. Most individual system states are unlikely, but the operation of the system must go through some sequence of states to function. When we look at the states in context of the operation of the system, being in a particular state is unavoidable. Looking at the states without the system context makes testing the appropriate states very unlikely.

I think hardware verification at times focuses too much on the unit-level perspective and not enough on the system level context. By understanding the system’s context, the function and performance requirements at the electronic system level, we can appropriately direct the verification for the individual units.

–Jon McDonald is a technical marketing engineer for the design and creation business unit at Mentor Graphics.