Pointing Fingers In Verification

With most EDA tools, the buying decision is related to improved quality of results or increased productivity. Will a new synthesis or clock optimization tool enable designers to do more, faster and are those gains worth the price? The equation is fairly simple.

When it comes to verification tools, things are more complex. You can still make productivity gains, or purchase an additional tool that will find more or different errors, but it cannot ensure bug-free silicon. All it can do is reduce the risk of a respin. Can you do too much verification? If it is not the right type of verification, then the efforts are wasted, and there are several indications that the verification process in use today is very wasteful.

Janick Bergeron, a Synopsys fellow, likens verification to an insurance policy: “You hope you never have to use it.”

Bill Grundmann, a Xilinx fellow, sees verification as a career for the paranoid. “You never want to throw anything away that found a bug in the past. There is nothing which says you are done. What is the metric for good enough?”

Jim Caravella, vice president of engineering at NXP Semiconductor, takes a more pragmatic view. “Every chip has bugs, but does it really impact the system or the customer?” Thus a discussion started about who created or is responsible for the often talked about verification gap, and not all see the issue in quite the same way.

The verification gap postulates that verification complexity increases faster than design complexity as the size of the design increases. In addition, it normally indicates that the productivity increases in verification tools and methodologies are not keeping up with the increased demands being placed on them. This means that teams have an increased risk of chip failure unless they continue to pour increasing amounts of resources into closing the gap.

Caravella reminds us that, “with infinite resources and time, it still does not mean your system is bug free.” Grundmann adds that building larger teams can make the problem worse because “with such large teams there is a problem with information handoff between people.”

Mike Stellfox, a Cadence fellow, agrees and broadens the scope to include software whose inclusion is necessary to ensure use cases are adequately covered in verification. “You need early and often communications between the teams.”

Tools or users?
Is the problem associated with the tools? The discussion turned to pointing the finger at the users. Bergeron claims “we have virtual prototypes but the industry does not have the discipline to use them.” Harry Foster, chief scientist for verification at Mentor Graphics, believes that “we rely on superstars too much.” These are the few people within each group that manage to perform magic and avert crisis.

The lack of a single verification tools adds to verification complexity. Stellfox says that “successful companies manage to integrate many flows together.” Bergeron adds that “you have to pick the right tools at the right time. There is no one right tool for all tasks. Coverage is the thing that brings them all together.”

Grundmann does not see tools as the total solution. “People are too willing to jump to a tool rather than do a full assessment of the situation.” Stellfox adds that capturing a plan upfront is the key to being able to successfully adopt tools and that many people use existing tools incorrectly. “Many people adopt UVM and then do directed testing,” he says.

Caravella accuses the industry of artificially creating a gap and then finding a solution to fill it. Other users were quick to lash back at the EDA industry. A Qualcomm representative in the audience asked why there wasn’t a good effort being directed towards SoC verification. Stellfox responded, “UVM was targeted at IP and bottom-up verification. In today’s SoCs there is not a lot of new IP and we need new flows that are optimized around these.”

Bergeron points out that “a standard shows that we are getting maturity. It is too early for SoC.” Qualcomm strikes back claiming that the SoC verification problem has existed for a long time and that EDA companies have been slow to address the need. Foster responds, “we acknowledge the need and we are working on it.”

Mentor has recently made a proposal for a new standardization effort for graph-based verification directed towards SoC verification. Stellfox agrees and says that Cadence has a significant focus on the development of tools, flows and methodologies in this area.

John Swan, an SoC technologist, asked how can we move more verification forward, such as using virtual prototypes. While the panelists said that leading-edge customers are doing this today, they had little insight as to how to move this into the mainstream. Foster believes HLS may be the key, while Bergeron counters that RTL design is not painful enough to make people look at this solution.

Caravella brought a sense of clarity back to the situation saying, “It comes down to ROI. If the gap is not big enough then solutions will not emerge.” Bergeron, still thinking in terms of moving the design abstraction up, says “making a change is a shock to the system. The last time was painful and we had to sacrifice a generation of designers.” He was referring to the migration from the gate level to RTL.

Another Qualcomm representative said “EDA created the gap. We needed a horse and got a camel. How will you prevent it from happening today? UVM was available 10 years ago and you have not made much progress.”

“Yes, there was a view that SystemVerilog was the answer to everything and we spent too much time and effort on the standards effort,” said Stellfox. “But this was driven by customers. Standards provide interoperability but they also stop innovation. Standards make things 100 times slower. We need more innovation in the SoC space rather than a standards play.”

Caravella pilled it on, adding “EDA guys should listen to customers and not their marketing teams. Stop recreating three letter acronyms.”

Will the industry learn from its mistakes or will the development and standardization of solutions that are focused on SoC verification suffer from the same problems? The only way to ensure that requirements are met is for all of the stakeholders to participate in the standards meetings. Without this first step, the industry may again go off course and everyone will pay the price.