Reduction In First Silicon Success

Every two years, Harry Foster, chief scientist for Mentor, a Siemens Business, works with Wilson Research to do a verification study. Those studies have influenced many in the industry, indicating where users are experiencing the most pain, spending their time, growing their team sizes and where money would be best spent. However, over the past four years, the ASIC industry has basically been in a no change pattern.

The study was 31% smaller this year and it was pointed out that there had been a growth in responses coming from smaller design sizes. While this could have skewed some results, most responses were essentially flat and within the margin of error.

Foster did point a few significant differences in the results. First was the first silicon success rate. “The number of spins is important because it can be used as a proxy to determine the effectiveness of our verification practices,” said Foster. “It turns out that it has been fairly consistent across the past 15 years where around 30% of all project were able to achieve first silicon success. This changed significantly in 2018 where only 26% of design projects were able to achieve first silicon success.”

This result may very well be influenced by the number of smaller designs which potentially were less sophisticated in their usage of verification methodologies. Then Foster mentioned the biggest change: “There are no corresponding metrics for FPGAs, making it difficult to ascertain their verification effectiveness. However, we can look at the number of bugs that escape into production. In 2018, only 16% of FPGA projects were able to achieve no bug escapes into production, which is worse than the decline in ASIC first silicon success.”

Another interesting piece of data related to the nodes being used. There is a lot of talk in the industry about companies resisting a movement to the newer nodes, but it would appear that the data does not seem to back this up.

Over the past two years a significant number of designs have migrated from 28nm to 14nm, and 7nm is gaining traction. Given the increased number of smaller designs in the survey, it is somewhat surprising that none of the older nodes showed growth and would indicate that smaller designs are also favoring the newer production nodes.

Some results are interested in showing no change. For example, the number of processors in a design has not grown significantly, although the results may not show what is happening at the extreme end. This is one area where FPGA designs are catching up quickly. The number of clock domains remains fairly constant and the number of designs that actively manage power is flat.

One significant change is that more designs are being done that are considered safety critical. That increased from 46% to 59% in 2 years. The biggest categories for this are automotive and industrial. There was a much smaller increase (2%) in concerns about security – a rather troubling response.

Back in 2014, the industry was shocked by the growth rate of the verification engineer to design engineer ratio. Thankfully, that has leveled out to approximately a 1:1 ratio. The number of verification engineers in an FPGA team continues to grow fairly modestly and is approaching the 1:1 ratio, and Foster believes this is following the ASIC trend related to rising complexity.

Debug continues to be the largest consumer of time for both ASIC and FPGA designs. “For management this can be a significant challenge when planning because debug is unpredictable and can vary significantly between projects,” warns Foster.

A curious trend is that typical regression time is showing a decrease. Foster suggests that this could be due to partitioning regressions across multiple servers, the adoption of emulation or even FPGA prototyping.

Very little change was seen in the adoption of languages, libraries or methodologies for ASIC design except for a migration to newer versions of the same standard.

Within the FPGA community, there are some significant signs related to the benefits of adopting current ASIC verification practices. “There is statistical significance between the group with no bug escapes that tended to have a higher adoption of various verification techniques. However, we are unable to asses the effectiveness of any of their methodologies. For example, some may claim to use functional coverage but may do so very poorly due to their level of inexperience with it.”

It also comes with a warning. “One might expect that projects working on safety critical design would yield better results in terms of bug escapes. However, that is not true.”

Foster suggests that the industry has been getting better at finding functional and clock bugs but is seeing more mixed-signal and timing bugs.

The biggest take-away is that FPGA design is approaching similar complexity levels compared to ASICs and are rapidly adopting many of the same verification techniques. The one big change for ASIC teams is that they are having increasing problems creating sufficient tests to verify their designs. This is not a problem for FPGA designs yet.