ASIC/IC Verification Trends With A Focus On Factors Of Silicon Success

At long last we come to the final installment of our four-part series presenting the findings of the Wilson Research Group Functional Verification 2020 study.

In this article we discuss verification trends in IC/ASIC language and library adoption, low power management, and verification effectiveness. We then take a deeper dive into two somewhat surprising phenomena revealed in the data: the relationship between verification maturity and non-trivial bug escapes into production and the effect of safety critical design practices on the level of silicon success. Finally, we conclude with a summary of the study and key findings.

Language and library adoption

First let’s look at language and library adoption trends. Figure 1 shows the aggregated adoption trends for languages used to create RTL designs across all market segments and all regions of the world. We see continual interest in SystemVerilog for RTL creation.

Fig. 1: IC/ASIC languages used for RTL design.

Figure 2 shows the adoption trends for languages used to build testbenches. It is not uncommon for IC/ASIC projects to use multiple languages when constructing their testbenches. We continue to see an increase in adoption of C/C++ for testbench development. In addition, we show the adoption levels for the Accellera Portable Test and Stimulus Standard (PSS). Finally, in 2020, for the first time, we explicitly asked about the adoption of Python for testbench development.

Fig. 2: IC/ASIC verification languages used for testbenches.

Figure 3 shows the adoption trends for various base-class library and methodology standards. We found that the Accellera UVM is currently the predominant standard that has been adopted to create IC/ASIC testbenches, and it continues to grow.

Fig. 3: IC/ASIC methodologies and testbench base-class libraries.

Figure 4 shows IC/ASIC project adoption trends for various assertion language standards. SystemVerilog Assertions (SVA) is the predominant assertion language in use today. It is not unusual to find IC/ ASIC projects create their RTL in VHDL and then create their assertions using SVA.

Fig. 4: IC/ASIC assertion language adoption.

Low power management

Now we will turn to trends in IC/ASIC design and verification for low power devices. In figure 5, we see the percentage of design projects that actively manage power according to design size. The data suggest that the larger the design, the greater the concern for power management. Obviously, a wide variety of techniques, ranging from simple clock-gating to complex hypervisor/OS-controlled power management schemes, are employed and require verification.

Fig. 5: ASIC/IC projects working on designs that actively manage power.

Figure 6 shows the various aspects of power-management that design teams who actively manage power must verify. The data from our study suggest that many projects, since 2012, have moved to more complex power-management schemes that involve software control (e.g., hypervisor/OS control and application-level power management). This adds a new layer of complexity to a project’s verification challenges, since these more complex power management schemes often require emulation for complete verification.

Fig. 6: Aspects of power-managed design verified.

Since the power intent cannot be directly described in an RTL model, alternative supporting notations have emerged to capture power intent. Figure 7 shows the various standards used to describe power intent that have been adopted. You might note that the that the newer UPF 3.0 standard was tracked in 2020. Also, some projects are actively using multiple standards (such as different versions of UPF or a combination of CPF and UPF).

Fig. 7: Notation used to describe power intent.

Effectiveness of ASIC/IC verification

A significant amount of effort is applied to IC/ASIC functional verification. An important question our series of studies have tried to answer is whether this increasing effort is paying off. To help answer that question, let’s look at the survey findings in terms of schedules, number of required spins, and classification of functional bugs.

A metric often used to measure efficiency is ASIC/IC project completion compared to the original schedule, as shown in figure 8. Here we found that 68% of IC/ASIC projects were behind schedule.

Fig. 8: Design completion compared to original schedule.

Another trend worth examining is the number of spins before production. Figure 9 presents industry trends from 2012 through 2020 in terms of required spins before ASIC/IC production. Even though designs have increased in complexity during this period, the data suggest that the number of required spins before production has not increased. Nonetheless, only about 32% of today’s projects achieve first silicon success.

Fig. 9: Required number of spins.

Figure 10 shows various categories of flaws contributing to IC/ASIC respins, where the percentage of “logic or functional flaws” remains the leading cause of bugs. In 2020 we began tracking flaws associated with safety (11%) and security (10%) features. Obviously multiple flaws can contribute to bug escapes.

Fig. 10: Types of flaws resulting in respins.

In 2020 we saw a huge increase in flaws attributed to tuning analog circuits (41%). I analyzed this year’s results to see whether the spike was restricted to designs at the latest technology nodes, or if it was more widespread. It turns out that while 7nm or below was the most popular answer for those experiencing the problem, it only accounted for about 16% of the cases. Almost all nodes, including 150nm and larger, were seeing these types of failures.

Figure 11 clearly shows that while all design sizes are experiencing a rise in problems associated with analog tuning, the biggest percentage involves the largest designs.

Fig. 11: Analog flaws by design size.

Figure 12 demonstrates the root cause of logical or functional flaws by various categories. Design errors remain the leading cause of functional flaws, and the situation is worsening. In addition, problems associated with changing, incorrect, and incomplete specifications are a common theme often voiced by many verification engineers and project managers.

Fig. 12: Root cause of functional flaws.

Non-trivial bug escapes and safety critical designs

We will conclude the series by having a little fun with some of the findings from our study with a deeper dive into the following:

• Verification maturity and non-trivial bug escapes into production
• Safety critical designs and silicon success

The study results show that the IC/ASIC market has matured its verification processes over time to address growing complexity, predominately driven by the emergence of SoC-class designs in the mid-2000 timeframe. Today we find the FPGA market is maturing its verification processes. This maturity is likely due to the growing complexity of designs and related efforts to control cost and engineering headcount through the adoption of FPGA design and verification solutions that increase productivity.

Perhaps the most concerning finding from this year’s study relates to the number of FPGA projects with non-trivial bug escapes into production. However, we did find an interesting correlation between the improvement of reduced functional flaws contributing to non-trivial bug escapes and the maturing of FPGA projects’ functional verification processes. The data suggest that projects that are more mature in their functional verification processes will likely experience fewer bug escapes.

To test this claim, we partitioned the study participants into two separate groups: FPGA projects with no bug escapes and FPGA projects that experienced a bug escape. We then examined the percentage adoption of various verification techniques. The results are shown in figure 13.

Fig. 13: 2020 FPGA projects with non-trivial bug escapes into production.

We then examined the percentage adoption of various verification techniques. The results are shown in figure 14.

Fig. 14: Process maturity and non-trivial bug escapes.

These findings are statistically significant in that the group with no bug escapes tended to have a higher adoption of various verification techniques, which suggest they are more mature in their verification process. However, from our study data, we are unable to assess successful or effective adoption for any particular verification technique.

Now let’s turn to safety critical designs and their silicon success. Intuitively, one might think that the rigid and structured process required to adhere to one of the safety critical development processes (such as DO-254 for mil/aero, ISO 26262 for automotive, IEC 60601 for medical, etc.) would yield higher quality in terms of preventing bugs and achieving silicon success.

We looked at the FPGA projects working on safety critical designs, and then examined the percentage of bug escapes and no bug escapes for projects working on safety critical designs, as shown in figure 15.

Fig. 15: Non-trivial bug escapes for safety critical vs. non-safety critical FPGA designs.

Clearly, the data suggest that a development process adopted to ensure safety does not necessarily ensure quality. Perhaps this is non-intuitive. However, to be fair, many of the safety critical features implemented in today’s designs are quite complex and increase the verification burden.

2020 study summary and key findings

The FPGA market is going through complexity growing pains like those the IC/ASIC market experienced in the early and mid-2000 timeframe. Like the IC/ASIC market during that timeframe, the FPGA market today has been forced to mature its functional verification processes. This was easily measured in our study by examining the increased adoption of functional verification technology, as well as growth in the number of verification engineers as FPGA team members. Perhaps the most disturbing finding from this year’s study relates to the number of FPGA projects with nontrivial bug escapes into production. We did find an interesting correlation between the improvement of reduced functional flaws contributing to non-trivial bug escapes and the maturing of FPGA projects’ functional verification processes.

For the IC/ASIC market, our study found continued maturing of the verification process. In particular, the continued increased adoption of formal technologies is encouraging. Also for the IC/ASIC market, we were surprised by the huge increase in flaws attributed to tuning analog circuits. One explanation to this issue is the recent increase integration of analog across the board in terms of various sized designs, and this is contributing to an increase in analog issues.

This concludes the findings from the 2020 Wilson Research Group Study. A big part of the motivation behind these bi-annual studies is to increase our community’s understanding of our vital industry so that we, and you, can make better decisions about how to go about our business. We hope we have accomplished this goal.

To learn even more, please read the whitepapers from Siemens EDA, a part of Siemens Digital Industries Software: 2020 Wilson Research Group Functional Verification Study – FPGA Functional Verification Trend Report and 2020 Wilson Research Group Functional Verification Study – IC/ASIC Functional Verification Trend Report.

Source for all images: Wilson Research Group and Mentor, A Siemens Business, 2020 Functional Verification Study © Mentor Graphics Corporation