Trends In FPGA Verification Effort And Technology Adoption

The more we know about the bigger picture, context, historical and projected trends, or simply how other people do the same thing we do, the more efficiently and successfully we can do our specific jobs. This perspective also informs the EDA industry in how to best assist and sustain the needs of the FPGA and ASIC engineering communities.

Providing this kind of information is the reason we conduct the worldwide Wilson Research Group Functional Verification Study every two years. A full report on our findings and analysis of the data are contained in two whitepapers presented by Siemens EDA, a part of Siemens Digital Industries Software. The highlights of these papers are presented in this publication as part of a four-part series on the Wilson Research Group 2020 study.

FPGAs have recently grown in complexity equal to many of today’s IC/ASIC designs. In this unbiased survey we quantify the impact of this growing complexity in terms of verification effectiveness and effort. In our first article, Verification Effectiveness in the Face of FPGA Complexity, we shared our findings on FPGA verification effectiveness. In this article, we will look deeper into FPGA trends pertaining to verification effort and verification technology adoption. We will close with some of our observations regarding various aspects of the FPGA market before turning to the ASIC market in the third and fourth articles.

Verification effort is high

There is no simple answer to the question, “How much effort was spent on verification in your last project?” To try to assess the effort spent in verification, let’s begin by looking at one data point: the total project time spent in verification.

Figure 1 shows the percentage of total FPGA project time spent in verification. You can see two extremes in this graph. In general, teams that spend very little time in verification are typically working on designs with a good deal of existing pre-verified design IP, which is integrated to create a new product. On the other extreme, projects that spend a significant amount of time in verification often have a high percentage of newly developed design IP that must be verified.

Fig. 1: Percentage of FPGA project time spent in verification.

Overall, we found an increase in the average percentage of FPGA project time spent in verification during the period 2014 through 2020. This is an indicator of growing design and verification complexity.

Perhaps one of the biggest challenges today is to control cost and engineering headcount, which means identifying FPGA design and verification solutions that increase productivity. To illustrate the need for productivity improvement, we discuss the trend in terms of increasing engineering headcount. Figure 2 shows the mean peak number of FPGA engineers working on a project.

Fig. 2: Mean peak number of FPGA engineers working on a project.

While, on average, the demand for FPGA design engineers grew at about a 1.5% CAGR between 2012 and 2020, the demand for FPGA verification engineers grew at a 5.5% CAGR. It is worth noting that during the period 2007 through 2014, the IC/ASIC market went through similar growth demands related to verification engineers to address growing verification complexity.

But verification engineers are not the only project stakeholders involved in the verification process. Design engineers spend a significant amount of their time in verification too, as shown in figure 3.

Fig. 3: Where FPGA design engineers spend their time.

In 2020, design engineers spent on average 53% of their time involved in design activities and 47% of their time in verification. However, when compared to 2014 and 2016, the data indicate a trend showing that FPGA design engineers are now spending slightly less time involved in verification tasks. There are two reasons for this trend. First, many FPGA projects have added verification engineers to their teams, which means design engineers can focus most of their effort on design. Second, in general, there has been increased adoption of larger, more complex FPGAs, which has increased the design engineer’s workload.

Figure 4 shows where verification engineers spend their time (on average). In general, there have been no statistically significant changes in the FPGA results during the period 2014 through 2020.

Fig. 4: Where FPGA verification engineers spend their time.

Our study found that FPGA verification engineers spend more of their time debugging than with any other activity. From a management perspective, this can be a significant challenge when planning future projects’ effort and schedule based on previous projects’ data since debugging is unpredictable and varies significantly between projects. The key takeaway here is that a significant amount of engineering time is spent in the verification process when you consider the combined design and verification engineering time.

FPGA verification technology adoption

An interesting trend we see in the FPGA space is a continual maturing of its functional verification processes. In fact, we find that the FPGA design space is about where the ASIC/IC design space was about seven years ago in terms of pre-lab verification maturity—and it is catching up quickly.

To address growing verification complexity, we find that many FPGA projects are starting to mature their pre-lab functional verification processes. In this section, we present FPGA trends related to the adoption of various verification techniques, which are standard practice today on most IC/ASIC projects.

Figure 5 shows the FPGA project adoption trends for various simulation-based techniques from 2012 through 2020, which include code coverage, functional coverage, assertions, and constrained-random simulation. Notice the increased adoption of these various techniques throughout this period.

Fig. 5: Simulation-based technique adoption trends for FPGA designs.

The adoption trends for formal property checking (e.g., model checking) and automatic formal applications are shown in figure 6. We found that the adoption of formal property checking on FPGA projects is growing at an impressive 21% CAGR, and the adoption of automatic formal applications is growing at a 29% CAGR. Historically, the formal property checking process has required specialized skills and expertise. However, the recent emergence of automatic formal applications provides narrowly focused solutions and does not require specialized skills for adoption. In general, formal solutions (i.e., formal property checking combined with automatic formal applications) are one of the fastest growing segments in functional verification.

Fig. 6: FPGA formal technology adoption.

The key observation from these adoption trends is that the FPGA market has matured its verification processes, as I previously stated. This maturity is likely due to the growing complexity of designs.

FPGA language and library trends

It is not uncommon for FPGA projects to use multiple languages when constructing their RTL and testbenches. This practice is often due to legacy code as well as purchased IP. Hence, you might note that the percentage adoption for some of the languages that I present sums to more than 100%.

Figure 7 shows the trends in terms of languages used to create FPGA RTL designs, comparing the 2012, 2016, and 2020 Wilson Research Group studies, as well as the projected design language adoption trends within the next twelve months.

Fig. 7: Trends in languages used for FPGA design.

In figure 8, we show the adoption trends for languages to build testbenches.

Fig. 8: Trends in languages used in verification to create FPGA simulation testbenches.

Historically, VHDL was the predominant language used for FPGA testbench development, but we have recently seen increasing interest in SystemVerilog adoption. Today, it is not unusual to find that the RTL design was created using VHDL, and the testbench was created using SystemVerilog.

The adoption trends for various base-class library and methodology standards are shown in figure 9, and we found that the Accellera UVM is currently the predominant standard that has been adopted to create FPGA testbenches worldwide. For our 2020 study, we tracked Python-based methodologies, such as cocotb, for the first time.

Fig. 9: FPGA methodology and class library adoption trends.

Finally, FPGA project adoption trends for various assertion language standards are shown in figure 10, where SystemVerilog Assertions (SVA) is the predominant assertion language in use today. Similar to languages used to build testbenches, it is not unusual to find FPGA projects create their RTL in VHDL and then create their assertions using SVA.

Fig. 10: Trends in assertion language and library adoption for FPGA designs.

Now that we’ve covered trends in FPGA functional verification trends, we will turn to ASIC/IC market trends in two articles appearing in the coming months. If you’re ready to dive deeper into the survey results right now, check out the full paper: 2020 Wilson Research Group Functional Verification Study FPGA Functional Verification Trend Report.