Intelligent Coverage Optimization: Verification Closure In Hyperdrive

Coverage dominates every aspect of verification for today’s complex IP and chip designs. Coverage metrics provide critical feedback on what has been verified and what has not, especially when automated stimulus generation techniques are used. All modern hardware design and verification languages include constructs for functional coverage specification and support a range of structural coverage measures (line, block, expression, etc.) as well. Coverage closure and bug rates play key roles in project management deciding when to tape out a design. It is unacceptably risky to commit to silicon before achieved coverage reaches 100% of the target goals, or at least asymptotically converges toward those goals.

Coverage specification is now built into verification plans as well as languages. Engineers no longer painstakingly define and hand-code long lists of tests for specific features. Instead, they specify the coverage properties that show when these features have been exercised. Planning is the first phase of the typical SoC verification process. The second is execution, in which simulation using constrained random verification (CRV) is the primary method of generating stimulus and checking results, and the reported coverage documents exactly which parts of the design have and have not been verified.

The verification team then analyzes the coverage results and tweaks constraints to steer CRV toward parts of the design not yet exercised and make progress toward closure in the final phase. It often takes multiple attempts for constraint refinements to be successful, and the process can be painful. Once coverage has plateaued at less than the goals, it is unlikely that simply running more of the same simulations will improve it. At this point, the verification engineers may revert to hand-writing tests to hit specific corner cases, further prolonging the process. The traditional CRV approach has worked well for a couple of decades, but it is not keeping pace with the latest generation of massive and complex chips.

Fig. 1: The four phases of SoC verification.

In response, the industry has been exploring the use of artificial intelligence (AI) and machine learning (ML) techniques to further automate CRV, accelerate coverage closure, and expose corner-case scenarios. Synopsys has moved AI/ML beyond the research phase and into actual products with VCS Intelligent Coverage Optimization (ICO). This innovative technology maximizes project resource utilization while reducing the verification schedule and fits seamlessly in the user’s verification environment. From the user standpoint, little changes in the process. The verification team provides the testbench, constraints, and coverage along with the design. ICO adds a layer of ML-based analysis between the user inputs and the tests executing in simulation in a compute farm, grid, or cloud environment.

At its heart, ICO learns what CRV tests are most effective at improving coverage most quickly and focuses on these. It learns across different randomizations within a single simulation and across many simulations. It uses AI/ML techniques to efficiently optimize constrained random stimulus generation to achieve better diversification and improved stimulus quality. It sets up CRV simulation runs that are more likely to hit unreached coverage and rare scenarios with fewer tests more quickly, significantly reducing coverage closure time and effort. ICO provides advantages at every step of the coverage convergence process.

In the first stage of planning and execution of CRV tests, predictive testbench visibility provides guidance to improve the coverage results. The ICO analysis reports provide insights such as distribution of stimulus, potential over-constraints, diversity measures, and root causes of failures. This yields better and faster initial coverage results due to better stimulus quality, more exposed bugs, and reduced debugging effort through root cause analysis. In the second stage, manual coverage analysis is replaced by adaptive input stimulus biasing, significantly shortening the project schedule. In addition to greatly reducing verification team effort, equivalent or better coverage results are achieved with fewer tests, saving expensive grid or cloud compute resources.

In the final stage of closure, faced with hard-to-hit coverage, ICO provides insight into over-constraint issues that may be hindering achievement of the tape out coverage goal. It performs root cause analysis for unreached stimulus space and gives prescriptive guidance to the verification team on how to modify the testbench and constraints to achieve better coverage results. This can deliver substantial improvement in overall achieved stimulus and functional coverage, which greatly increases tape out confidence. Typically, ICO delivers this improved coverage in less time than the traditional CRV approach.

Fig. 2: Effects of ICO on coverage convergence.

Although it uses cutting edge AI/ML techniques, ICO is a robust and proven technology. It is available today, tightly integrated within the Synopsys VCS simulation solution. A recent white paper documents results from multiple real-world customer projects, including one verification team who achieved 5% higher overall coverage in considerably less time than for previous projects. Development teams deploy ICO at all stages of coverage-based verification, from planning and test development all the way through closure. There is no learning curve to benefit from the technology; every user of CRV can benefit immediately from higher coverage, fewer design bugs and shorter time to market.