Raising SoC Development Productivity With Portable Stimulus

The semiconductor industry has achieved significant productivity increases by virtue of the development, deployment, and scalability of reusable design IP. The EDA industry has also achieved significant productivity increases by virtue of the development, deployment, and scalability of reusable verification IP. A remaining bottleneck in the SoC development process stems from the inability to reuse verification stimulus across design scope and verification engines.

With this in mind, Accellera formed the Portable Stimulus Specification (PSS) working group to establish a standardized specification that enables test intent (i.e., stimulus) reuse. The working group consists of leading technologists from semiconductor, systems, and EDA companies. Mentor was the initiator of the standardization effort and donated its inFact graph-based syntax, which formed the basis of the standard. Both Breker and Cadence have subsequently contributed to the standard as well, and several of the semiconductor and systems companies have contributed in terms of defining objectives and requirements.

Many programming languages are “procedural” in nature: they describe a sequence of events that contain what is to be done, what is to be executed, and what is to be performed. On the other hand, the graph-based portable stimulus syntax (and hence the developing Accellera PSS standard) is “declarative” in nature: it describes the space of design functionality to exercise, or it describes a set of legal tests that can be created. Unlike a procedural description of test intent, a declarative description of test intent does not pre-suppose a pre-defined sequence or ordering. It describes all sequences and ordering, in the most efficient way, often referred to as a graph.

This type of graph should not be confused with the term graphical. Although a graphical rendering of a graph-based testbench is a standard feature of most portable stimulus toolsets, a graph-based testbench can also be rendered in a textual code form. Other terms for this type of graph include tree, nodes and leaves, rules and actions, and even automata.

Technically speaking, there are no tools that support the Accellera PSS today, as the standard itself has yet to be ratified. However, multiple EDA companies offer portable stimulus solutions that are based on a subset of contributions to the working group. (Note the difference between “portable stimulus,” which is not necessarily standardized, and “the Accellera PSS standard”.) Portable stimulus technology is coming, and its adoption is growing. The ratification of an Accellera PSS standard will enable EDA companies to compete, not on language syntax, but on what can be done with the syntax. A common syntax will enable verification users to adopt a single syntax, and then evaluate competing toolsets based on their relative abilities to perform useful work on a single syntax format.

Portable stimulus will help verification engineers improve SoC verification in three important areas.

Testbench stimulus reuse
Test scenarios for IP block-level verification are very different than test scenarios for system-level verification. Block IP test scenarios are usually in SystemVerilog, generated by a constrained random solver, and applied on a regression farm of simulators. System-level test scenarios are usually in C/C++ or assembly code, generated either by hand or by running system-level software as a testbench, and are applied on hardware assisted verification engines like an emulator or FPGA prototyping system. Bridging this gap and being able to reuse test scenarios in both environments is the desired outcome for portable stimulus.

Portable stimulus enables reuse of a test intent across verification engines and across multiple design scopes. Portable stimulus specifications of an AXI bus interface and an Ethernet interface can be combined to verify an Ethernet MAC IP block design, generating block level scenarios in SystemVerilog. Later in the design process, the same portable stimulus specifications can be aggregated with other portable stimulus specifications to verify the SoC in which the Ethernet MAC is instantiated. Portable stimulus can generate scenarios in either SystemVerilog or C, depending on the type of verification desired. The inFact toolset enables this because it supports SystemVerilog, C/C++, assembly code, and even other various sequence languages as well as simulators, emulators, and FPGA prototyping systems. It can also be used for block IP and system-level verification.

Testbench stimulus effectiveness
SystemVerilog constrained random tests are just that – random. Therefore each individual test scenario is generated randomly, with lots of redundancy. As such, over time, these tests consistently generate a standard bell-curve distribution, where the most common tests are repeated several times, and the most challenging (“corner case”) tests are never generated. This causes a gap in coverage closure, which is prevalent across the industry. In practice, it’s nearly impossible to write a perfect set of constraints that neither “under constrain” nor “over constrain” the test scenario generator (“solver”). There is undeniable mathematic proof that this is a major problem with constrained random testing. Yet verification engineers have moved to constrained random testing because it offers the allure of “testing what I didn’t think of” – and this is indeed a benefit.

The desired situation is one where verification engineers can realize the benefit of productive testbench automation (the benefit of “testing what I didn’t think of” through random means) but also flatten the bell-curve distribution to hit those difficult-to-find corner cases. This is a second key benefit of portable stimulus.

Portable stimulus enables actual targeting of scenarios, rather than random scenario generation with coverage measurement after-the-fact. Portable stimulus enables verification engineers to use the very same stimulus model to generate different stimuli, depending on the objectives of the day. In this way, the term “portable stimulus” is a misnomer. It isn’t actually stimulus at all. It is a specification describing behavior, from which the stimulus can be derived. The effectiveness of the resulting stimulus relies on both the quality of the portable stimulus description and the toolset used to operate on it. The portable stimulus syntax enables tools like inFact to use formal algorithms to generate test scenarios, giving verification engineers much more power in test scenario generation.

Testbench stimulus verification process improvements
Existing testbench technologies require that verification engineers choose between productivity and simplicity. Directed testing offers simplicity, but little productivity. Each test scenario must be generated with intent, by the verification engineer. There’s no opportunity for “testing what I didn’t think of.” Constrained random testing offers significant gains in productivity, but introduces equally significant increases in complexity. Some testbenches contain thousands of complex interdependent constraints. Debugging these is nearly impossible, and requires initial simulation runs before analysis. This presents a Catch-22 situation. The engineer can’t debug the testbench before simulating. But simulating with a defective testbench yields incorrect results.

The declarative nature of a portable stimulus description enables several different types of overall verification process improvements. For example, over or under constraining a testbench is a very common problem and can be very difficult (and time consuming) to debug and correct. An effective, portable stimulus toolset can use formal methods to analyze a testbench, before running a single test.

Also, a portable stimulus toolset can process a stimulus description into a coverage model. Coverage modeling often falls below the cutline in a verification process, as priority is given to stimulus creation. Portable stimulus lends itself to an incredibly intuitive (and readable) graphical rendering of a design’s functional or behavioral specification, making many design bugs evident by mere inspection of these renderings. The desired situation here is to use a toolset, like inFact, that can operate on a portable stimulus testbench to automate the testbench debug process, automate the coverage model generation process, and achieve both productivity and simplicity.

Next generation verification
Portable stimulus is coming, the only question is when.

Over a dozen semiconductor companies and four EDA companies are investing their time and energy in the Accellera PSS working group, demonstrating the shared opinion that reuse is needed for stimulus IP — and not just for design IP and verification IP. The resulting standard that emerges may be somewhat misnamed, as it will be more of a behavior specification than actual stimulus. But it will provide a common syntax from which stimulus that is portable can be derived.

Companies like Mentor are already investing heavily in technology that can operate on the Accellera standard and generate stimulus for simulation, emulation, FPGA prototyping, and potentially even the target silicon itself. What’s equally interesting about the emerging standard is the additional step-function gains in verification effectiveness and productivity that it enables.