Portable Stimulus Status Report

The first release of the Portable Stimulus (PS) standard is slated for early next year. If it lives up to its promise, it could be the first new language and abstraction for verification in two decades.

Accellera uncorked the PS Early Adopter release at the Design Automation Conference (DAC) in June. The standard has been more than two years in the making by the PS working group, whose members include some of the largest chipmakers.

Design verification evolved from the creation of directed tests to a methodology where effort is spent creating a testbench, and a constraint solver generates stimulus from that. The language (SystemVerilog), library (UVM) and methodology worked well for a number of years, but they are having problems scaling to designs typically seen today. They have failed to navigate the increase in execution engines often used by verification teams today and have extreme difficulty handling designs containing processors.

Portable Stimulus aims to solve all of those problems and tackle verification problems associated with the dominant design methodology used today, which is one of block reuse and integration. Verification at the system level requires the generation of tests that can handle multiple processors concurrently executing pieces of scenarios that equate to product requirements. These tests are being manually created today, but their complexity makes this an expensive, incomplete and error-prone task.

The road to the new standard has not been easy. Only Mentor, a Siemens Business, and Breker have been in this field long enough to have learned some of the subtleties of the user requirements, and they are certainly not in agreement about the right approach to be used for the standard. This has led to a highly charged development effort that has required a departure from preconceived notions.

Fig 1. Portable Stimulus Concept. Source Breker Verification Systems

The reason why this effort is so different from many other standards efforts within the EDA industry is related to the development of system-level tools in the 1990s. Most of those tools, which went under the moniker of ESL, were abandoned. EDA companies felt that the market was not big enough to support the necessary tool development. It was argued that there were only one or two system-level people in most electronics companies, making it uneconomic to support them. They also concentrated on design tools rather than verification tools.

This led semiconductor companies to invest in their own in-house system-level verification tools and methodologies. At the same time designs have evolved, so what was considered a system-level task in the past now is performed by a significant percentage of the development team.

Users are the experts in this subject area and they are making sure that EDA companies hear their requirements. They will not let the standard be what EDA companies know how to implement. Instead, they are insisting on what is required.

There were two events at this year’s DAC that provided a platform for users to talk about Portable Stimulus, to define what the standard means to them and to air some of their concerns. This article contains excerpts from those events, along with comments received from tool developers.

The user perspective
Portable Stimulus means different things to different user communities. “The real value of the PS standard is in bringing new abstractions to the verification community,” said Mark Glasser, principal verification architect at NVIDIA. “The PS language is based around the concept of processes, resources and constraints, and putting those things together in such a way that you can generate stimulus. This includes things such as scheduling semantic. It is really important to think about this beyond just being about generating stimulus. It is a new way of thinking about design and verification and will impact a lot of areas going forward. Most importantly, it brings a system-level view to the whole thing. We used to talk about verifying hardware, but now we talk about verifying a system, and a system is hardware and software mixed together in some combination.”

The impact on the entire development flow could be major. “We see the capability of developing a framework in which test comes first and design comes later,” said Monica Farkash, verification scientist for NXP. “You first design the intent of the hardware in an executable way, and then it can be implemented. Whatever you define as being hardware intent, you validate that what was implemented satisfies it.”

Verification engineers may be happy. “Verification has been the Cinderella for such a long time,” Farkash noted. “Verification grew with different capabilities and different languages, but it was still the result of the needs that came from design. PS means that their time is up and we are enabling the capability to change, or at least challenge, the model. Development will start with expressed intent about the design using PS, which can be executed on a number of different platforms. It is a framework on which we can validate the design and support test-driven development.”

Another view is one that supports the predominant design methodology. “We are optimistic about its application to IP integration at the SoC level,” said Karl Whiting, central verification and methodology tools group within AMD. “The SoC team does not see the full functionality of the IP teams, so having that portability of information will help with the integration. We see SoC teams struggle as they move into emulation, so we are optimistic about using PS in that arena. The really good news is that we are bringing software and verification people together and they are starting to talk.”

For some, it is an improvement to the verification methodology. “Look at quality aspects that the standard will give us—a lot of automation, a lot of test generation automation, things that we were not able to pull off with UVM today,” said Faris Khudakjie, senior verification technical lead at Intel and chair of the portable stimulus committee. “For example, there is a construct called schedule. It means that I, as a user, am allowing the tool to decide between parallel and sequential execution for these sub-activities. That is powerful, and it does not exist in SystemVerilog or UVM.”

Quality and automation are also key for Sanjay Gupta, director of engineering for SoC verification at Qualcomm. “It will provide a whole lot of automation using various tools to do the things that are more generic, things that tend to be repetitive in nature. You can do those things very easily and save a lot of time. Second is quality. It is almost assumed that the quality of the test will be better than manual tests. The third aspect is that there are certain testcases that are extremely hard to write manually at the system level, so being able to specify intent at a higher level and being able to create those testcases much easier. Resource management is one example of that.”

Initial assessments
But are the language and the methodology ready for widespread adoption? That is where things become more difficult, and there is a little bit of chicken and egg. Without access to the standard and tools that implement the standard, it difficult to assess if their users’ needs are being met. This is part of the reason Accellera released an Early Adopter version of the standard. Prior to that, only Accellera members could access the standard or know what was happening within the committee. It has now become available to a much wider group of people.

Glasser pointed to a number of areas for possible concern. “The language, which is currently in draft form, has some flaws. It is not functionally complete. It is a new language, and there is a large barrier to getting a new language out to the world.”

Khudakjie agreed. “The standard itself is revolutionary. It is asking people to take a step back and think about the system level. That will require a lot of effort from the leads in any company, a lot of effort from the vendors not just to promote their tools, but also to promote the ideas of the standard itself. They need to show how it benefits people.”

And this raises another concern. “When you talk about portability, think about the people who write the intent based on the spec and the group that benefits the greatest. That is the group that gets to re-use it,” said Whiting. “So what motivates the group to initially write the intent?”

There is a barrier to overcome. “What is the biggest challenge?” asked Farkash. “Existing solutions.”

There are people who feel that more time is required to assess various aspects of the standard. “Because PS is trying to cover so many areas, can the tools actually target them effectively?” asked Daniel Schostak, CPU verification architect at ARM. “For example, can a single model effectively target different areas such as a model for simulation or a model for FPGA. They have different concerns, and all of these models require domain-specific knowledge. Will the tools actually be able to include that in their abstraction?”

Schostak would also like to see the scope increased as well. “To me, the main advantage of PS is merging many aspects of verification, but they did leave out formal.”

Review deadline ahead
The review period was set to be three months. “The goal of the Early Adopter release is to socialize the draft standard across a wider audience of verification experts and to solicit their opinions,” said Adnan Hamid, CEO at Breker. “The inclusion of feedback from a broader range of potential users, who are aware of its practical application to real world verification challenges, would strengthen the proposed standard significantly.”

To be able to provide feedback, the industry needs to be educated quickly. “Our primary focus is educating our users and others about what the EA draft includes,” said Tom Fitzpatrick, verification evangelist at Mentor and vice chair of the committee. “We have created a video course that explains the concepts behind Portable Stimulus and walks the user through an example to learn how to apply it in a typical situation.”

Steve Brown, director of product management in Cadence‘s Systems Verification Group, had a similar viewpoint. “We have developed early, working demonstrations of several of the more interesting use cases that the standard is aimed to address. We are educating customers, and this includes seminars, customer transition planning, and clarifying any misperceptions that may exist.”

Today, only the proprietary tools are available, which can make the review more challenging. “We are actively developing and planning for the official release that will support the standard,” Brown said. “We will have that schedule finalized after all public feedback is received and the standard has been officially voted in Accellera.”

Some pieces are available today. “We provided the basis for the C++ part of the format and are implementing aspects of the draft standard,” Hamid said. “This means users are able to try out these ideas on their projects, particularly in the area of UVM-based verification, embedded systems running on an emulator, and production device bring up. It is only by trying out the format on real designs that the committee will appreciate issues that need to be resolved.”

How close is the EA release? “We have about 80% of the requirement met with the EA release,” said Fitzpatrick. “There are a few other things that we want to look at such as additional ways to constrain the graph-base model. We can add sequential constraints. There are things that are not quite in there yet in terms of coverage. We have a little more work to do for 1.0. After that, we will wait for feedback from the user community.”

Concluding remarks
Feedback on the PSS v1.0 draft specification can be submitted through Sept. 15. “We are encouraging our customers to review the specification and provide any feedback directly to the Accellera Portable Stimulus Working Group (PSWG),” said Brown. “The timeframe provided should be adequate for evaluation and feedback. We have several customers that are up to speed on the standard already, and there are many whose interest was raised by the announcements at DAC.”

Time is running out, and there is less than a month before this review period closes. “The feedback period is short, so quick action is required,” Hamid stressed. “We will work with any engineer who wants to review the standard and get their feedback into the process.”

After the review period closes, the committee will have to decide if the draft is close enough to drive to completion in the near future, or if they will have to take a longer period of time to address concerns that have been raised.

There is a delicate balance between making something quickly, even if it is not perfect, and spending longer to make something better, but too late to be useful. However, without everyone’s input, they cannot make that decision effectively.

Related Stories
What Is Portable Stimulus?
The standard for verification intent modeling has a misleading name. It should be changed.
Verification Unification
Experts at the Table, part 2: Strategies for using Portable Stimulus to drive formal and simulation, as well as the common ground with coverage.
Verification Unification
Experts at the Table, part 1: The verification task relies on both dynamic execution and formal methods today, but those technologies have few connection points. That may be changing with Portable Stimulus.
Tuesday At DAC 2017
Why did Siemens acquire Mentor and who will be next? Portable stimulus contemplates a tough adoption challenge.