Is Hardware-Assisted Verification Avoidable?

Emulation is emerging as the tool of choice for complex and large designs, but companies that swap from simulation to emulation increasingly recognize this is not an easy transition. It requires money, time, and effort, and even then not everyone gets it right.

Still, there are significant benefits to moving from simulation to emulation, providing these systems can be utilized efficiently enough to warrant the investment. HDL simulation, which has been the industry workhorse for functional verification since the dawn of the semiconductor industry, started to run out of steam when processor performance stalled because HDL simulation was unable to benefit from multiple cores. So as designs got larger and HDL performance remained essentially constant, companies no longer could close their functional verification goals by buying more simulator licenses. They were forced, instead, to start using hardware-assisted verification (HAV). Today, companies spend more on emulators and physical prototyping systems than they do on simulation (see figure 1.)

Fig 1. Revenue for hardware-assisted verification tools and HDL simulators. Source: ESD Alliance

Can small designs continue to rely on simulation only? “In general, smaller design houses focus on small- or mid-size designs, exemplified by the proliferation of IoT applications, and can benefit from traditional simulation farms,” says Jean-Marie Brunet, senior director of marketing at Mentor, a Siemens Business. “When that is not feasible — for instance, when embedded software validation is required, or a shorter verification time frame is imposed by a challenging TTM — the acquisition of an HAV platform becomes necessary.”

Hardware-assisted verification is becoming mandatory for a growing portion of designs. “The size of designs is growing, and the definition of the small design has changed,” says Zibi Zalewski, general manager of the hardware division at Aldec. “Even small design houses can now create SoCs thanks to the scalability of processor cores and ready to use IPs. That impacts the verification methodologies and tool environment. Simulation speed is still good enough for the development stage, but not for verification, testing and debugging when the design grows to 10 million to 20 million ASIC gates.”

While processor cores generally do not need to be verified, they do change the dynamic of verification. “Today, every meaningful design has a processor, and if you’re not designing the piece with the processor you probably design a piece of IP that sits next to the processor,” says Johannes Stahl, senior director of product marketing at Synopsys. “Somehow, you need to be able to execute the cycles that the processor requires, and that means many millions or billions of cycles. That is a challenge for everyone that has to make the transition. There’s no way around it. If you want to stay relevant, if you want to deliver a service to somebody that builds a big SoC, how do you make sure that the piece that you deliver really plays in the bigger context?”

For those who want to verify a processor core, such as RISC-V, simulation remains the preferred method by the Open Hardware Group. But emulation can provide a significant gain over simulation.

Alan Pippin, member of technical staff for Hewlett Packard Enterprise, recently gave a presentation at DV Club in which he reported results using emulation for a RISC-V design. “We had a design that was focused on the security application space with about 25 million gates. The stimulus was entirely in Verilog and was synthesizable. We loaded program instructions in memory and were able to generate all the stimulus needed for that test bench. This particular application, generating large security keys, took about three hours for simulation, and we could do that on the emulator in 35 seconds. That is a 300X speed up.”

You can verify a processor core with relatively short test sequences. “With simulation, you’ll be able to run hundreds, maybe thousands of transactions,” says Khaled Labib, vice president of engineering for Arteris IP. “But you need millions and billions of transactions just to boot a system, let alone run real traffic on it. So you will end up with a non-representative verification process that could expose you to lots of bugs — or fail to expose you to performance issues that you can only see once you start to pump in a lot of traffic. Those are the most precious bugs and provide the highest return on investment.”

Hybrid solutions are the avenue of choice for many people. “Software developers require an efficient work environment, with fast boot-up time and debugging, which is simply not possible with an HDL simulator,” says Aldec’s Zalewski. “Quite often virtual platforms, commonly used by software teams, can be integrated with the HAV, which connects software and hardware domains at the very early stage of the project and allows you to find many software-hardware interface problems.”

This can lead to very creative solutions. “We worked with one company that used fast processor models to boot the system,” says Simon Davidmann, CEO of Imperas Software. “Then they did a hot swap into the Verilog and hardware-assisted environment. This type of thing used to be very difficult, but today the processor models come with interfaces around them for the major EDA vendors’ simulation environments. They can then put that into the emulator with little effort. Before, they weren’t well interfaced and there was a lot of work that had to be done. Issues remain on the testbench side, but it is now possible with days of work rather than months.”

Making the transition
Hardware-assisted verification has become more approachable. “The industry has gone through a lot of changes over the past five years and it has become more affordable,” says Zalewski. “An in-hardware verification solution doesn’t have to be a huge and expensive system available only for the biggest ASIC houses. The growth of FPGA capacity has enabled the acceleration of emulation solutions even for small designs. Single FPGA devices can be used to implement even 20 million ASIC gate designs without the need of multi-FPGA partitioning.”

Bringing hardware-assisted verification in for the first time is not easy. “For a small company, emulation systems are expensive,” says Arteris’ Labib. “The ones with more advanced features run you about a million dollars for an average starting configuration. If you want to start lower than that, you’re going to have to either give up on features or go with a smaller commercial configuration. But this has become critical to our business, and even as a small company we had to find the necessary budget to buy emulation.”

There is a tradeoff. “Assuming that verification cannot be accomplished via simulation farms, a verification team is left with two alternatives,” says Mentor’s Brunet. “They can tradeoff an emulation platform for an enterprise-class FPGA prototyping platform, saving some upfront cost. Or, they can access a cloud-based HAV platform on a pay-per-use model and avoid the cost of acquisition. The cloud deployment of an HAV platform in virtual mode, as opposed to in-circuit emulation (ICE) mode, is equivalent to the cloud deployment of any other EDA tool once the equivalent requirements are met. And not to be ignored, the verification team must be properly trained to use the new tool.”

Training is critical. “Smaller companies might have some experience with prototyping, because emulation from a pricing perspective is out of reach,” says Synopsys’ Stahl. “They normally acquire prototyping systems, but they are optimized for a specific design and often cannot be re-used. Emulation is a little different because it may be cost-prohibitive for a single project. In addition, they might not meet all the requirements, like the space or the long-term justification for an emulator, so they might lease a hosted emulator. Many smaller companies start with leasing an emulator so they don’t incur the effort of having your own space ready for an emulator.”

Provisioning space can be a serious issue. “We had physical limitations,” said Labib. “These systems require minimum cooling requirements. They require minimum power — a very specific kind of three-phase. We even struggled with a suitable place, because you could be on a second floor and the box is just too heavy. How do I get something of this kind of size into the office? That’s why I initially thought about the cloud, but it was cost-prohibitive if I am going to use it over time. I did the math on the back of an envelope, and then I said, ‘If I am committing to this methodology, for that many number of years, it does not make sense — at least with the financial structure that was given to me.’ It was then much easier for me to just buy it and and utilize it.”

For others, the cost of the emulation system is easily justified. “If you have very large and complex chips where the risk of re-spins is more costly than the investment required to acquire the emulation hardware, then this is an easy decision,” says Bipul Talukdar, director of applications engineering in North America for SmartDV. “Hardware prototyping via FPGAs is used extensively for less complex chips and is a lower-cost solution. HAV also is employed for post-silicon verification where FPGAs, coupled with verification IP, are used to test the product.”

Getting the necessary ROI
Getting good enough utilization of emulation hardware from a single project may make it difficult to justify from an ROI perspective, even for large companies. “Unless you are constantly feeding jobs into the emulator, you’re certainly not going to get good ROI,” says HPE’s Pippin. “You have to keep the emulator busy. We continue to work on that in our lab, and implement the technologies to make that happen.”

With enough designs, the math becomes easier. “Our IP solution is configurable, and I can have many configurations simultaneously emulating on the same hardware,” says Labib. “We also have a regression environment, which was a considerable investment upfront, but now that it is set up the box can be kept busy 24 x 7 running regressions, reporting results automatically, and failures are saved on the side.”

Testbench issues
The biggest hurdle to emulation adoption is the testbench. “When a company first adopts emulation, the natural tendency is to use emulation as a faster simulator,” says Stahl. “And then they learn Amdahl’s Law. If the testbench consumes so much of your simulation time, then you may only get 2X increase in performance. Then they ask if there is something they can do to the testbench, and this quickly evolves to the topic of needing to run software. They quickly learn to build transaction-based environments to run software. Simulation acceleration becomes a smaller part of the use model, and maybe never happens because they have so much software to run.”

It requires a big investment. “The initial investment is for the DUT itself,” says Labib. “I have to synthesize it, worry about mapping all the memories, worry about clocks, resets, and so forth. But the really large investment is migrating my testbench, and that’s a very substantial investment. We integrated a lot of transactors, which are half hardware, half software. We exchange what are normally the agents in the VIPs with those transactors. If I try including scoreboards, I immediately impact my performance. We ended up stripping our test benches to a bare minimum.”

That creates a dilemma, because without scoreboards issues can hide in the design. “Scoreboards flag things early,” says Labib. “If I take an average design, I typically have a top-level testbench in simulation. But underneath that, I have block-level test benches that are monitoring every block and flagging things early. When a failure happens, it is caught immediately in simulation. When I come to triage these bugs, I am close to the cycle that failed. When I go to emulation, I’m going to have to remove many of those testbench elements. Now I have a very long path and we are only going to see the failure towards the end of the test. Debug now becomes a much more difficult problem.”

Finding the right mix of simulation and emulation is important. “Simulation compilation is fast and runtime is slow,” says Stahl. “In emulation, compilation takes longer but execution is much faster. They need to find the middle point between what is done on the simulation side, which is typically very few cycles, maybe coming out of the reset sequence and figuring out what’s going on at a very basic level, and then they jump into emulation. In emulation, design teams have to find the right balance between how many times they have to recompile the design, versus how many times they run the design and look at the issues. It takes time to develop a methodology. What is a good debug methodology for emulation? How do you instrument your design to look at the many signals you could look at, but still run fast?”

While the difficultly of debug is seen as a negative when compared to simulation, it is a positive when compared to real silicon. Pippin reported a surprise when HPE first tried various use cases for emulation. “We tried emulation with our post-silicon tool stack before we actually received the real hardware back from the fab. We were able to get the software brought up before the hardware was available. From a post-silicon validation perspective, this was a big surprise, and we weren’t expecting to have good results here. We were pleasantly surprised by the outcome. And we found that it was easier to debug and get some early bugs resolved before the hardware even arrives so that we could hit the ground running once it did.”

Conclusion
When inserting emulation for the first time, the cost of the hardware and the porting of the DUT may seem like the biggest issues, but the testbench is the real challenge. How well you deal with that issue will dictate how successful you are. Companies that attempt to rejig an existing testbench developed for simulation will likely experience sub-par results.

So how much support can you expect from the HAV vendor? Not enough, according to Labib. “You came to a customer that had a massive testbench deployed, and you’re asking us to trim all of this stuff out, go bare bones. Everybody claims to have automatic testbench tools, but when you come to do the work, you end up either paying massively on your speed, or you’re signing up for nine months of work to rewrite that testbench to get it to do something useful.”

The industry still needs help in putting together emulation flows. “This type of advancement is often done in private,” says Imperas’ Davidmann. “Even when companies do present their findings, they don’t tend to give any details. With the open source guys, it is not emulation today, but they are making everything they do visible, and people can see the methodologies being used and how they evolve over time. Today, they do a lot of stuff in the Cloud. Interestingly, many Cloud systems now contain FPGAs and people are starting to try and put the RTL into these Cloud FPGA blocks for simulation purposes.”

A transition into hardware-assisted verification is not easy. It requires money, time, and perhaps most importantly, training to get good results. For many companies, finding someone with emulation experience may be their best investment.