An Insider’s View Of Verifying Custom RISC-V Processor Cores

By Shubhodeep Roy Choudhury, Valtrix Systems, and Lee Moore, Imperas Software
Supporting images courtesy of Bill McSpadden, Seagate Technology

This article is derived from a talk at the RISC-V Summit in December 2020 that Bill McSpadden, principal verification engineer at Seagate Technology, gave on the challenges and experiences his team faced in the verification of two custom RISC-V processor cores. While a technical presentation at a technical conference may not be completely unexpected, the unique part was the frank acknowledgment that without any hype or fanfare the team tackled some of the most challenging verification problems in the industry today—how an experienced SoC team can verify a custom RISC-V processor. This article captures some of the highlights of the presentation and includes extra information from the tool suppliers Valtrix and Imperas.

The problem
The challenge was to improve the performance for the HDD (hard disk drive) servo controllers with a new embedded processor core beyond the mainstream offering of the available IP providers. The key requirements for HDD servo controllers can be summarized as: math-intensive application workload, real-time computation, and within fixed temporal boundaries. While some aspects may appear similar to other general embedded applications, it was clear that the current processor IP family roadmap was insufficient to address the needs. As a result, the servo processor performance was about to max out, thus becoming a defining limit of the top-level features and performance of the HDD.

The solution
The solution developed around a new high-performance core (HPC) with the following high-level summary of the technical details: RISC-V spec references [1] RV32IMFCN; machine and user modes; speculative out-of-order pipeline; 4-wide fetch and dispatch; 8-wide issue (peak), high-performance instruction fetch including branch prediction; return address stack; and loop buffer. Plus, dual-ported load-store unit; large L1 tightly-integrated memories; multi-way L1 caches; decoupled high-performance FPU; PLIC (platform-level interrupt controller); native ECC support; and RISC-V debug and trace support. In short, an advanced processor design targeted at the key requirements for the HDD servo controller.

As a companion to this first core, the project scope was expanded to also co-develop an area optimized general controller core as a parallel project. Highlights of the technical details for the area optimized cores include: RISC-V spec references [1] RV32/64IMCBN; machine; supervisor; user modes, SV39/SV32 virtual memory support; 5 stage, single-issue, in-order pipeline; CLIC (core-local interrupt controller); enhanced PMP; focus on optional security features; native ECC support; and RISC-V debug. Clearly this companion project achieved many efficiencies and advantages overall, but also expanded the workload and pressure on the verification team.

The problem of verification
These engineering specification-based solutions also present a new problem. As a consequence of all the features and configuration options, the verification task will be significant. To address a range of requirements and use case needs across multiple designs, the configuration options included: core architecture (32/64 bit, extensions supported) and configurations of the core architecture (machine/supervisor/user modes, supported addressing, CSR bits, implemented, etc.). This led to a huge number of permutations, which resulted in a very serious problem for verification and validation engineers.

“How can we possibly test all of the configuration/modes?” asked Bill McSpadden as part of his enthusiastic opening remarks. He then explained, “Experienced DV engineers know of at least one way to attack the problem: Constrained Random Verification Environments (CRVEs).”

Thus, the stage was set for an interesting guide through the verification plan and some deep insights in to the DV challenges around RISC-V processor design.

The test bench
The test bench is central to the test plan. This was based around verification IP (VIP), instruction retirement monitor, and UltraSoC (now part of Siemens EDA) instruction trace decoder. The fabric VIP and debug port VIP are set up to inject stimulus into the DUT (device under test) while executing code so that activities could be observed—such as program buffer executing, single stepping, register reads, CSR reads, abstract debug commands, and interrupts—to directly interact with the execution of the core and test that it responded correctly. See figure 1 for the block diagram.

Fig. 1: The test bench pieces.

The test bench with random instruction sequences and scoreboarding
To exercise the core within the test bench framework, the Seagate team developed a software flow, shown in figure 2. The instruction generator was supplied by Valtrix Systems [2], which generates tests with tens of thousands of files. These are run on the RTL simulation, and so this produces two log files based on the instruction retirement monitor and the UltraSoC trace output. This allows both options to be compared with the same input stimulus running on the Imperas RISC-V Golden Reference Model [3], which was the team’s go-to model for any “go/no-go” tape-out decisions. This trace compare approach supports scoreboarding while checking instruction sequences, CSR’s at exception boundaries and filters out trap handler execution.

Fig. 2: Random instruction sequences and scoreboarding.

Highlights of the test plan in action
For each of the cores, the Seagate plan utilized many similar components and structure:

1. Build the test bench with fabric VIP, plus debug and trace VIP
2. Use the Valtrix STING random instruction sequence generation as a common source
3. Inject random stimulus (such as debug abstract commands on the debug port, program buffer execution, single-stepping, interrupts)
4. Compare the same instruction sequence running on both the core RTL and the architectural Imperas golden reference model to make a direct comparison for correct behavior

The team followed this method and flow with lots of simulation cycles, running thousands to tens of thousands of tests on a nightly basis, followed by careful review and analysis of the resulting log file comparison reports.

The RISC-V verification ecosystem
Across the EDA and tools ecosystem a number of options are available to assist on the verification effort. These tools include:

• Random test sequence generator from Valtrix Systems, https://www.valtrix.in
• An architectural Golden reference model from Imperas, https://www.imperas.com
• Architectural tests from the RISC-V International Compliance Task Group, https://github.com/riscv/riscv-compliance
• SystemVerilog UVM from Accellera, https://www.accellera.org
• SystemVerilog simulator and waveform viewer were from an industry standard vendor.

The constrained random instruction sequence generator used for this project was STING, supplied by Valtrix Systems. STING is constrained to allow tests to be focused around particular sequences. Test sequences can be based on snippets of code or the Valtrix supplied snippets, with new snippets added to address coverage gaps. This is a classic example of coverage driven verification which is extremely useful way to measure progress towards completion of the processor core DV tasks.

The Golden RISC-V Reference Model was used as the “go/no-go” model to determine the RTL correct behavior with and discrepancies, bugs or issues with the design, tests, specifications, or test bench.

The combination of the Valtrix instruction sequence generator and Imperas Golden RISC-V Reference Model became the essential partnership supporting the verification task. Unexpected situations were first reported to Valtrix, which manage the resolution process with Imperas, including a bi-weekly progress meeting with regular updates and good response times across multiple time zones.

“The relationship has worked very well,” said McSpadden. “The STING generator has provided excellent functional coverage which we use to measure in our test bench environment.”

“The Imperas Golden RISC-V Reference Model helped us find many bugs in our cores,” he said. “However, the RISC-V architectural tests yielded no bugs, which is expected since the architectural tests are a subset of full verification.”

The results
“We had servo firmware up and running within weeks of silicon arrival,” McSpadden noted. “Real application software running on real hardware doing real disk accesses. The summary ‘works as expected’ is ultimately down to all the effort from our dedicated team plus the support from Valtrix, Imperas, and other members of the RISC-V verification ecosystem.”

Conclusion
The open standard ISA of RISC-V permits all design teams to explore significant flexibility in approaching a custom processor design. From the initial concept and feature tradeoff discussions, consideration, and implications for processor verification should be included. The growing RISC-V verification ecosystems—including Valtrix, Imperas, and many others—are available to help and support the full flexibility of the RISC-V specification.

References
[1] See the RISC-V specifications at https://riscv.org/technical/specifications/
[2] Valtrix Systems https://www.valtrix.in
[3] Imperas Software https://www.imperas.com
[4] riscvOVPsimPlus, the free RISC-V reference model available at https://www.ovpworld.org/riscvOVPsimPlus/

Lee Moore is the lead engineer at Imperas for RISC-V processor models and simulation tools. Prior to Imperas, Moore worked as a senior consulting engineer for EDA vendors such as Co-Design Automation and Ambit, and for ASIC vendor NEC Electronics. Moore is also a private pilot, and recently developed PilotAware, a low-cost air traffic awareness device, helping aircraft to see and be seen.