Verifying an extensible processor is more than a one-step process, especially when software compatibility is important.
Key Takeaways
Multiple disciplines are associated with verification, each important and each becoming increasingly complex. RISC-V adds another one — architectural conformance — which until recently has been tackled behind closed doors by just a few companies.
A key reason for adopting RISC-V is a desire to improve performance or power characteristics. What isn’t clear, though, is how to effectively measure those benefits. How can architectural features be traded off against these application-specific goals in ways that do not compromise software portability?
RISC-V International (RVI) is assessing the degree to which it needs to be involved. Does it have a responsibility to the community when it comes to defining what a RISC-V core is? And how do you get to the point of being able to say an implementation conforms to the relevant specifications? Short of becoming a full-blown verification effort, that isn’t an easy task.
At least part of the confusion stems from the fact that the functional verification community never fully defined what verification completeness means. When viewed as the cross product of all possibilities, very quickly leads to an intractable problem. Still, the industry does manage to release products that work, which is a testament to those engineers who can decide what matters and how to implement verification flows that provide a good return on the time and money spent on the problem.
Conformance is a verification skill that has been lost to most engineers, but the industry must play a fast catch-up game. “The distinction between architectural conformance and implementation verification is fundamental, though they often get lumped together in a way that makes verification engineers twitch,” says Vladislav Palfy, solutions management director for Siemens EDA. “Architectural conformance is about asking if what you have designed is actually a RISC-V core. You want to validate if it executes instructions as defined, handles exceptions properly, implements the memory model correctly, etc. It is validating that you are speaking proper RISC-V according to the official specification. Implementation verification, on the other hand, is all about making sure that your specific design works in reality, which has a habit of finding creative ways to break things. We’re talking about all the microarchitectural details like pipeline implementation, cache coherency, branch prediction, and those special corner cases that only reveal themselves at the least convenient moments, usually involving caffeine and deadlines.”
While both tasks may sound similar, they require different approaches and, in some cases, responsibility for defining and executing these tasks may be in different teams or organizations. “While there is commonality between these two activities, they are subtly different and require equally different approaches in their verification,” says Dave Kelf, CEO for Breker Verification Systems. “RISC-V core vendors are now facing the same issues as Arm, Intel, and others as they work to solve these issues, investing vast sums in new verification process development.”
RVI has been working to fill a portion of that task. “RVI has been tackling certification,” says Adnan Hamid, founder and CTO for Breker. “Certification is envisioned as a tiny subset of architectural verification, but remains a big and growing problem due to the growing set of extensions. The scope is huge, and it’s not static. They keep adding new things. Collectively, the industry is having to invent a process for coverage traceability, and this amounts to going through normative rules, which means going through a spec section by section and effectively taking a highlighter and marking sentences that need to be verified. This is step one. Then pull them out, cluster them, and figure out what are the degrees of freedom that need to be covered? This is organization. Then figure out a test flow and decide how to implement that test flow. You want traceability all the way from those normative rules, which essentially links into the original spec.”
Software compatibility
The success of RISC-V, and any other processor, is closely tied to the ecosystem that surrounds it. “RISC-V standardization efforts today focus heavily on architectural conformance, ensuring that all software-visible aspects of an implementation behave as defined by the ISA and platform specifications,” says Ashley Stevens, director of product management and marketing at Arteris. “Architectural compliance test suites validate instructions, CSRs, privilege modes, interrupt behavior, the memory model, and other ISA-visible components — including interrupt controllers or IOMMUs when they are part of the software contract. These suites continue to mature through contributions from the community, companies, and standards bodies, providing a solid baseline for functional completeness. This is measured through ISA-level coverage metrics or differential testing against reference models.”
Not everyone cares about software compatibility. “If you’re a big vendor, you may not be as worried about the openness and the interoperability of the standards because you own it all,” says Frank Schirrmeister, executive director for strategic programs and system solutions at Synopsys. “You are just doing it for yourself and are not bound to doing things like proving whether the core interprets the ISA and the profiles architecturally so that software can run on those profiles across different cores. That’s a very different scenario than designing it for openness. If you look at the RVI organization, they want something like you have in other spaces — the compliance check. And that is really meant to enable interoperability of software. If you pass that profile, then your software should work across them.”
RISC-V does not make this easy. “The RISC-V Achilles heel lies in its greatest strength, the flexibility of the open instruction set architecture,” says Breker’s Kelf. “This valuable flexibility can also result in the potential for incompatibility between devices from different sources as engineers strive to understand the evolving ISA specification. This incompatibility reduces the portability of software stacks between devices, leading to significant engineering overhead.”
This is why the problem needs to be constrained. “The origins and the philosophy behind RISC-V is openness and extensibility,” says Aimee Sutton, product manager for design verification solutions at Synopsys. “There’s no single definition of RISC-V. The creation of profiles was a way to facilitate software portability, but nobody is ever going to be in the position where they say that isn’t a RISC-V core because you’ve deviated from the ISA. With our RISC-V verification solutions, we have defined ISA coverage, and we’ve made it configurable and extendable so that customers can take advantage of that code. There are over 100,000 lines of SystemVerilog. They can use that and then focus on writing the more interesting coverage that is specific to their design.”
Standards remain at the core of compatibility. “Architectural conformance and implementation verification are handled independently but are integral to ensuring overall IP verification integrity,” says Pierre Selwan, technical fellow in Microchip Technology’s FPGA business unit. “Architectural conformance is typically achieved through compliance to all relevant standards and specifications. Golden reference models are used to ensure ISA compliance across the relevant extensions. All standard bus and respective interfaces are tested along with any other standardized modules.”
Ultimately, conformance means that a certain set of tests has been compared against a golden model. “RVI, through Harvey Mudd College, has been developing a large set of relatively simplistic test cases,” says Kelf. “They can do most of the unprivileged tests, but when getting into the privileged tests, automation becomes much harder. They have to handwrite them. For the things they feel they cannot do, and that RVI cannot source from other open-source solutions, we are stepping in with the required tests. They are going to be the harder features that are harder to write by hand, but we can generate those through our test synthesis tools.”
From conformance to implementation
“There are two main challenges with establishing compliance. First, to ensure that the core works at all, and secondly, to establish that the core always works correctly,” said Ashish Darbari, CEO of Axiomise. “Whereas, simulation-based unit tests and compliance suites address the problem of checking the existential challenge — a test passed — they are not ideal to solve the problem of establishing complete compliance, i.e., that all interleavings of all instructions work correctly for all operands, regardless of when the instruction was sent for execution. Formal techniques are a natural choice for this kind of exhaustive analysis and continue to demonstrate this successfully using powerful invariants that get proven with formal model checking tools. What is great about this approach is that, in addition to finding functional bugs, it also catches deadlocks, livelocks, safety, and security issues. Once the bugs are fixed, the invariants are run again to get an exhaustive proof establishing without doubt that the bugs were caught. Scenario coverage, which is a part of the six-dimensional coverage, offers proof-based insights into possible scenarios, providing deep insights to architects and verification engineers.”
Still, having complete confidence that something works is challenging. “Coverage metrics are like different instruments on a verification dashboard, each measuring something important, but none telling the complete story alone,” says Siemens’ Palfy. “Code coverage tells you which logic has been exercised. Functional coverage shows you which scenarios have been tested. Assertion coverage confirms that specific properties held true during execution. Each metric gives you a slice of truth, but here is the catch — they’re all telling different stories, often in isolation. You might have stellar code coverage, but completely miss a critical corner case in your RISC-V instruction pipeline. You might nail your functional scenarios, but never actually stress the architectural compliance points that matter for certification.”
The process of exercising a design to get this coverage information has largely remained constant for several decades. “Extensive torture testing is applied, streaming thousands of randomized instructions across the specified extensions,” says Microchip’s Selwan. “Numerous testbenches are applied to verify the functional integrity of the implementation.”
The faster those tests can be executed, the better. “One of the ways that we mitigate the verification cycle challenge is by using hardware-assisted verification to speed up the execution of tests,” says Synopsys’ Sutton. “Coverage is part of that, because that helps you to know what you’re testing. It’s not just like blasting it with tests and running zillions of cycles, but that is certainly one way to try to address the challenge of 10 to the 18 cycles that we were talking about.”
An increasing array of engines is being tasked with helping to speed it up. “There is a goal to be able to deploy the entire verification continuum, meaning that all types of verification engines can be used,” says Synopsys’ Schirrmeister. “It starts all the way from the virtual platform architect, through to hardware-assisted verification and the verification IP, which is especially important for the interfaces. The tools for test generation are rolling up into this, as well. You look at architectural coverage — for example, instructions, privilege, pass, exceptions. To exercise these, you have cover groups, which may be run in simulation. Then you look at scenarios. So it gets more complicated. Formal shines in some aspects of the problem.”
One of the capabilities associated with test synthesis is that it can target an array of execution engines. “You need to be able to synthesize content for simulation, emulation, FPGA, and post-silicon,” says Breker’s Hamid. “That may include both transaction-level tests, or software that executes on a core. At each level, the amount of coverage you get goes up by maybe a factor of 102. Maybe you can do 104 cases in simulation, 106 cases in emulation, and 108 cases in silicon. 108 or 109 in silicon is reasonable.”
You cannot afford to keep verifying the same things. “Implementation verification is where most engineering effort resides,” says Arteris’ Stevens. “This includes microarchitectural corner cases, timing interactions, coherence, ordering, security properties, and hardware-hardware protocol interfaces that software cannot see. Achieving completeness requires simulation, emulation, UVM environments, coverage-driven tests, and increasing use of formal verification. Unlike architectural testing, which naturally maps to instruction-level coverage, implementation completeness is much harder to quantify. It depends heavily on each design’s specific structures and interfaces.”
Coverage metrics show progress toward your goals. “Coverage metrics are key to assessing the quality of the design verification,” says Microchip’s Selwan. “Functional coverage is complemented by structural coverage metrics, such as statement, branch, condition, toggle, and FSM coverage. Extracting coverage metrics for highly configurable embedded IP cores, such as the MIV_RV32, is a more difficult proposition, generally requiring extensive runs to meet coverage requirements. Tool vendors are currently migrating toward advancing their products to extract coverage metrics from highly configurable implementations.”
It takes a highly skilled engineer to define those metrics. “What in the microarchitecture is it worth measuring coverage on? Which queues are likely to overflow? This is not cheap to do, and I don’t know any automation for it,” Hamid says. “The customer has to go through and mark up what they want to cover. It is an expensive problem, but at least we can describe it, and with enough time and money, we can go do it. Now we discover that Q5 in this system is not getting an overflow. Can it be done? Is it reachable? If so, what scenario is needed? This information tends to be unknown in the big picture, as in nobody knows — not even the architect. We do not have enough fidelity to know whether this case is possible. We are making engineering decisions about what to cross deterministically and randomizing everything else. But that’s just thinking about individual scenarios, and how you sequence those scenarios can lead to different issues in hardware.”
Completeness requires understanding the relationships between different coverage types. “It is not enough to know that you hit line 247 in your RTL,” says Palfy. “You need to know whether you hit it under all the conditions that matter, whether the assertions around it fired correctly, whether your test plan actually intended to verify that scenario, and whether it maps back to a specific architectural requirement. You need all that coverage data unified with relationships preserved, hierarchy maintained, and traceability intact. Imagine being able to see coverage from every verification engine (simulation, emulation, formal) merged intelligently, maintaining the design structure, tracking which tests contributed to what coverage, and most importantly, showing you the gaps.”
Verification gaps
In addition to coverage holes in an implementation, there are coverage gaps in the whole philosophy today. Over time, people are working to fill these.
“A major gap in the RISC-V ecosystem is the lack of standardized hardware interfaces beyond the core ISA,” says Stevens. “Scalable SoC integration depends on predictable, interoperable, and verifiable connections between processors and the rest of the system, including interconnects. This is where implementation verification becomes especially important. While many RISC-V developers adopt existing interfaces, such as AMBA CHI, these specifications have grown to exceed 1,000 pages, much of which is unnecessary for typical RISC-V systems. What would truly benefit the community is a RISC-V-focused subset, or a set of streamlined interface standards aligned with real-world use cases. Such standardization would significantly reduce redundant verification work, improve coverage consistency, and accelerate multi-vendor interoperability.”
Some of the gaps go beyond functionality. “I was talking to an engineer recently who had just taped out a RISC-V core for an automotive application,” says Palfy. “The core passed all the architectural tests, but three months into software bring-up they discovered their branch predictor had the accuracy of a coin flip. It was technically correct, functionally useless. All that performance they had promised was simply not there. And no test suite in the world would have caught it because performance verification is not something the ecosystem has standardized. Everyone is building their own custom infrastructure to answer, ‘Does this thing actually run fast?'”
Fast can create other problems, though. “For some designs, when you wind the clock performance up, parts of the design turn into hot spots that you really have to start worrying about,” asks Kelf. “We can run real workloads and synthetic workloads and see if we trigger hotspots. This plays into power discussions. The companies who are really concerned about this, such as automotive and data centers, want to make sure that they can exercise the hot spots and certify that they can handle the thermal resistance and the power consumption by manipulating clocks.”
Automotive, just like some other application domains, adds new layers to the problem. “If you are building RISC-V for automotive or industrial applications, you are facing a whole new nightmare in safety and reliability verification,” says Palfy. “ISO 26262 does not care that you passed the ISA tests. It wants to know about fault injection, error handling, and how well your functional safety mechanisms work. The existing test suites were not built with any of that in mind, so you are starting from scratch.”
A role for other techniques
Formal verification adds a complementary set of capabilities that have an important role to play. “Early RISC-V conversations brought up formal methods,” says Palfy. “There was this optimistic idea: ‘It is an open spec. We can formally verify entire cores against the ISA and prove correctness.’ That is theoretically beautiful, but practically brutal.”
It still can play an important role, however. “Formal verification is becoming a more important part of the solution,” says Stevens. “It is particularly strong for architectural compliance — ensuring ISA properties hold for all legal instruction sequences — and for enforcing correctness of hardware protocols in implementation verification. But formal is not a replacement for dynamic verification or system-level validation, especially when full SoC behavior and real workloads are involved. In practice, formal methods play a complementary role, proving deep corner-case correctness while simulation and emulation establish end-to-end completeness.”
Formal is actively being used. “The adoption of formal verification has provided a method to exhaustively test, within bounds, the validity of our designs and hence increase the code quality from the outset,” says Selwan. “Static formal tools are extensively used to identify and resolve bugs early, hence minimizing the subsequent verification cycle times. Formal verification is an important aspect of our development cycle, and through verification, its use will continue to play a valuable role in the realization of quality embedded IP products.”
New AI capabilities also can be leveraged. “RISC-V is an excellent domain for applying agentic AI to verification,” says William Wang, CEO of ChipAgents. “We’ve seen strong success using agentic AI for formal verification, particularly because processor designs are dominated by control-based signals, making them well-suited for formal reasoning and symbolic exploration. This contrasts with AI accelerators, which tend to involve more data-centric flows and are less amenable to purely formal techniques. As RISC-V adoption grows, AI-driven formal approaches can significantly accelerate both architectural conformance and implementation verification, offering scalable ways to ensure correctness and coverage in increasingly complex, configurable designs.”
Related Content
RISC-V’s Increasing Influence
Does the world need another CPU architecture when that no longer reflects the typical workload? Perhaps not, but it may need a bridge to get to where it needs to be.
Livelocks And Deadlocks In NoCs
How to prevent complex systems from freezing up.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Leave a Reply