How Much Verification Can One Engineer Handle?

By Frank Schirrmeister
When reviewing the agenda of our upcoming Verification Summit here in San Jose this Thursday, the question came to mind of who can actually execute the required complex verification tasks. Can they understand enough detail in hardware, software, and the system aspects to efficiently rid the design of bugs? The reality is that the task requires not one engineer who can do it all, but instead a team of engineers that is enabled by our development tools to more efficiently interact within a combined verification infrastructure, with debug views for software, hardware, and system aspects, as they are familiar to the individual experts.

In the upcoming Verification Summit, we have two keynotes from Brian Bailey and Jim Hogan, followed by user presentations from Zenverge, NVIDIA, Ambarella, and two presentations from Broadcom. Zenverge’s Kent Goodin will talk about firmware integration and debug on the Palladium XP II platform for their range of products providing targeted levels of performance for bridging broadcast video and audio content to a variety of IP-enabled devices like DVRs and gateways. NVIDIA’s Vik Singh will talk about software bring-up and verification using software test on top of Palladium emulation in a hybrid use models together with the processor sub-system in a virtual platform. Broadcom’s Mehran Ramezani will present on how they represented the system environment as part of an ”embedded testbench” within Palladium emulation, as well as how they accelerated embedded OS verification using a hybrid of an ARM Fast Model-based virtual platform with Palladium emulation.

So what types of users are involved in those verification tasks? Well, they range from hardware engineers who are typically not aware of the software until a problem arises, treat software as a black box and are not familiar with software-development environments, to pure software developers who develop at the application level, treat the hardware as black box, and are not familiar with hardware-design and hardware-debug tools. In between, we find three types of engineers. First, verification engineers who are hardware-oriented but have some software background and are traditionally focused at the hardware level, but are occasionally asked to verify the software, and, because of their mixed background, are familiar and open to both domains. Second, hardware-aware software developers who develop software at the bare-metal or operating-system level and may be familiar with the hardware and the hardware design environments, being more open to debug the hardware dependencies. Third, system integrators are the “middle-men,“ so to speak, who are typically involved with operating system bring up. They treat both the software and that hardware as black box, but to some extent are familiar with both.

This figure shows an example setup of how debug could look like.

The software is loaded from memory to run on the actual RTL of a processor in an emulator, that may be connected using JTAG and ARM’s DSTREAM box to a host on which the ARM DS-5 debugger resides. That’s the software view. The rest of the system on chip (SoC)’s hardware also runs as RTL on the emulator. Using FullVision debug capabilities, it is debugged using SimVision on a host workstation to which the emulator is connected. That’s the hardware view. In addition, system debug parameters like memory bandwidth and bus transactions are measured and visualized as well. The actual system environment like SATA, PCIe, USB, and Ethernet devices can be connected using rate adaptors—SpeedBridges—that take care of the required speed adaption to run the system at the MHz speed range on emulation.

And this is a straightforward view—everything runs on an emulator. As we will hear during the upcoming Verification Summit, the execution engines can be mixed quite a bit. The processor sub-system may run as a virtual platform on a host, connected via SCE-MI to the rest of the chip’s hardware running in emulation. The system environment may be represented as software running on a special processor as an “embedded testbench” in emulation, representing the system environment that the chip resides in at higher speed within the emulator to enable faster software bring-up.

Finding the one engineer who understands all of the aspects and debug views in full detail is difficult, if not impossible. What we find in our customer engagements on this is, that our verification infrastructure increasingly is used to bring different team constituencies together to jointly debug a problem that spans several disciplines. For the users, the situation is like looking into the same house—the design under debug—through different windows tailored to be specific to their discipline, that is, the software, hardware, and system aspects.

Hey, that means our tools enable users to meet new people and make new friends…