Defining Functional Accuracy

I have been heavily involved in a project that recently completed. It involved creating virtual platforms (VPs) for a number of Altera’s FPGA SoCs. If you’re interested in more information, an announcement on the VP availability went out last week. Some of the modeled platforms existed and some were in various stages of development.

The goal of the project was to deliver functionally accurate platforms. For the devices that already existed, we were verifying the virtual platforms against the same test programs that were used to verify the cycle accurate representation of the device. For me personally, an interesting issue that came out of our verification activity was a better appreciation for different assumptions as to what “functional” accuracy consists of and what kinds of differences would be acceptable in different scenarios.

At the simplest level if we are executing a single-threaded application on a device with no outside input, then the functionally accurate platform should correlate exactly with the cycle-accurate representation. The challenge is that very few real world applications fall into such a restricted domain. Generally, some of the drivers for creating an embedded system are made to deal with multiple things at once, and we usually care about the interaction of our device with external activity. On our first pass through the verification suite we ran right into this issue. Many of the tests did not agree when using the purely functional representation of the platform. On further inspection we found that differences in cache activity, variations in system state on interrupt arrival and fluctuation of availability of peripheral resources caused the bulk of our differences. Not that there weren’t a few bugs as well, but the bugs were in some ways easier to deal with.

The infrastructure tool we were using, Mentor’s Vista, supports a scalable model, which allows incremental enhancement of the timing accuracy of the platform. Arguably the pure functional VP was accurate even though we were seeing differences with the cycle accurate representation, but we decided to see if we could improve the correlation. By turning on some of the timing attributes, we were able to eliminate many, if not all, of the differences we had been seeing in the platform results. For some the differences became much less significant, such as an interrupt occurring a few cycles off, but still affecting the system in the same way.

Ultimately we are dealing with an abstract model, which is sacrificing accuracy for modeling and simulation efficiency. It is important to understand what level of accuracy is required and ensure that the platforms and models being used effectively represent the characteristics being analyzed.