ResNet-50 Does Not Predict Inference Throughput For MegaPixel Neural Network Models


Customers are considering applications for AI inference and want to evaluate multiple inference accelerators. As we discussed last month, TOPS do NOT correlate with inference throughput and you should use real neural network models to benchmark accelerators. So is ResNet-50 a good benchmark for evaluating relative performance of inference accelerators? If your application is going to p... » read more

One More Time: TOPS Do Not Predict Inference Throughput


Many times you’ll hear vendors talking about how many TOPS their chip has and imply that more TOPS means better inference performance. If you use TOPS to pick your AI inference chip, you will likely not be happy with what you get. Recently, Vivienne Sze, a professor at MIT, gave an excellent talk entitled “How to Evaluate Efficient Deep Neural Network Approaches.” Slides are also av... » read more

Optimizing What Exactly?


You can't optimize something without understanding it. While we inherently understand what this means, we are often too busy implementing something to stop and think about it. Some people may not even be sure what it is that they should be optimizing and that makes it very difficult to know if you have been successful. This was a key message delivered by Professor David Patterson at the Embedde... » read more

AI Inference Acceleration


Geoff Tate, CEO of Flex Logix, talks about considerations in choosing an AI inference accelerator, how that fits in with other processing elements on a chip, what tradeoffs are involved with reducing latency, and what considerations are the most important. » read more

Apples, Oranges & The Optimal AI Inference Accelerator


There are a wide range of AI inference accelerators available and a wide range of applications for them. No AI inference accelerator will be optimal for every application. For example, a data center class accelerator almost certainly will be too big, burn too much power, and cost too much for most edge applications. And an accelerator optimal for key word recognition won’t have the capabil... » read more

Understanding The Performance Of Processor IP Cores


Looking at any processor IP, you will find that their vendors emphasize PPA (performance, power & area) numbers. In theory, they should provide a level playing field for comparing different processor IP cores, but in reality, the situation is more complex. Let us consider performance. The first thing to think about is what aspect of performance you care about. Do you care more about the ... » read more

Performance Metrics For Convolutional Neural Network Accelerators


Across the industry, there are few benchmarks that customers and potential end users can employ to evaluate an inference acceleration solution end-to-end. Early on in this space, the performance of an accelerator was measured as a single number: TOPs. However, the limitations of using a single number has been covered in detail in the past by previous blogs. Nevertheless, if the method of cal... » read more

The Murky World Of AI Benchmarks


AI startup companies have been emerging at breakneck speed for the past few years, all the while touting TOPS benchmark data. But what does it really mean and does a TOPS number apply across every application? Answer: It depends on a variety of factors. Historically, every class of design has used some kind of standard benchmark for both product development and positioning. For example, SPEC... » read more

Evaluating NVMe SSD Multi-Gigabit Performance


The multi-channel parallelism and low-latency access of NAND flash technology have made Non-Volatile Memory express (NVMe) based SSDs very popular within the main segments of the data storage market, including not only the consumer electronics sector but also data center processing and acceleration services, where the key role is played by specialized FPGA-based hardware for application-specifi... » read more

Power/Performance Bits: Jan. 28


Accelerator-on-chip Researchers at Stanford University and SLAC National Accelerator Laboratory created an electron-accelerator-on-chip. While the technique is much less powerful than standard particle accelerators, it can be much smaller. It relied upon an infrared laser to deliver, in less than a hair’s width, the sort of energy boost that takes microwaves many feet. The team carved ... » read more

← Older posts