中文 English

How To Measure ML Model Accuracy


Machine learning (ML) is about making predictions about new data based on old data. The quality of any machine-learning algorithm is ultimately determined by the quality of those predictions. However, there is no one universal way to measure that quality across all ML applications, and that has broad implications for the value and usefulness of machine learning. “Every industry, every d... » read more

Customized Micro-Benchmarks For HW/SW Performance


Raw performance used to be the main focus of benchmarks, but they may have outlived their usefulness for many applications. Dana McCarty, vice president of sales and marketing for AI Inference Products at Flex Logix, talks about why companies need to develop and utilize their own specific models to accurately gauge hardware and software performance, which can be slowed by bottlenecks in I/O and... » read more

The Best AI Edge Inference Benchmark


When evaluating the performance of an AI accelerator, there’s a range of methodologies available to you. In this article, we’ll discuss some of the different ways to structure your benchmark research before moving forward with an evaluation that directly runs your own model. Just like when buying a car, research will only get you so far before you need to get behind the wheel and give your ... » read more

The Problem With Benchmarks


Benchmarks long have been used to compare products, but what makes a good benchmark and who should be trusted with their creation? The answer to those questions is more difficult than it may appear on the surface, and some benchmarks are being used in surprising ways. Everyone loves a simple, clear benchmark, but that is only possible when the selection criteria are equally simple. Unfortuna... » read more

Edge-Inference Architectures Proliferate


First part of two parts. The second part will dive into basic architectural characteristics. The last year has seen a vast array of announcements of new machine-learning (ML) architectures for edge inference. Unburdened by the need to support training, but tasked with low latency, the devices exhibit extremely varied approaches to ML inference. “Architecture is changing both in the comp... » read more

Standard Benchmarks For AI Innovation


There is no standard measurement for machine learning performance today, meaning there is no single answer for how companies build a processor for ML across all use cases while balancing compute and memory constraints. For the longest time, every group would pick a definition and test to suit their own needs. This lack of common understanding of performance hinders customers' buying decis... » read more

ResNet-50 Does Not Predict Inference Throughput For MegaPixel Neural Network Models


Customers are considering applications for AI inference and want to evaluate multiple inference accelerators. As we discussed last month, TOPS do NOT correlate with inference throughput and you should use real neural network models to benchmark accelerators. So is ResNet-50 a good benchmark for evaluating relative performance of inference accelerators? If your application is going to p... » read more

One More Time: TOPS Do Not Predict Inference Throughput


Many times you’ll hear vendors talking about how many TOPS their chip has and imply that more TOPS means better inference performance. If you use TOPS to pick your AI inference chip, you will likely not be happy with what you get. Recently, Vivienne Sze, a professor at MIT, gave an excellent talk entitled “How to Evaluate Efficient Deep Neural Network Approaches.” Slides are also av... » read more

Optimizing What Exactly?


You can't optimize something without understanding it. While we inherently understand what this means, we are often too busy implementing something to stop and think about it. Some people may not even be sure what it is that they should be optimizing and that makes it very difficult to know if you have been successful. This was a key message delivered by Professor David Patterson at the Embedde... » read more

AI Inference Acceleration


Geoff Tate, CEO of Flex Logix, talks about considerations in choosing an AI inference accelerator, how that fits in with other processing elements on a chip, what tradeoffs are involved with reducing latency, and what considerations are the most important. » read more

← Older posts