tag: benchmarks

BYO NPU Benchmarks
By Steve Roddy - 14 Dec, 2023 - Comments: 0

In our last blog post, we highlighted the ways that NPU vendors can shade the truth about performance on benchmark networks such that comparing common performance scores such as “Resnet50 Inferences / Second” can be a futile exercise. But there is a straight-forward, low-investment method for an IP evaluator to short-circuit all the vendor shenanigans and get a solid apples-to-apples result... » read more

Does Your NPU Vendor Cheat On Benchmarks?
By Steve Roddy - 12 Oct, 2023 - Comments: 0

It is common industry practice for companies seeking to purchase semiconductor IP to begin the search by sending prospective vendors a list of questions, typically called an RFI (Request for Information) or simply a Vendor Spreadsheet. These spreadsheets contain a wide gamut of requested information ranging from background on the vendor’s financial status, leadership team, IP design practices... » read more

Looking Beyond TOPS/W: How To Really Compare NPU Performance
By Paul Karazuba - 23 Mar, 2023 - Comments: 0

There is a lot more to understanding the true capabilities of an AI engine beyond TOPS per watt. A rather arbitrary measure of the number of operations of an engine per unit of power, the TOPS/W metric completely misses the point that a single operation on one engine may accomplish more useful work than a multitude of operations on another engine. In any case, TOPS/W is by no means the only spe... » read more

AI Benchmarks Are Broken
By Jon Bunting - 14 Mar, 2023 - Comments: 0

Artificial Intelligence (AI) is shaping up to be one of the most revolutionary technologies of our time. By now you’ve probably heard that AI’s impact will transform entire industries, from healthcare to finance to entertainment, delivering us richer products, streamlined experiences, and augment human productivity, creativity, and leisure. Even non-technologists are getting a glimpse of... » read more

Is RISC-V Ready For Supercomputing?
By Brian Bailey - 09 Feb, 2023 - Comments: 0

RISC-V processors, which until several years ago were considered auxiliary processors for specific functions, appear to be garnering support for an entirely different type of role — high-performance computing. This is still at the discussion stage. Questions remain about the software ecosystem, or whether the chips, boards, and systems are reliable enough. And there are both business and t... » read more

ETH Zurich: PIM (Processing In Memory) Architecture, UPMEM & PrIM Benchmarks
By Technical Paper Link - 17 Jun, 2022 - Comments: 0

New paper technical titled "Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecture" led by researchers at ETH Zurich. Researchers provide a comprehensive analysis of the first publicly-available real-world PIM architecture, UPMEM, and introduce PrIM (Processing-In-Memory benchmarks), a benchmark suite of 16 workloads from different application domai... » read more

Why Comparing Processors Is So Difficult
By Brian Bailey - 10 Mar, 2022 - Comments: 2

Every new processor claims to be the fastest, the cheapest, or the most power frugal, but how those claims are measured and the supporting information can range from very useful to irrelevant. The chip industry is struggling far more than in the past to provide informative metrics. Twenty years ago, it was relatively easy to measure processor performance. It was a combination of the rate at ... » read more

Is Programmable Overhead Worth The Cost?
By Brian Bailey - 13 Jan, 2022 - Comments: 0

Programmability has fueled the growth of most semiconductor products, but how much does it actually cost? And is that cost worth it? The answer is more complicated than a simple efficiency formula. It can vary by application, by maturity of technology in a particular market, and in the context of much larger systems. What's considered important for one design may be very different for anothe... » read more

How To Measure ML Model Accuracy
By Bryon Moyer - 01 Apr, 2021 - Comments: 0

Machine learning (ML) is about making predictions about new data based on old data. The quality of any machine-learning algorithm is ultimately determined by the quality of those predictions. However, there is no one universal way to measure that quality across all ML applications, and that has broad implications for the value and usefulness of machine learning. “Every industry, every d... » read more

Customized Micro-Benchmarks For HW/SW Performance
By Ed Sperling - 23 Mar, 2021 - Comments: 0

Raw performance used to be the main focus of benchmarks, but they may have outlived their usefulness for many applications. Dana McCarty, vice president of sales and marketing for AI Inference Products at Flex Logix, talks about why companies need to develop and utilize their own specific models to accurately gauge hardware and software performance, which can be slowed by bottlenecks in I/O and... » read more

← Older posts

  • Sponsors

About

Navigation

Connect With Us

Copyright ©2013-2024 SMG   |  Terms of Service  |  Privacy Policy