tag: NPU

BYO NPU Benchmarks
By Steve Roddy - 14 Dec, 2023 - Comments: 0

In our last blog post, we highlighted the ways that NPU vendors can shade the truth about performance on benchmark networks such that comparing common performance scores such as “Resnet50 Inferences / Second” can be a futile exercise. But there is a straight-forward, low-investment method for an IP evaluator to short-circuit all the vendor shenanigans and get a solid apples-to-apples result... » read more

Considerations For Accelerating On-Device Stable Diffusion Models
By Pat Donnelly - 30 Nov, 2023 - Comments: 0

One of the more powerful – and visually stunning – advances in generative AI has been the development of Stable Diffusion models. These models are used for image generation, image denoising, inpainting (reconstructing missing regions in an image), outpainting (generating new pixels that seamlessly extend an image's existing bounds), and bit diffusion. Stable Diffusion uses a type of dif... » read more

Flipping Processor Design On Its Head
By Ann Mutschler - 09 Nov, 2023 - Comments: 2

AI is changing processor design in fundamental ways, combining customized processing elements for specific AI workloads with more traditional processors for other tasks. But the tradeoffs are increasingly confusing, complex, and challenging to manage. For example, workloads can change faster than the time it takes to churn out customized designs. In addition, the AI-specific processes may ex... » read more

Does Your NPU Vendor Cheat On Benchmarks?
By Steve Roddy - 12 Oct, 2023 - Comments: 0

It is common industry practice for companies seeking to purchase semiconductor IP to begin the search by sending prospective vendors a list of questions, typically called an RFI (Request for Information) or simply a Vendor Spreadsheet. These spreadsheets contain a wide gamut of requested information ranging from background on the vendor’s financial status, leadership team, IP design practices... » read more

Your AI Chip Doesn’t Need An Expensive Insurance Policy
By Steve Roddy - 14 Sep, 2023 - Comments: 0

Imagine you are an architect designing a new SoC for an application that needs substantial machine learning inferencing horsepower. The team in marketing has given you a list of ML workloads and performance specs that you need to hit. The in-house designed NPU accelerator works well for these known workloads – things like MobileNet v2 and Resnet50. The accelerator speeds up 95+% of the comput... » read more

Compiler-Driven Performance Boosts For GPNPUs
By Steve Roddy - 10 Aug, 2023 - Comments: 0

The GNU C Compiler – GCC – was first released in 1987. 36 years ago. Several version streams are still actively being developed and enhanced, with GCC13 being the most advanced, and a GCC v10.5 released in early July this year. You might think that with 36 years of refinement by thousands of contributors that penultimate performance has been achieved. All that could be discovered has bee... » read more

A Packet-Based Architecture For Edge AI Inference
By Pat Donnelly - 27 Jul, 2023 - Comments: 0

Despite significant improvements in throughput, edge AI accelerators (Neural Processing Units, or NPUs) are still often underutilized. Inefficient management of weights and activations leads to fewer available cores utilized for multiply-accumulate (MAC) operations. Edge AI applications frequently need to run on small, low-power devices, limiting the area and power allocated for memory and comp... » read more

A Bridge From Mars To Venus
By Steve Roddy - 20 Jul, 2023 - Comments: 0

In a now-famous 1992 pop psychology book titled "Men Are from Mars, Women Are from Venus," author John Gray posited that most relationship troubles in couples stem from fundamental differences in socialization patterns between men and women. The analogy that the two partners came from different planets was used to describe how two people could perceive issues in completely different and sometim... » read more

A Buyers Guide To An NPU
By Paul Karazuba - 22 Jun, 2023 - Comments: 2

Choosing the right AI inference NPU (Neural Processing Unit) is a critical decision for a chip architect. There’s a lot at stake because as the AI landscape constantly changes, the choices will impact overall product cost, performance, and long-term viability. There are myriad options regarding system architecture and IP suppliers, and this can be daunting for even the most seasoned semicondu... » read more

(Vision) Transformers: Rise Of The Chimera
By Jon Bunting - 20 Jun, 2023 - Comments: 0

It’s 2023 and transformers are having a moment. No, I’m not talking about the latest installment of the Transformers movie franchise, "Transformers: Rise of the Beasts"; I’m talking about the deep learning model architecture class, transformers, that is fueling anticipation, excitement, fear, and investment in AI. Transformers are not so new in the world of AI anymore; they were first ... » read more

← Older posts Newer posts →

  • Sponsors

About

Navigation

Connect With Us

Copyright ©2013-2024 SMG   |  Terms of Service  |  Privacy Policy