tag: NPU

The Coming NPU Population Collapse
By Steve Roddy - 12 Jun, 2025 - Comments: 0

At some point in everyone’s teenage years of schooling we were all taught in a nature or biology class about cycles of population surges and then inevitable population collapses. Whether the example was an animal, plant, insect or even bacteria, some external event triggers a rapid surge in the population of a species which leads to overpopulation and competition for resources (food, space, s... » read more

Trapped By Legacy
By Steve Roddy - 15 May, 2025 - Comments: 0

At Quadric, we do a lot of first-time introductory visits with prospective new customers. As a rapidly expanding processor IP licensing company that is starting to get noticed (even winning IP Product of the Year!) such meetings are part of the territory. Which means we hear a lot of similar-sounding questions from appropriately skeptical listeners who hear our story for the very first time. Th... » read more

Side-by-Side Benchmark of NPU Platforms (Imperial College London, Cambridge)
By Technical Paper Link - 05 Apr, 2025 - Comments: 0

A new technical paper titled "Benchmarking Ultra-Low-Power μNPUs" was published by researchers at Imperial College London and University of Cambridge. Abstract "Efficient on-device neural network (NN) inference has various advantages over cloud-based processing, including predictable latency, enhanced privacy, greater reliability, and reduced operating costs for vendors. This has sparked t... » read more

The Rise Of Generative AI On The Edge
By Gordon Cooper - 03 Apr, 2025 - Comments: 0

Artificial intelligence (AI) and machine learning (ML) have undergone significant transformations over the past decade. The revolution of convolutional neural networks (CNNs) and recurrent neural networks (RNNs) is evolving toward the adoption of transformers and generative AI (GenAI), marking a pivotal shift in the field. This transition is driven by the need for more accurate, efficient, and ... » read more

No Fooling With Voxel Pooling
By Steve Roddy - 13 Mar, 2025 - Comments: 0

A variety of new and complicated transformer models have emerged in the past 18 to 24 months as new “must have” networks in advanced automotive use cases. These novel architectures often introduce new network operators or novel ways of combining tensors – often from different types of sensors – in ways to enhance detection and recognition of objects in L3 / L4 / L5 ADAS and autonomous d... » read more

NPU Acceleration For Multimodal LLMs
By Pat Donnelly - 12 Dec, 2024 - Comments: 0

Transformer-based models have rapidly spread from text to speech, vision, and other modalities. This has created challenges for the development of Neural Processing Units (NPUs). NPUs must now efficiently support the computation of weights and propagation of activations through a series of attention blocks. Increasingly, NPUs must be able to process models with multiple input modalities with ac... » read more

To (B)atch Or Not To (B)atch?
By Steve Roddy - 18 Nov, 2024 - Comments: 0

When evaluating benchmark results for AI/ML processing solutions, it is very helpful to remember Shakespeare’s Hamlet, and the famous line: “To be, or not to be.” Except in this case the “B” stands for Batched. Batch size matters There are two different ways in which a machine learning inference workload can be used in a system. A particular ML graph can be used one time, preced... » read more

Reducing SoC Power With NoCs And Caches
By Andy Nightingale - 31 Oct, 2024 - Comments: 0

Today’s system-on-chip (SoC) designs face significant challenges with respect to managing and minimizing power consumption while maintaining high performance and scalability. Network-on-chip (NoC) interconnects coupled with innovative cache memories can address these competing requirements. Traditional NoCs SoCs consist of IP blocks that need to be connected. Early SoCs used bus-based archi... » read more

Can You Rely Upon Your NPU Vendor To Be Your Customers’ Data Science Team?
By Steve Roddy - 12 Sep, 2024 - Comments: 0

The biggest mistake a chip design team can make in evaluating AI acceleration options for a new SoC is to rely entirely upon spreadsheets of performance numbers from the NPU vendor without going through the exercise of porting one or more new machine learning networks themselves using the vendor toolsets. Why is this a huge red flag? Most NPU vendors tell prospective customers that (1) the v... » read more

Fantastical Creatures
By Russell Klein - 12 Aug, 2024 - Comments: 2

In my day job I work in the High-Level Synthesis group at Siemens EDA, specifically focusing on algorithm acceleration. But on the weekends, sometimes, I take on the role of amateur cryptozoologist. As many of you know, the main Siemens EDA campus sits in the shadow of Mt. Hood and the Cascade Mountain range. This is prime habitat for Sasquatch, also known as “Bigfoot”. This weekend, ar... » read more

← Older posts

  • Sponsors

About

Navigation

Connect With Us

Copyright ©2013-2025 SMG   |  Terms of Service  |  Privacy Policy