tag: NPUs

The On-Device LLM Revolution
By Steve Roddy - 20 Feb, 2026 - Comments: 0

The AI world is experiencing a fundamental shift. After years of cloud-centric inference dominated by massive data center GPUs, we're witnessing an accelerating migration of language models to edge devices. These are not the trillion-parameter behemoths that require server farms, but the "Goldilocks zone" models: 3B to 30B parameters — large enough to deliver genuinely useful AI capabilities,... » read more

Voice is the New UI
By Cadence - 28 Jan, 2026 - Comments: 0

Recent years have seen a paradigm shift in the user interface (UI) of our computers and client devices, and this is gaining momentum. Advancements in large language models (LLM), small language models (SLM), energy-efficient systems on chip (SoC), and on-device AI processing are making voice input the new “keyboard”. Read more here.   Fig.1: Voice Processing Pipeline On-De... » read more

Balancing Training, Quantization, And Hardware Integration In NPUs
By Ann Mutschler - 26 Jan, 2026 - Comments: 0

Experts At The Table: AI/ML is driving a steep ramp in neural processing unit (NPU) design activity for everything from data centers to edge devices such as PCs and smartphones. Semiconductor Engineering sat down to discuss this with Jason Lawley, director of product marketing, AI IP at Cadence; Sharad Chole, chief scientist and co-founder at Expedera; Steve Roddy, chief marketing officer at Qu... » read more

Balancing Workloads In AI Processor Designs
By Ann Mutschler - 11 Sep, 2025 - Comments: 0

A growing number of AI processors are being designed around specific workloads rather than standardized benchmarks, optimizing performance and power efficiency, but often with enough flexibility to adapt to future changes. While the fundamentals of matrix multiplication and software optimization still apply, those alone are no longer sufficient. Designs need to address specific data types, w... » read more

Workload-Specific Hardware Accelerators
By Ed Sperling - 13 Aug, 2025 - Comments: 0

Workload-specific hardware accelerators are becoming essential in large data centers for two reasons. One is that general-purpose processing elements cannot keep up with the workload demands or latency requirements. The second is that they need to be extremely efficient due to limited electricity from the grid and the high cost of cooling these devices. Sharad Chole, chief scientist and co-foun... » read more

Chiplet-Based NPUs to Accelerate Vehicular AI Perception Workloads
By Technical Paper Link - 02 Dec, 2024 - Comments: 0

A new technical paper titled "Performance Implications of Multi-Chiplet Neural Processing Units on Autonomous Driving Perception" was published by researchers at UC Irvine. Abstract "We study the application of emerging chiplet-based Neural Processing Units to accelerate vehicular AI perception workloads in constrained automotive settings. The motivation stems from how chiplets technology i... » read more

Sensor Fusion Challenges In Automotive
By Ed Sperling - 02 May, 2024 - Comments: 0

The number of sensors in automobiles is growing rapidly alongside new safety features and increasing levels of autonomy. The challenge is integrating them in a way that makes sense, because these sensors are optimized for different types of data, sometimes with different resolution requirements even for the same type of data, and frequently with very different latency, power consumption, and re... » read more

Data Formats For Inference On The Edge
By Brian Bailey - 14 Dec, 2023 - Comments: 0

AI/ML training traditionally has been performed using floating point data formats, primarily because that is what was available. But this usually isn't a viable option for inference on the edge, where more compact data formats are needed to reduce area and power. Compact data formats use less space, which is important in edge devices, but the bigger concern is the power needed to move around... » read more

Application-Optimized Processors
By Ed Sperling - 29 Sep, 2023 - Comments: 0

Executing a neural network on top of an NPU requires an understanding of application requirements, such as latency and throughput, as well as the potential partitioning challenges. Sharad Chole, chief scientist and co-founder of Expedera, talks about fine-grained dependencies, why processing packets out of order can help optimize performance and power, and when to use voltage and frequency scal... » read more

Complex Tradeoffs In Inferencing Chips
By John Koon - 17 Oct, 2022 - Comments: 0

Designing AI/ML inferencing chips is emerging as a huge challenge due to the variety of applications and the highly specific power and performance needs for each of them. Put simply, one size does not fit all, and not all applications can afford a custom design. For example, in retail store tracking, it's acceptable to have a 5% or 10% margin of error for customers passing by a certain aisle... » read more

← Older posts Newer posts →

  • Sponsors

About

Navigation

Connect With Us

Copyright ©2013-2026 SMG   |  Terms of Service  |  Privacy Policy