tag: inference

Nightmare Fuel: The Hazards Of ML Hardware Accelerators
By Steve Roddy - 13 Apr, 2023 - Comments: 0

A major design challenge facing numerous silicon design teams in 2023 is building the right amount of machine learning (ML) performance capability into today’s silicon tape out in anticipation of what the state of the art (SOTA) ML inference models will look like in 2026 and beyond when that silicon will be used in devices in volume production. Given the continuing rapid rate of change in mac... » read more

HW-SW Co-Design Solution For Building Side-Channel-Protected ML Hardware
By Technical Paper Link - 19 Feb, 2023 - Comments: 0

A technical paper titled "Hardware-Software Co-design for Side-Channel Protected Neural Network Inference" was published (preprint) by researchers at North Carolina State University and Intel. Abstract "Physical side-channel attacks are a major threat to stealing confidential data from devices. There has been a recent surge in such attacks on edge machine learning (ML) hardware to extract the... » read more

Will Floating Point 8 Solve AI/ML Overhead?
By Karen Heyman - 12 Jan, 2023 - Comments: 2

While the media buzzes about the Turing Test-busting results of ChatGPT, engineers are focused on the hardware challenges of running large language models and other deep learning networks. High on the ML punch list is how to run models more efficiently using less power, especially in critical applications like self-driving vehicles where latency becomes a matter of life or death. AI already ... » read more

Don’t Let Your ML Accelerator Vendor Tell You The ‘F-Word’
By Steve Roddy - 08 Dec, 2022 - Comments: 0

Machine learning (ML) inference in devices is all the rage. Nearly every new system on chip (SoC) design start for mobile phones, tablets, smart security cameras, automotive applications, wireless systems, and more has a requirement for a hefty amount of ML capability on-chip. That has silicon design teams scrambling to find ML processing power to add to the existing menu of processing engines ... » read more

GDDR6 Memory Enables High-Performance AI/ML Inference
By Frank Ferro - 10 Nov, 2022 - Comments: 0

A rapid rise in the size and sophistication of inference models has necessitated increasingly powerful hardware deployed at the network edge and in endpoint devices. To keep these inference processors and accelerators fed with data requires a state-of-the-art memory that delivers extremely high bandwidth. This blog will explore how GDDR6 supports the memory and performance requirements of artif... » read more

Getting Better Edge Performance & Efficiency From Acceleration-Aware ML Model Design
By Vlad Bronstein - 04 Nov, 2021 - Comments: 0

The advent of machine learning techniques has benefited greatly from the use of acceleration technology such as GPUs, TPUs and FPGAs. Indeed, without the use of acceleration technology, it’s likely that machine learning would have remained in the province of academia and not had the impact that it is having in our world today. Clearly, machine learning has become an important tool for solving... » read more

Tradeoffs Between Edge Vs. Cloud
By Brian Bailey - 09 Sep, 2021 - Comments: 0

Increasing amounts of processing are being done on the edge, but how the balance will change between what's computed in the cloud versus the edge remains unclear. The answer may depend as much on the value of data and other commercial reasons as on technical limitations. The pendulum has been swinging between doing all processing in the cloud to doing increasing amounts of processing at the ... » read more

GDDR6 Memory On The Leading Edge
By Frank Ferro - 12 Aug, 2021 - Comments: 0

With the accelerating growth in data traffic, it is unsurprising that the number of hyperscale data centers keeps rocketing skyward. According to analysts at the Synergy Research Group, in nine months (Q2’20 to Q1’21), 84 new hyperscale data centers came online bringing the total worldwide to 625. Hyperscaler capex set a record $150B over the last four quarters eclipsing the $121B spent in ... » read more

RaPiD: AI Accelerator for Ultra-low Precision Training and Inference
By Technical Paper Link - 02 Jul, 2021 - Comments: 0

Abstract—"The growing prevalence and computational demands of Artificial Intelligence (AI) workloads has led to widespread use of hardware accelerators in their execution. Scaling the performance of AI accelerators across generations is pivotal to their success in commercial deployments. The intrinsic error-resilient nature of AI workloads present a unique opportunity for performance/energy i... » read more

Challenges Of Edge AI Inference
By Vinay Mehta - 01 Jul, 2021 - Comments: 0

Bringing convolutional neural networks (CNNs) to your industry—whether it be medical imaging, robotics, or some other vision application entirely—has the potential to enable new functionalities and reduce the compute requirements for existing workloads. This is because a single CNN can replace more computationally expensive image processing, denoising, and object detection algorithms. Howev... » read more

← Older posts Newer posts →

  • Sponsors

About

Navigation

Connect With Us

Copyright ©2013-2024 SMG   |  Terms of Service  |  Privacy Policy