tag: inference

Issues And Challenges In Super-Resolution Object Detection And Recognition
By Jeremy Roberson - 04 May, 2023 - Comments: 0

If you want high performance AI inference, such as Super-Resolution Object Detection and Recognition, in your SoC the challenge is to find a solution that can meet your needs and constraints. You need inference IP that can run the model you want at high accuracy. You need inference IP that can run the model at the frame rate you want: higher frame rate = lower latency, more time for dec... » read more

From Data Center To End Device: AI/ML Inference With GDDR6
By Rambus - 03 May, 2023 - Comments: 0

Created to support 3D gaming on consoles and PCs, GDDR packs performance that makes it an ideal solution for AI/ML inference. As inference migrates from the heart of the data center to the network edge, and ultimately to a broad range of AI-powered IoT devices, GDDR memory’s combination of high bandwidth, low latency, power efficiency and suitability for high-volume applications will be incre... » read more

Can Compute-In-Memory Bring New Benefits To Artificial Intelligence Inference?
By Paul Karazuba - 27 Apr, 2023 - Comments: 0

Compute-in-memory (CIM) is not necessarily an Artificial Intelligence (AI) solution; rather, it is a memory management solution. CIM could bring advantages to AI processing by speeding up the multiplication operation at the heart of AI model execution. However, for that to be successful, an AI processing system would need to be explicitly architected to use CIM. The change would entail a shift ... » read more

Nightmare Fuel: The Hazards Of ML Hardware Accelerators
By Steve Roddy - 13 Apr, 2023 - Comments: 0

A major design challenge facing numerous silicon design teams in 2023 is building the right amount of machine learning (ML) performance capability into today’s silicon tape out in anticipation of what the state of the art (SOTA) ML inference models will look like in 2026 and beyond when that silicon will be used in devices in volume production. Given the continuing rapid rate of change in mac... » read more

HW-SW Co-Design Solution For Building Side-Channel-Protected ML Hardware
By Technical Paper Link - 19 Feb, 2023 - Comments: 0

A technical paper titled "Hardware-Software Co-design for Side-Channel Protected Neural Network Inference" was published (preprint) by researchers at North Carolina State University and Intel. Abstract "Physical side-channel attacks are a major threat to stealing confidential data from devices. There has been a recent surge in such attacks on edge machine learning (ML) hardware to extract the... » read more

Will Floating Point 8 Solve AI/ML Overhead?
By Karen Heyman - 12 Jan, 2023 - Comments: 2

While the media buzzes about the Turing Test-busting results of ChatGPT, engineers are focused on the hardware challenges of running large language models and other deep learning networks. High on the ML punch list is how to run models more efficiently using less power, especially in critical applications like self-driving vehicles where latency becomes a matter of life or death. AI already ... » read more

Don’t Let Your ML Accelerator Vendor Tell You The ‘F-Word’
By Steve Roddy - 08 Dec, 2022 - Comments: 0

Machine learning (ML) inference in devices is all the rage. Nearly every new system on chip (SoC) design start for mobile phones, tablets, smart security cameras, automotive applications, wireless systems, and more has a requirement for a hefty amount of ML capability on-chip. That has silicon design teams scrambling to find ML processing power to add to the existing menu of processing engines ... » read more

GDDR6 Memory Enables High-Performance AI/ML Inference
By Frank Ferro - 10 Nov, 2022 - Comments: 0

A rapid rise in the size and sophistication of inference models has necessitated increasingly powerful hardware deployed at the network edge and in endpoint devices. To keep these inference processors and accelerators fed with data requires a state-of-the-art memory that delivers extremely high bandwidth. This blog will explore how GDDR6 supports the memory and performance requirements of artif... » read more

Getting Better Edge Performance & Efficiency From Acceleration-Aware ML Model Design
By Vlad Bronstein - 04 Nov, 2021 - Comments: 0

The advent of machine learning techniques has benefited greatly from the use of acceleration technology such as GPUs, TPUs and FPGAs. Indeed, without the use of acceleration technology, it’s likely that machine learning would have remained in the province of academia and not had the impact that it is having in our world today. Clearly, machine learning has become an important tool for solving... » read more

Tradeoffs Between Edge Vs. Cloud
By Brian Bailey - 09 Sep, 2021 - Comments: 0

Increasing amounts of processing are being done on the edge, but how the balance will change between what's computed in the cloud versus the edge remains unclear. The answer may depend as much on the value of data and other commercial reasons as on technical limitations. The pendulum has been swinging between doing all processing in the cloud to doing increasing amounts of processing at the ... » read more

← Older posts Newer posts →

  • Sponsors

About

Navigation

Connect With Us

Copyright ©2013-2024 SMG   |  Terms of Service  |  Privacy Policy