tag: inferencing

Edge-Inference Architectures Proliferate
By Bryon Moyer - 04 Feb, 2021 - Comments: 0

First part of two parts. The second part will dive into basic architectural characteristics. The last year has seen a vast array of announcements of new machine-learning (ML) architectures for edge inference. Unburdened by the need to support training, but tasked with low latency, the devices exhibit extremely varied approaches to ML inference. “Architecture is changing both in the comp... » read more

Fast, Low-Power Inferencing
By Russell Klein - 10 Dec, 2020 - Comments: 1

Power and performance are often thought of as opposing goals, opposite sides of the same coin if you will. A system can be run really fast, but it will burn a lot of power. Ease up on the accelerator and power consumption goes down, but so does performance. Optimizing for both power and performance is challenging. Inferencing algorithms for Convolutional Neural Networks (CNN) are compute int... » read more

Difficult Memory Choices In AI Systems
By Ann Mutschler - 12 Nov, 2020 - Comments: 0

The number of memory choices and architectures is exploding, driven by the rapid evolution in AI and machine learning chips being designed for a wide range of very different end markets and systems. Models for some of these systems can range in size from 10 billion to 100 billion parameters, and they can vary greatly from one chip or application to the next. Neural network training and infer... » read more

Faster Inferencing At The Edge
By Ed Sperling - 09 Nov, 2020 - Comments: 0

Cheng Wang, senior vice president of engineering at Flex Logix, talks about inferencing at the edge, what are some of the main considerations in designing and choosing an inferencing chip, why programmability and modularity are important, and how hardware-software co-design with algorithms can improve performance and power. » read more

Neural Networks Without Matrix Math
By Katherine Derbyshire - 17 Sep, 2020 - Comments: 0

The challenge of speeding up AI systems typically means adding more processing elements and pruning the algorithms, but those approaches aren't the only path forward. Almost all commercial machine learning applications depend on artificial neural networks, which are trained using large datasets with a back-propagation algorithm. The network first analyzes a training example, typically assign... » read more

AI Inference Acceleration
By Ed Sperling - 14 Sep, 2020 - Comments: 0

Geoff Tate, CEO of Flex Logix, talks about considerations in choosing an AI inference accelerator, how that fits in with other processing elements on a chip, what tradeoffs are involved with reducing latency, and what considerations are the most important. » read more

Compiling And Optimizing Neural Nets
By Bryon Moyer - 10 Sep, 2020 - Comments: 0

Edge inference engines often run a slimmed-down real-time engine that interprets a neural-network model, invoking kernels as it goes. But higher performance can be achieved by pre-compiling the model and running it directly, with no interpretation — as long as the use case permits it. At compile time, optimizations are possible that wouldn’t be available if interpreting. By quantizing au... » read more

From Data Center To End Device: AI/ML Inferencing With GDDR6
By Rambus - 09 Sep, 2020 - Comments: 0

Created to support 3D gaming on consoles and PCs, GDDR packs performance that makes it an ideal solution for AI/ML inferencing. As inferencing migrates from the heart of the data center to the network edge, and ultimately to a broad range of AI-powered IoT devices, GDDR memory’s combination of high bandwidth, low latency, power efficiency and suitability for high-volume applications will be i... » read more

eFPGAs Vs. FPGA Chiplets
By Ed Sperling - 22 Jul, 2020 - Comments: 0

Embedded FPGAs are a totally different concept from discrete FPGA chiplets, and that is reflected in size, cost, power and performance. Geoff Tate, CEO of Flex Logix, talks about which applications are best for each, how each maximizes power and performance, and why choices will vary greatly by application. Related eFPGA Knowledge Center FPGA Knowledge Center Increasing EFPGA Densit... » read more

The Murky World Of AI Benchmarks
By Ann Mutschler - 30 Apr, 2020 - Comments: 0

AI startup companies have been emerging at breakneck speed for the past few years, all the while touting TOPS benchmark data. But what does it really mean and does a TOPS number apply across every application? Answer: It depends on a variety of factors. Historically, every class of design has used some kind of standard benchmark for both product development and positioning. For example, SPEC... » read more

← Older posts Newer posts →

  • Sponsors

About

Navigation

Connect With Us

Copyright ©2013-2026 SMG   |  Terms of Service  |  Privacy Policy