Challenges Of Edge AI Inference

By Vinay Mehta - 01 Jul, 2021 - Comments: 0

Bringing convolutional neural networks (CNNs) to your industry—whether it be medical imaging, robotics, or some other vision application entirely—has the potential to enable new functionalities and reduce the compute requirements for existing workloads. This is because a single CNN can replace more computationally expensive image processing, denoising, and object detection algorithms. Howev... » read more

Integrating Embedded FPGA Made Easy

By Andy Jaros - 03 Jun, 2021 - Comments: 0

Chip designers have been integrating hard and soft IPs for decades – some being easy to integrate and others much more difficult. But what about eFPGA? It’s a relatively new IP on the IP landscape and according to data from Gartner, the market share of semiconductors with eFPGA is expected to approach $10B in 2023 with greater than 50% compounded annual growth. So, this raises the question ... » read more

Why Reconfigurability Is Essential For AI Edge Inference Throughput

By Vinay Mehta - 06 May, 2021 - Comments: 0

For a neural network to run at its fastest, the underlying hardware must run efficiently on all layers. Through the inference of any CNN—whether it be based on an architecture such as YOLO, ResNet, or Inception—the workload regularly shifts from being bottlenecked by memory to being bottlenecked by compute resources. You can think of each convolutional layer as its own mini-workload, and so... » read more

Maximizing Edge AI Performance

By Vinay Mehta - 01 Apr, 2021 - Comments: 0

Inference of convolutional neural network models is algorithmically straightforward, but to get the fastest performance for your application there are a few pitfalls to keep in mind when deploying. A number of factors make efficient inference difficult, which we will first step through before diving into specific solutions to address and resolve each. By the end of this article, you will be arm... » read more

The Best AI Edge Inference Benchmark

By Dana McCarty - 04 Mar, 2021 - Comments: 0

When evaluating the performance of an AI accelerator, there’s a range of methodologies available to you. In this article, we’ll discuss some of the different ways to structure your benchmark research before moving forward with an evaluation that directly runs your own model. Just like when buying a car, research will only get you so far before you need to get behind the wheel and give your ... » read more

Edge Inference Applications And Market Segmentation

By Geoff Tate - 03 Dec, 2020 - Comments: 0

Until recently, most AI was in data centers/cloud and most of that was training. Things are changing quickly. Projections are AI sales will grow rapidly to tens of billions of dollars by the mid 2020s, with most of the growth in edge AI inference. Data center/cloud vs. edge inference: What’s the difference? The data center/cloud is where inference started on Xeons. To gain efficiency, much ... » read more

ResNet-50 Does Not Predict Inference Throughput For MegaPixel Neural Network Models

By Geoff Tate - 05 Nov, 2020 - Comments: 1

Customers are considering applications for AI inference and want to evaluate multiple inference accelerators. As we discussed last month, TOPS do NOT correlate with inference throughput and you should use real neural network models to benchmark accelerators. So is ResNet-50 a good benchmark for evaluating relative performance of inference accelerators? If your application is going to p... » read more

One More Time: TOPS Do Not Predict Inference Throughput

By Geoff Tate - 08 Oct, 2020 - Comments: 0

Many times you’ll hear vendors talking about how many TOPS their chip has and imply that more TOPS means better inference performance. If you use TOPS to pick your AI inference chip, you will likely not be happy with what you get. Recently, Vivienne Sze, a professor at MIT, gave an excellent talk entitled “How to Evaluate Efficient Deep Neural Network Approaches.” Slides are also av... » read more

Apples, Oranges & The Optimal AI Inference Accelerator

By Geoff Tate - 03 Sep, 2020 - Comments: 0

There are a wide range of AI inference accelerators available and a wide range of applications for them. No AI inference accelerator will be optimal for every application. For example, a data center class accelerator almost certainly will be too big, burn too much power, and cost too much for most edge applications. And an accelerator optimal for key word recognition won’t have the capabil... » read more

Performance Metrics For Convolutional Neural Network Accelerators

By Vinay Mehta - 06 Aug, 2020 - Comments: 0

Across the industry, there are few benchmarks that customers and potential end users can employ to evaluate an inference acceleration solution end-to-end. Early on in this space, the performance of an accelerator was measured as a single number: TOPs. However, the limitations of using a single number has been covered in detail in the past by previous blogs. Nevertheless, if the method of cal... » read more

← Older posts Newer posts →

Knowledge Centers
Entities, people and technologies explored

Shift Left Is The Tip Of The Iceberg

A transformative change is underway for semiconductor design and EDA. New languages, models, and abstractions will need to be created.

by Brian Bailey

Partitioning In The Chiplet Era

Understanding how chiplets interact under different workloads is critical to ensuring signal integrity and optimal performance in heterogeneous designs.

by Ann Mutschler

3.5D: The Great Compromise

Pros and cons of a middle-ground chiplet assembly that combines 2.5D and 3D-IC.

by Ed Sperling

AI’s Role In Chip Design Widens, Drawing In New Startups

Focus is on letting engineers do much more with the same or fewer resources — and less drudgery.

by Karen Heyman

What Comes After HBM For Chiplets

The standard for high-bandwidth memory limits design freedom at many levels, but that is required for interoperability. What freedoms can be taken from other functions to make chiplets possible?

by Brian Bailey

Memory Fundamentals For Engineers

eBook: Nearly everything you need to know about memory, including detailed explanations of the different types of memory; how and where these are used today; what's changing, which memories are successful and which ones might be in the future; and the limitations of each memory type.

by The SE Staff

Why Small Fab And Assembly Houses Are Thriving

Megafabs churning out the most advanced processors are not the only game in town.

by Bryon Moyer

New AI Processors Architectures Balance Speed With Efficiency

Hot Chips 24: Large language models ratchet up pressure for sustainable computing and heterogeneous integration; data management becomes key differentiator.