How Inferencing Differs From Training in Machine Learning Applications


Machine learning (ML)-based approaches to system development employ a fundamentally different style of programming than historically used in computer science. This approach uses example data to train a model to enable the machine to learn how to perform a task. ML training is highly iterative with each new piece of training data generating trillions of operations. The iterative nature of the tr... » read more

On-Chip FPGA: The “Other” Compute Resource


When system companies discuss processing requirements for their next generation products, the typical discussion invariably leads to: what should the processor subsystem look like? Do you upgrade the embedded processors in the current subsystem to the latest and greatest embedded CPU? Do you add more CPUs? Or perhaps add a little diversity by adding a DSP or GPU? One compute resource tha... » read more

Getting Better Edge Performance & Efficiency From Acceleration-Aware ML Model Design


The advent of machine learning techniques has benefited greatly from the use of acceleration technology such as GPUs, TPUs and FPGAs. Indeed, without the use of acceleration technology, it’s likely that machine learning would have remained in the province of academia and not had the impact that it is having in our world today. Clearly, machine learning has become an important tool for solving... » read more

Make Your SoC Upgradable Like A Tesla


I’ve always been a fan of Tesla. Not for the quick acceleration, nice lines, great handling or leading the world away from the using the internal combustion engine. I’m a big fan because they plan products not just for use today, but for the future. In the not too distant past, in order to get the latest automotive technology, you’d have to buy a new car. With Tesla, you don’t have to. ... » read more

How Dynamic Hardware Efficiently Solves The Neural Network Complexity Problem


Given the high computational requirements of neural network models, efficient execution is paramount. When performed trillions of times per second even the tiniest inefficiencies are multiplied into large inefficiencies at the chip and system level. Because AI models continue to expand in complexity and size as they are asked to become more human-like in their (artificial) intelligence, it is c... » read more

Integrate FPGAs For A Customizable MCU


MCUs come in a broad range of flavors, meaning you can pick the best one for the application with the right performance, feature set, peripherals, memory, and software programmability. So, then, why do many systems also use FPGAs next to the MCUs? Usually, it’s because there’s not a “perfect” MCU for their application. MCUs by definition are built to be generic for a wide variety of app... » read more

Challenges Of Edge AI Inference


Bringing convolutional neural networks (CNNs) to your industry—whether it be medical imaging, robotics, or some other vision application entirely—has the potential to enable new functionalities and reduce the compute requirements for existing workloads. This is because a single CNN can replace more computationally expensive image processing, denoising, and object detection algorithms. Howev... » read more

Integrating Embedded FPGA Made Easy


Chip designers have been integrating hard and soft IPs for decades – some being easy to integrate and others much more difficult. But what about eFPGA? It’s a relatively new IP on the IP landscape and according to data from Gartner, the market share of semiconductors with eFPGA is expected to approach $10B in 2023 with greater than 50% compounded annual growth. So, this raises the question ... » read more

Why Reconfigurability Is Essential For AI Edge Inference Throughput


For a neural network to run at its fastest, the underlying hardware must run efficiently on all layers. Through the inference of any CNN—whether it be based on an architecture such as YOLO, ResNet, or Inception—the workload regularly shifts from being bottlenecked by memory to being bottlenecked by compute resources. You can think of each convolutional layer as its own mini-workload, and so... » read more

Maximizing Edge AI Performance


Inference of convolutional neural network models is algorithmically straightforward, but to get the fastest performance for your application there are a few pitfalls to keep in mind when deploying. A number of factors make efficient inference difficult, which we will first step through before diving into specific solutions to address and resolve each. By the end of this article, you will be arm... » read more

← Older posts Newer posts →