Processors Are Exciting Again

Today is a very exciting time in the world of processor architectures. Domain-specific processor architectures are now fully realized as the best answers to the challenges of low power and high performance for many applications. Advancements in artificial intelligence are leading the way to exciting new experiences and products today and in our future. There have been more advances in deep learning and neural networks in the last three years than all the years before. And the innovation is continuing at a very fast pace.

What’s driving this innovation? It’s the move away from general-purpose CPUs, or at least the realization that these CPUs have their place controlling everything and running the OS, but they’re not efficient at many other tasks. We’ve known this for about 20 years, which has given rise to the GPU for graphics and specialized DSPs.

But now things are changing even more and even faster, particularly in the area of deep learning and neural networks. There are two main components of neural networks: training and inferencing. Training a neural network almost always takes place in the cloud using 32-bit floating-point data, which requires massive processing capabilities. The biggest innovations here are ways to use more compute power than ever before in large server farms to speed up training times.

Neural networks use inferencing to evaluate new data based on their training. Inferencing is used most often on the edge of the network, usually with mobile battery-operated devices, for fast response and for the lowest possible power budget. Using techniques like pruning, sparsity and quantization, these inferencing processors are now becoming very efficient. For example, we have found excellent results by reducing the precision to 8 bits and using lots of 8-bit MACs for vision and AI processing.

The pixel explosion is driving the development of innovative new vision architectures. It’s not all traditional camera operations. Radar, lidar and ultrasound are becoming more important applications used in automotive ADAS, and innovative vision applications are driving significant advancements in many fields. On-device neural network inference applications span autonomous vehicles, surveillance, robotics, drones, augmented reality/virtual reality, smartphones, smart homes and IoT. And this innovative technology is moving beyond vision to audio applications, so users can personalize their sound experience.

Often these innovative inferencing processors are very scalable, and multiple processors can be used to provide the inferences needed for real-time analysis of huge amounts of data without going back to the cloud. Autonomous vehicles, for example, cannot depend on cellular connectivity before they decide if a traffic light is red or green. And that’s just the beginning. With on-vehicle cameras, radar, lidar and ultrasound, the amount of data that needs to be quickly processed is huge.

Deep neural networks require a sophisticated, efficient software infrastructure, which is now developing. Frameworks such as Caffe, TensorFlow and TensorFlow Lite are popular. Our Tensilica Neural Network Compiler takes the output from these frameworks and maps them onto the Tensilica Vision and DNA processors, performing several optimizations and generating device-specific code. Our newly announced Tensilica DNA 100 AI processor IP delivers more than 4X performance and 2X power efficiency compared to other solutions with similar physical arrays by taking advantage of the sparsity in the neural networks and optimizing neural networks to increase sparsity.

CB Insights tracks the most promising 100 AI startups, which have raised $11.7B in equity funding across 367 deals. Yes, even investors think AI is making processors very interesting again!

And it’s not just the startups that are innovating in this space. All of the leading semiconductor companies are working on this, and they have deeper pockets than most startups. At the recent Hot Chips conference, a major theme was deep learning. NVIDIA gave a presentation on their deep learning accelerator. There were discussions on blockchain, mobile/power efficient processors, graphics processors, IoT/edge computing and server processors, and a major theme was innovation everywhere.

The upcoming Linley Processor Conference has an exciting list of presentations on these innovations, including “An Advanced DSP Architecture for Neural Networks and Audio Processing,” by Sachin Ghanekar from Cadence. Other presentations will focus on AI, machine learning, security, IoT, mobile and automotive.

I’m very excited to be working right in the middle of this major change on architectures for AI, vision processing and other complex tasks. And I can’t wait to see all the innovative products that will come out as a result of these new architectures!