Knowledge Center
Knowledge Center

Deep Learning

Deep learning is a subset of artificial intelligence where data representation is based on multiple layers of a matrix.


The definition of deep learning remains a little fuzzy. It sits under the artificial intelligence umbrella, either alongside machine learning or as a subset. The difference is that machine learning uses algorithms developed for specific tasks. Deep learning is more of a data representation based upon multiple layers of a matrix, where each layer uses output from the previous layer as input. The approach more closely mimic the activity of the human brain, which can tell not only that a baseball is in motion, but approximately where it will land.

Yet behind all of this there is no consensus about exactly how deep learning works, particularly as it moves from training to inferencing. Deep learning is more of mathematical distribution for complex behavior. To achieve that representation, and to shape it, there are a number of architectures being utilized. Deep neural networks and convolutional neural networks are the most common. Recurrent neural networks are being used, as well, which add the dimension of time. The downside of RNNs is the immense amount of processing, memory and storage requirements, which limits its use to large data centers.

Geoffrey Hinton, a British computer scientist and cognitive psychologist, also has been pushing the idea of capsule networks, which stack up layers inside of layers of a neural network, basically increasing the density of those layers. The result, according to Hinton, is much better because it recognizes highly overlapping digits. And this is one of the themes that runs through much of the research today-how to speed up this whole process.

The problem is so complex that it is well beyond the capability of the human brain to figure out everything, so all of this has to be modeled or theorized. For chipmakers, this is nothing new. Ever since chips reached the 1 micron process node, it became hard to visualize all of the different pieces in a design. But in computer science, many advancements have been largely two dimensional. Rotating or tilting objects is much more difficult to represent mathematically, and that requires a lot of compute resources. In the interest of speed and efficiency, researchers are trying to figure out ways to prune those computations. Still, it’s a big challenge, and one that is largely opaque to deep learning experts.

Like AI and machine learning, deep learning has been kicking around in research for decades. What’s changing is that it is being added into many types of chips, from data centers to simple microcontrollers. And as algorithms become more efficient for both training and inferencing, this part of the machine learning/AI continuum is beginning to show up across a wide spectrum of use models, some for very narrow applications and some for much broader contextual decisions.