Inside Neuromorphic Computing

Semiconductor Engineering sat down to talk about neuromorphic technology with Guy Paillet, chief executive of General Vision. The fabless IC design house is a pioneer and supplier of neuromorphic chips. What follows are excerpts of that conversation.

SE: In 1993, you invented and co-patented a neural networking chip with IBM. Then, you joined General Vision in 1999. Briefly tell us about General Vision.

Paillet: At General Vision, we have been delivering neural network chips since about 2007. We recently licensed our technology to Intel. Intel is starting to ship the Curie Module with our neurons inside.

SE: Neuromorphic computing and its previous incarnation, neural networks, isn’t a new idea. The idea is to take specialized chips and develop systems, which are inspired by the computational functions of the brain. How does this work?

Paillet: First, we can start with the definition of a neuron. A neuron is something which we call reactive memory. It actually does react to a stimulus that it has seen or learned previously. This reactive memory is different and the opposite of every single memory type today, except for content addressable memory. When you have a lot of neurons connected together, let’s call it a neural network.

SE: Can you elaborate?

Paillet: For example, at General Vision, our chip features 1,024 neurons, all interconnected and working in parallel. A neuron is basically a 256-byte memory, with about 3,000 logic gates associated with the memory. Typically, a neuron is really a storage element. For us, it’s volatile. It’s an SRAM. We can save the content of the neuron and restore it very quickly.

SE: So each device has 1,024 neurons. Multiple chips can be daisy-chained to form the basis of a neural network. What does your chip do?

Paillet: It can do parallel pattern matching. One of our features is to be able to match one pattern versus many patterns in a constant time. It can be 100,000, a million or more patterns. This is 30 clock cycles regardless of the number.

SE: How does your chip and a neural network work?

Paillet: Let’s say we have a demonstration where we are recognizing Chinese characters. In this demonstration, you have 300 Chinese characters. You write a block of pixels into the input of a neuron, which corresponds to an unknown Chinese character. All of the neurons in the device are going to actually evaluate it. So this block of pixels is going to be broadcast to all of the neurons at the same time. All of the neurons are going to evaluate the similarity of the incoming block of pixels. And the winner is going to shut down every other neuron candidate and say, ‘I am the closest one.’ It comes down to a winner-takes-all methodology.

SE: So a neuron in a neuromorphic chip is far different from a neuron in the brain, right?

Paillet: Our neurons are distant cousins from a biological neuron. It’s like comparing a robot arm to the human arm. To this extent, a robot arm can be more powerful than a human arm in terms of strength and so on, but not as agile as a human arm. Here, we are in the same situation. We can recognize billions of patterns per second with a chip, but my cat can do better.

SE: Can you do pattern matching with today’s computers rather than with neuromorphic technology?

Paillet: Today, computing is not good for technologies like parallel pattern recognition, but they are good for a lot of other things.

SE: What’s the problem using today’s systems for pattern matching? For that matter, what’s the problem with today’s computing in general?

Paillet: Basically, there are things you can try to do with von Neumann computation and software. But you are actually burning a lot of power just reading the instruction manual. By that, I mean moving the data back and forth from the storage. Plus, every single memory that is used in the computer industry today is like a mailbox. You need an address. You take the mail and put it into the mailbox. If you don’t have the right address, you are lost.

SE: What else?

Paillet: Everything can be done in software more or less. It’s just not that efficient. In addition, everybody today wants the IoT and low-power processing. But if you have to run a system at 1 gigahertz just to read the instructions and move the data back and forth, you are wasting 60% to 70% of the power. And therefore, your battery life is short or dies. So I am not saying a computer, a CPU, a GPU and all can’t do it. But they are not very efficient.

SE: As a result, some believe the industry needs a new paradigm in computing. Neuromorphic computing is one idea. This is not a new field, right?

Paillet: In the 1980s, everyone was very hopeful about neural networks. But then it went into what we call the dark ages. Now you have people beating their chest and saying they found something new. This something new is the same thing the industry talked about in the 1980s. Some call it deep learning. It’s basically the same recipe called the perceptron.

SE: What’s changed?

Paillet: We have brand new hardware. It is faster, but it consumes more power.

SE: How many companies are shipping neuromorphic chips today?

Paillet: This has to be verified, but I believe we are the only company in the world that is shipping. We also have a licensee in Intel, which is also shipping now. There are a lot of other companies trying to do something.

SE: General Vision is shipping chips based loosely on a hardware-based artificial neural network approach. But others are struggling to ship neuromorphic chips. What’s holding the industry back?

Paillet: A lot of people are trying to replicate the brain in silicon. Some people believe it’s important to be very close to the biology. They want to mimic the way that information is moving from a group of neurons to another group of neurons using something called spiking. Is that better? This is exactly the same way of thinking as putting legs on your car or having flapping wings on a plane. You are closer to the biological model, but it’s totally inefficient.

SE: General Vision’s chip solves pattern recognition problems. What do you mean by pattern recognition?

Paillet: Pattern recognition is a generic thing. A pattern is a block of pixels or a signal. It could also be a noise, voice, bio-signal or vibration. It could also be a pattern for deep packet inspection. So for example, people are using it for inspecting a specific pattern in a TCP/IP packet.

SE: Do your chips need to be programmed?

Paillet: There is no programming at all. We don’t have a programming language. You just input a signature.

SE: Your chip is based on a 130nm process. Is there any need to scale the device to 14nm and beyond?

Paillet: There is no urgency for us to do this. These are very expensive technologies.

SE: What’s next for your chip?

Paillet: Our next push will be for better memory technology. And also, we want to reduce the gate count.

SE: Does neuromorphic chip technology follow Moore’s Law? Do you have to double the number of neurons every two years?

Paillet: I don’t think so.

SE: What were some of the first applications for your chips?

Paillet: Vision was the first application. For example, we have neural networks that have been inspecting fish on Norwegian and Icelandic boats for some time. The fishermen have trained themselves to do this. There is no software involved on the boat or on shore. It’s a simple application. They have big fishing boats. They are cutting and packaging the fish on board. The problem was that people were watching the fish all day passing by on a conveyor belt. This was to make sure the fish are the right species, not damaged, and in the right orientation to be processed. By replacing the people with sensors and neurons, and using pattern matching, they can run six fish per second on a conveyor belt. They have been saving $2 million per year per boat since 2003. They have 50 systems installed now.

SE: What are the other applications?

Paillet: We are interested in artificial vision. For us, medical imaging is a strong thrust. Factory automation is a big market, but the volumes might be lower. Then, consumer is a big market for products like educational toys. You also have the automotive market, with the advanced driver assistance technology. It’s going to be a good market one day. For now, the barrier of entry is very difficult because there are so many regulations. It’s basically an R&D market.

SE: What about any other apps?

Paillet: The security market with security cameras and other products is a big market. The biggest problem with pattern recognition is that it’s basically a decision-making process. For example, a system must decide whether a bad or good guy is on your property. But if it makes a mistake, you can pay a penalty for it.

SE: How can an OEM integrate your technology?

Paillet: We’ve licensed our technology to the Indian government for face recognition on a Xilinx FPGA. So we can license our IP for FPGAs. It’s possible to put a decent number of neurons on an FPGA. We are delivering this IP on Xilinx and Altera. For now, our main partner has been Lattice.

SE: You also have a deal with Intel, right?

Paillet: We license them the IP. They make their own chip. It’s called the Curie Module. The Curie is actually the Quark SE. The Quark SE is a processor, a small DSP and what we call a pattern-matching accelerator, which is based on our neurons.

SE: What can your IP do for wearables?

Paillet: It does specific things. When people are jogging, for example, it detects something. It’s a little bit like Fitbit, but much more accurate.

SE: Where is neuromorphic technology heading in the future?

Paillet: A new industry that is going to come is what I call the knowledge industry. People will be able to train neurons, take the content and proliferate them, either for free or for a fee. And that will remove the big fence that exists today and being able to use artificial intelligence.

SE: What do you envision as one of the future apps for the technology?

Paillet: One day, my coffee machine would recognize me in the morning and it would say: ‘You need a strong espresso.’ Unfortunately, I can’t buy that at Best Buy yet. This is the idea. It has to be $20. If it’s a million dollars, it’s not very interesting.