Engineering Within Constraints

One of the themes of DAC this year was the next phase of machine learning. It is as if CNNs and RNNs officially have migrated from the research community and all that is left now is optimization. The academics need something new. Quite correctly, they have identified power as the biggest problem associated with learning and inferencing today, and a large part of that problem is associated with memory access forced upon all architectures by the memory architectures imposed by the von Neumann architecture that propelled CPUs for the past 50 years.

Academics have been looking at the human brain for inspiration, noting that pulsing networks are closer to the way that the brain works than large matrix manipulations against a bank of stored weights, which are at the heart of systems today. Pulses fire when something important changes and do not require completely new images or other sensor data every time the equivalent of a clock fires. Early work shows that these approaches can be 20X to 50X more power efficient.

But those directions are not a certainty. Part of the problem is that while they may be able to show such advances, they are not part of a flow. In addition, the tooling is not in place to make them practical alternatives today. Some research has gone even further and prototyped systems. But then we find, at the end of the day, that we have to fabricate such systems using the same technology we do for general purpose computing, and that places constraints on what can be done and forces certain decisions to be made.

Other researchers point out we often have settled on solutions that do not mimic bodily functions. One such example is that ever since the industrial revolution, motion has been based on motors, cogs and other things that rely on circular motion — nothing like the way that bodies work. Circular components are easy to control, especially in fabrication, and allow for a repeatable manufacturing process. For most applications, this is unlikely to change.

But when power becomes more important than cost or accuracy, other technologies do provide benefits, and these do look a lot more like the systems we find in bodies. Hydraulics are used for pretty much all construction equipment and applications that are associated with large weights or forces. When accuracy is also a requirement feedback systems can be put in place, but most of the time this is left as a function of the operator. Engineering solutions do not have to accept an all-or-nothing solution, unlike most lifeforms where having that kind of redundancy would be extremely wasteful. We do see it some areas, such as teeth, which are quite specialized within the animal kingdom. Even in the human mouth we have three different kinds of teeth, each optimized for a particular function.

Both sides of these arguments are right. It is the role of engineering to provide the best solutions given the constraints. At the same time, it is the role of scientists to identify new methods that may provide advantages in the long run, or which may help us to overcome limitations with the existing approaches. Existing machine learning techniques have already encountered problems with power and many applications are pushing toward bigger and more complex problems that will require even more compute horsepower.

We, as a community, cannot sit back and accept the extreme amounts of energy being wasted in these systems, and we should welcome such research. We also should be open to finding ways to tie those into the existing infrastructures. But the problem here is that the further we go down into a rat-hole, the more expensive it becomes to get out and start a new hole.

The adoption of machine learning has happened as fast as it has because a few companies decided to invest huge amounts of money building the infrastructure to get it into production systems quickly. They did that because they had a financial incentive. Researchers need to keep that in mind, and they have to identify who will gain financially. I know that is an ugly thought, but that is the modern-day reality. Science that doesn’t make the transition to engineering just remains in textbooks, but if they can find the right industrial sponsors, they will help them transfer their breakthroughs from science to engineering.