Less Food, More Thought

A trillion “things” are expected to be connected to the Internet sometime in the next decade. No matter how power-efficient these things are, they probably will require enough coin-sized lithium batteries to drain the world’s supply of element No. 3 on the Periodic Table. They also will increase the demand for power everywhere, and that’s even before tacking on electric vehicles, the edge, robots, and all of the AI systems that are expected to proliferate in places where processing has never existed in the past.

While much attention has been paid to orders of magnitude improvements in performance with some of the new AI and heterogeneous designs, work is underway to provide commensurate improvements in power efficiency once these devices head toward production. The initial versions are designed to work as fast as possible. The next versions are expected to include sparser algorithms, more hardware-software co-design to optimize how those algorithms run on the hardware, and even shorter distances to memory, which will provide further improvements in performance and huge improvements in power efficiency.

None of this is for free, though, and there are several major challenges to making it all work.

First, data needs to be partitioned based on a consistent hierarchy for each application area. One size does not fit all, and it may not suffice for the same application over time, so it needs to be highly programmable. For instance, there’s a big difference between data that comes from a vibration sensor and streaming video data, both in terms of value per bit and in terms of volume. While every bit generated in a streaming video has value to someone, every bit generated from a camera mounted on a car windshield does not. And if you add up all of the cameras on hundreds of millions of autonomous/semi-autonomous vehicles, that’s a lot of data that needs to be processed, sorted and acted upon.

This is where things get really difficult. Determining what has value and to whom, and whether it needs an immediate response, is a difficult problem for which no one has a good answer yet. This is a massive and complex data partitioning problem, and it requires understanding the data as well as the quality of the sensors generating that data at the moment the data was created, not to mention the potential for interruptions in that data from various forms of noise. Put simply, if two cars or robots react differently to the same data, or if two vision sensors record an event differently, bad things can happen.

Second, data needs to be secure, whether it’s in place or in motion. That requires layers of security, as well as redundancy in both hardware and software. This is system-level margin, and margin requires extra circuitry, more power and it impacts performance—at least the way it has been implemented so far. The key questions here are how much security is required and how will that security will be implemented. Security typically works best when compartmentalized into layers, from hardware boot-up to on-chip monitoring of any activity all the way up to the passwords used to secure access to software. None of that is free from a power/performance standpoint, and no single approach is sufficient.

The third challenge is understanding where silos exist within chipmakers, IP companies, and system companies, and how to abstract beyond them. Power, performance and area are system-wide issues, and they affect every step in the flow and every business unit within a company. While those silos may still make sense from a business standpoint, data needs to flow in and out of these silos seamlessly in order to develop chips, packages and entire systems that are efficient, extremely fast, and capable of handling an almost endless barrage of engineering change orders. That also means the people within those silos need to recognize they are part of a bigger organization, and the only way to make that work effectively is from the bottom up, with total buy-in from the very top of these organizations.

Energy consumption needs to go down and performance needs to go up by orders of magnitude, and that will require new architectures, business processes and technology — and a recognition that big changes will be required to make all of this work.