Architecture, Materials And Software

AI, machine learning and autonomous vehicles will require massive improvements in performance, at the same power consumption level (or better), over today’s chips. But it’s obvious that the usual approach of shrinking features to improve power/performance isn’t going to be sufficient.

Scaling will certainly help, particularly on the logic side. More transistors are needed to process a huge increase in data, particularly in edge devices and in the cloud. But scaling by itself isn’t sufficient. It’s impossible to fit enough memory on single chip of reasonable size to achieve the kind of performance required for some of these new applications, and it’s likewise impossible to move that data around a single chip fast enough in a 7/5nm chip.

Advanced packaging offers a partial solution. Interposers, direct bonding and through-die TSVs certainly provide the necessary bandwidth, both between logic and memory and within the memory itself. HBM 2/3/3+/4 will provide massive throughput compared to what’s possible today, and new phase-change memories such as 3D XPoint can help fill in the gap between DRAM and SRAM.

New materials also will be required to make this shift possible, most of which are man-made rather than naturally occurring. There is a huge effort underway in the materials space, both to improve the flow of electrons with III-V materials, as well as to improve the purity of all of these materials. Impurities in manufacturing gases can cause small defects in production that can reduce performance and reliability over time, even if they fall under the radar of advanced inspection equipment.

The third and perhaps biggest knob to turn is in the area of hardware-software co-design. The current terminology is software-defined hardware, which basically means that the software is the starting point to improve efficiency rather than the other way around, but this is still an iterative process. The fact that chip design and manufacturing has reached a point where this is even possible is a huge leap forward, because in the past it was hard enough just to be able to manufacture a chip with sufficient yield. Still, the idea is roughly the same, meaning that hardware and software will be much more closely aligned to improved efficiency.

That works particularly well for algorithms, which can be much more closely matched to the hardware than operating systems and middleware. But it’s the OSes and middleware that really need work. Decades of backward compatibility, patches upon patches, layered-on security, and general-purpose APIs have turned software into a bloated, inefficient mess. The fact that it works is admirable, given all of this complexity. But cleaning up software, or better still developing new software with efficient execution and security included in the architecture from the beginning, would result in much faster, far more efficient devices.

There also is a bunch of new stuff on the horizon that can boost the power/performance equation even further—quantum computing in the cloud, silicon photonics at the edge, and far more efficient algorithms that run directly on the hardware.

Much of this technology has been developed independently but it increasingly appears that no one approach will be sufficient. Achieving the power and performance goals to provide AI at your fingertips and massive data crunching in the cloud will require all of these approaches and technologies—and more.

In the competitive quest toward one-upmanship, it increasingly appears that everyone might have been on the right track in developing all of these technologies and approaches. And while that’s all good, it’s also a rather unexpected outcome.