Semiconductors And The Climate Curve

On July 22 I participated in a panel at the virtual SEMICON West conference called “Bending the Climate Curve: Enabling Sustainable Growth of Big Data, AI, and Cloud Computing.” Virtual conferences are mandatory these days, but give a different experience than physical ones. They are very good at disseminating information and are reasonably effective at networking. But, in my experience, they are still missing the chance encounters in the hall that makes physical events so worthwhile. The prep work is also very different. Unlike the corporate video calls from our makeshift offices that we have all gotten used to during the pandemic, the audio and visual prep for this panel involved an impressive collection of cameras and lights that came in a couple of big black cases. So conspicuous that the delivery guy said, “Here is your James Bond kit”, when he dropped it off. I was also given detailed instructions on what should and should not be visible in the background. Delegates were stuck with me in the foreground. As you can see in the screenshot, I was well lit, and the background was non-distracting – ideal given the importance of the panel subject matter.

Our panel discussion followed a keynote by former US Vice President Al Gore, where he talked about climate science and the damage we have already done to the planet. The panel was put together to address two specific aspects of climate change:

• How much of our current situation has been brought on by the recent upsurge in cloud computing, specifically around data analytics and artificial intelligence?

And,

• In which ways can we expect the semiconductor industry to bend the curve by enabling ever-more efficient computing systems, especially in light of the observed slowdown in Moore’s Law?

The panel was moderated by Eric Masanet of UC Santa Barbara. The other panelists were Samantha Alt of Intel, Nicola Peill-Moelter of VMWare, Moe Tanabian of Microsoft, Ellie Yieh of Applied Materials, and Cliff Young of Google. Eric began the discussion by pointing out that the combined energy footprint of the world’s data centers, networks, and devices is around 6% of global electricity use. That demand for data is growing rapidly. He mentioned that some studies have extrapolated from existing trends to predict that this growth in data volume leads to ever-increasing and unsustainable electricity consumption. He also talked about some work he and others had done in conjunction with the Lawrence Berkeley National Laboratory, showing that data center energy use has plateaued. This is even while computation has grown, due to corresponding increases in energy efficiency.

The discussion soon turned to how much room for improvement remains in processors and their underlying manufacturing technologies.

I highlighted three key points: First, energy-efficiency gains with conventional transistor scaling are slowing, and whether Moore’s Law has another 2X or 10X or more left to go, the runway is not infinite. Efficiency gains in analytics and artificial intelligence, now and in the future, are coming from improvements in algorithms and microarchitecture. These are using approaches designed to get the best answer achievable within a fixed energy envelope, rather than a theoretical best possible answer. Second, we are seeing the beginnings of improvements due to die stacking and advanced packaging. The solutions coming to market now are still largely two-dimensional in their overall connectivity architecture. However, true 3D connections between chips allow for many more connections at much lower capacitance, reducing energy while increasing performance, which brings us to the third point. Moving large amounts of memory so that it is physically close to high performance compute has the potential to dramatically improve system energy efficiency by reducing data transport cost, even without further Moore-style device scaling.

Other panelists agreed, sharing their thoughts on possible options to continue to achieve processor improvements. Ellie Yieh talked about non-volatile memory technologies such as MRAM to reduce data storage energy. Samantha Alt talked about architecting cloud computing centers to time their operations to availability of renewable power sources. Cliff Young elaborated on the ways that moving data is a huge inefficiency. Moe Tanabian argued for edge computing and ultra-low-power customized chips, and Nicola Peill-Moelter talked about the efficiency gains that virtualization provides.

Going back to the first of the two questions above, the consensus of the panel was that data analytics and artificial intelligence were measurable contributors to global energy use, continuous improvements in technology were keeping their effects under control. The second, and key takeaway from the panel, was that while there were plenty of improvements to be had throughout the ecosystem. The biggest gains would come from looking at energy efficiency at a broad level and co-developing solutions that span multiple layers of the system.

What would that look like in practice? A clear place to start is reducing data movement – do not send data to the cloud if it can be processed locally, in an edge server, for example, but perhaps in a gateway device or even right at the sensor. Adding an accelerator to a microcontroller may increase the energy devoted to computation, but dramatically reduces energy going into radio transmission or flash access. Data movement can also be reduced at the chip level by die stacking, improved memory system architecture, and compute-near-memory techniques at multiple levels within the memory hierarchy. Thinking across abstraction layers and being willing to change our assumptions about where and how information needs to be processed and stored can provide us and our industry with opportunities to bend the climate curve. Arm Research is working to do just that, and we hope you are too. Listen to the following full panel talk below!