New challenges call for across-the-stack innovation spanning new materials and devices to novel computing models.
By Toshi Nishida, Avik W. Ghosh, Swaminathan Rajaraman, and Mircea Stan
Commercial-off-the-shelf (COTS) components have enabled a commodity market for Wi-Fi-connected appliances, consumer products, infrastructure, manufacturing, vehicles, and wearables. However, the vast majority of connected systems today are deployed at the edge of the network, near the end user or end application, opening the door for innovation at the edge in the key enabling hardware technologies: computing, sensing, wireless, power, and integration.
The edge cuts to the chase, right to the physical world that matters, where the most accurate data may be sensed and made sense of. Accuracy enables sharper focus and the potential for more intelligent decision making. Moreover, the world is changing, often rapidly and unexpectedly, frequently requiring the real-time processing of edge data without relying on the cloud, which, for many applications, may be unavailable, unreliable, or insecure.
Maximizing edge intelligence clearly requires tailored algorithms and hardware to process data intelligently in real time using machine learning (ML) and artificial intelligence (AI). Edge systems in real-world conditions need to be resilient to sub-optimal power, unreliable internet connectivity, and harsh environments. As a result, we need to revisit the fundamental duo of on-chip and mist computing and edge-of-the-node sensing, accompanied by advancements in wireless communication, power budgeting, and integration, to maximize the utility of edge intelligence.
Below, we outline some of the challenges and opportunities in the primary computing-sensing power sector, along with issues at the edge in the wireless and integration.
Enormous progress across the entire hardware-software stack has led to unprecedented advancement in machine learning, with notable entrants such as ChatGPT and Dall-E. While these applications deliver incredible capabilities, they rely on hardware architectures designed for high performance, which consequently leads to massive computing and energy requirements [R1]. Not surprisingly, the overall energy consumed by information and communication technologies relative to global power consumption from all sources is projected to double from its current share of 5-9% in the next couple of years.
However, computing at the edge is fundamentally constrained by energy; energy resources available at the edge may be scarce and/or unreliable. Recent estimates show that hardware for the edge must deliver unprecedented energy efficiency (~1000 TOPS/W; TOPS: Tera Ops per second) and performance (1 TOPS) [FE Ref]. Therefore, designing computing platforms at the edge remains an open challenge that entails across-the-stack innovation, spanning new materials and devices to novel computing models.
Artificial intelligence with ChatGPT or Dall-E owes their very existence to the power-performance superiority of advanced CMOS nodes and evolving GPU/TPU/FPGA/ASIC architectures. However, there are evolutionary tricks with the brain that we still would do well to tap into, given the energy efficiency of the latter. This becomes particularly important for applications at the edge, such as mine robots, autonomous vehicles, or Mars rovers, where a large volume of data must be processed instantly on chip with limited memory and energy budgets, with the cloud being unavailable or unreliable.
A common problem with on-chip storage is catastrophic forgetfulness (sequential data overwrite), as new data overwrites existing information, such as in the training cycles of neuromorphic chips. The brain bypasses this so-called stability-plasticity problem by using higher dimensional representations that build synthetically on past knowledge (e.g., reframing a leopard as a large spotted cat), and being particularly frugal with costly, long-distance synaptic connections.
The evolutionary trick is a dense and recurrently connected supervisory network (the hippocampus) that encodes uncorrelated initial data, and later runs a time-compressed signal to direct distant neurons in a sparsely connected neocortex (the center of associative memory) to iteratively build Hebbian (associative) connections through electro-cortical coincidence detectors. Unlike conventional silicon VLSI, these connections are rare, probabilistic, wire-on-the-fly, and externally directed, leading eventually to a network of sparsely connected cortical neurons whose firing patterns correlate uniquely with class signals, allowing a single neuron to fire for more than one class.
Along with better algorithms, there is an opportunity to develop better devices and architectures. For instance, nanomagnets are now commercially integrated onto silicon, and can act as non-volatile memory elements as well as true random number generators at very low footprint (just a few I/O transistors). They can couple wirelessly using dipolar interactions that reduce footprint, energy, and latency, and can be programmed as a coupled probabilistic network of stochastic bits to solve NP-complete optimization problems.
Their non-volatility coupled with gateability is well suited for PiM architectures where data processing (P) happens naturally in or near memory (iM), reducing the so-called Von Neumann bottleneck with shuttling data between the processor and memory cores in conventional architectures. In an analog embodiment, neurons can be wired into an echo-state network, to process on-chip time-series data in hardware as a nonlinear signal filter, such as needed for real-time trajectory prediction.
At the same time CMOS-based oscillators, based on mature, time-tested technology, trade non-volatility for accuracy and can be readily scaled for size. Perhaps a hybrid technology combining short range non-volatile memory tiles with CMOS based inter-tile matrix would ultimately be of practical use.
While readily available commercial sensors may be adequate for consumer IoT systems, the physical world for many edge applications can be harsh and involve one or more extremes in temperature, pressure, liquids and radiation. Important application markets such as aerospace, industrial power-gas turbine, bio/chemical-reactor, marine-seawater, and in vivo-bio fluids require accurate and robust sensing of the physical world to enable intelligence at the edge. The challenge and opportunity for edge sensing is the development of optimal harsh environment sensor technologies using the appropriate materials, transducer/device, and packaging.
For example, a key challenge for image sensors, often vital at the edge, is detecting any significant change among the many pixels. Specialized event cameras are now commercially available in the visible domain, where a focal plane array of silicon sensors filters data through a differentiator that singles out time-varying signals for processing. Such a camera can eventually be equipped with a neuro-morphic backend for saliency detection, so that signals that are not just varying but are time-correlated (e.g., leading and trailing edges of a moving object connected by a conformal map) can be prioritized for immediate data processing for edge intelligence.
These examples of machine vision seek to mimic retinal pre-processing and data culling prior to its transfer to the visual cortex. They need low latency and low SWaP (Size-Weight and Power). One challenge is extending event-based imaging from visible to infrared (night vision), where there are considerably fewer training data sets. One way forward is synthetic IR data generation out of visible data streams, powered by ML/AI.
Low SWaP is particularly important for wireless at the edge. The size of the antenna may be reduced below order lambda or wavelength by innovative approaches such as magneto-electric nanowires [Ref Nanowire]. Lossy interconnect is an increasing challenge at higher GHz frequency bands which is an opportunity for novel materials such as metaconductors [Ref Metaconductor].
Metaconductors consisting of multiple non-ferromagnetic and ferromagnetic nano metal layers are utilized for the reduction of radio frequency (RF) resistance and therefore RF power consumption. The size of the metaconductor based components can be reduced with architectures loaded with metamaterial unit cells such as split ring resonators, complementary split ring resonators, and spiral resonators. With the combination of the metaconductors and metamaterial resonator architecture, highly compact and energy efficient RF/wireless components and systems can be realized.
On-chip wireless communication platforms can avoid cross-talk, signal loss and electro-migration. However, nano-antennas lose efficiency at sizes smaller than their wavelength. Using acoustic or spin waves, one can alter the operating frequency to the resonant rather than EM frequency, allowing much smaller sizes, higher frequency compatible with 5G/6G, as well as low-frequency lossy communication (e.g. underwater, implants). Energy-efficient actuation of lattice or spin waves, along with engineered nonlinearity, can lead to compact on-chip miniature antennas.
In particular, antiferromagnets operate at very high (~THz) resonant frequencies compared to ferromagnets because of their reliance on exchange coupling rather than anisotropy, while miniature magnetic vortices (skyrmions) in antiferromagnets have been current driven at ~1000 m/s speeds. These fast information carrying bits can then be gated by magneto-elastic and electric methods in the THz regime, such as using magnetostrictive strain or voltage-controlled magnetic anisotropy across an oxide interface.
At the edge, power efficiency is crucial, necessitating energy-efficient hardware but also diverse power sources, including the ubiquitous batteries but augmented with energy harvesting sources like solar, thermal or RF. The projected Internet of a Trillion Things (IoTT) will not happen if devices rely on batteries that need replaced or recharged. Harvesting energy will be a first order requirement at the edge.
Energy harvesting can take many forms, from the opportunistic (harvest whatever happens to be available in the environment, similar to a hunter-gatherer strategy) to planned deterministic (have a way to explicitly send power, similar to an industrial agriculture strategy). Opportunistic energy harvesting is potentially less expensive but also unreliable, a planned deterministic harvesting strategy such as wireless power transmission (similar to RFID which initially inspired the idea of IoT) can lead to more reliable IoT solutions.
Edge integration can be divided between IC integration and system integration. Aggressive scaling of edge IC architecture will typically require dense 3D heterogeneous integration, with high alignment accuracy on silicon chiplets. It will need dense vias for compact non-volatile memory, System-on-Chip designs for 2.5D monolithic and heterogeneous integration of complementary modules such as beyond-CMOS devices, RF/Analog devices, and rad-hard spintronics, along with wide bandwidth on-chip spintronic and chip-to-chip optical interconnects. It will also need design automation tools based on ML/AI.
The memory elements need to be compatible Back-end-of-the-Line (BEOL); materials requiring high processing temperatures will need to be fabricated separately and hybrid bonded with precision die-to-wafer bonding set-ups. System integration is application specific, ranging from hybrid bonding on rigid substrates to flexible substrates. However, hybrid bonding comes with its own challenges, as sub-micron particles can create large hot spots that are usually avoided with micro bump bonding.
Overall, the future is bright for edge intelligence with a synergistic approach turning challenges into opportunities in computing, sensing, wireless, power, and integration.
— Swaminathan Rajaraman is a tenured academic and a successful entrepreneur. He is an Associate Professor in the NanoScience Technology Center and the Department of Materials Science and Engineering at the University of Central Florida (Orlando, Florida). Prior to his academic appointment, he has worked in the MEMS industry and co-founded Axion BioSystems Inc. (Atlanta, Georgia), a provider of high-throughput Microelectrode Arrays (MEAs) and MEA systems. He has published more than 80 articles and holds 32 patents and applications.
— Avik Ghosh is a professor of Electrical and Computer Engineering and professor of Physics at the University of Virginia. He has over 200 refereed papers and book chapters and two published books in the areas of quantum transport, computational nanoelectronics and low power devices, specializing in materials to systems modeling (DFT2SPICE), including 2D materials, thin films for photodetectors, molecular electronics, nanomagnetic materials and devices, and nanoscale heat flow. Ghosh received the NSF CAREER award, the IBM faculty award, several best paper awards, and is Fellow of the Institute of Physics UK and IEEE Senior member.
— Mircea Stan teaches and conducts research in the areas of AI hardware, Processing in Memory, Cyber-Physical Systems, Computational RFID, spintronics, and nanoelectronics. Since 1996 he has been with the Electrical and Computer Engineering Department at University of Virginia, where he is the Virginia Microelectronics Consortium Professor, Director of Computer Engineering, and leads the High-Performance Low-Power lab. He is a fellow of the IEEE, received the NSF CAREER award in 1997 and co-authored best papers presented at ASILOMAR19, LASCAS19, SELSE17, ISQED08, GLSVLSI06, ISCA03 and SHAMAN02.
— Jennifer Andrew and Yong-Kyu Yoon of the University of Florida, Gainesville; Nikhil Shukla of the University of Virginia, Charlottesville; and Jiann-Shiun Yuan and Hyoung Jin Cho of the University of Central Florida, Orlando contributed to this blog.
References
FE Ref
R1
Ref Nanowire
Ref Metaconductor: Renuka Bowrothu, Hae-in Kim, Woosol Lee, Timothy Clingenpeel, and Yong Kyu Yoon, “Highly Energy-Efficient Metaconductor-Based Integrated RF Passives,” IEEE Microwave Magazine, vol. 23, no. 8, 10 pages, May 2022, DOI:10.1109/MMM.2022.3173468
 |
 |
 |
 |
 |
 |
 |  |
Leave a Reply