PCM-based analog accelerators are a sensible choice for deep learning workloads, even for large natural language processing models like BERT.
Abstract:
“Recent advances in deep learning have been driven by ever-increasing model sizes, with networks growing to millions or even billions of parameters. Such enormous models call for fast and energy-efficient hardware accelerators. We study the potential of Analog AI accelerators based on Non-Volatile Memory, in particular Phase Change Memory (PCM), for software-equivalent accurate inference of natural language processing applications. We demonstrate a path to software-equivalent accuracy for the GLUE benchmark on BERT (Bidirectional Encoder Representations from Transformers), by combining noise-aware training to combat inherent PCM drift and noise sources, together with reduced-precision digital attention-block computation down to INT6.”
View this technical paper here. Published 07/2021.
Spoon K, Tsai H, Chen A, Rasch MJ, Ambrogio S, Mackin C, Fasoli A, Friz AM, Narayanan P, Stanisavljevic M and Burr GW (2021) Toward Software-Equivalent Accuracy on Transformer-Based Deep Neural Networks With Analog Memory Devices. Front. Comput. Neurosci. 15:675741.
Is there room for an even smaller version of a RISC-V processor that could replace 8-bit microcontrollers?
How much do we pay for a system to be programmable? It depends upon who you ask.
Concerns about security and supply chain availability prompt new push.
Some 300mm tools are converted to 200mm; equipment prices and chip manufacturing costs are rising.
Short-term IC supply-chain problems and long-term architectural and business changes top the list of what’s ahead.
Some segments are normalizing, others may be impacted through 2022.
Imec’s SVP drills down into GAA FETs, interconnects, chiplets, and 3D packaging.
Threats are growing and widening, but what is considered sufficient can vary greatly by application or by user. Even then, it may not be enough.
Massive expansion campaign targets wide variety of chips, but export controls limit growth at leading edge.
Sharing resources can significantly improve utilization and lower costs, but it’s not a simple shift.
Abstraction is the key to custom processor design and verification, but defining the right language and tool flow is a work in progress.
Is there room for an even smaller version of a RISC-V processor that could replace 8-bit microcontrollers?
Higher density of interconnects will enable faster movement of data, but there’s more than one way to achieve that.
Leave a Reply