Researchers from Google and University of California, Berkeley published a technical paper titled “Google’s Training Supercomputers from TPU v2 to Ironwood: Architectural Stability, Scale, Resilience, Power Efficiency, and Sustainability Across Five Generations.”
The paper summarizes five generations of Google TPUs, from TPU v2 through Ironwood, and examines how the systems evolved into scalable, resilient, power-efficient, and more sustainable supercomputers for AI training. It describes the architectural stability of the TPU platform across rapidly changing neural-network workloads, including Transformers. Across eight years, the paper reports major gains in HBM capacity and bandwidth per node, peak node performance, and total supercomputer performance. It also discusses optical circuit switches, built-in self test, and hardware replay for resilience, as well as improvements in performance per watt and carbon emissions per floating-point operation. The paper concludes with six features the authors identify as likely characteristics of successful training accelerators in this decade.
Find the technical paper here. June 2026.
arXiv:2606.15870v1
Jouppi, Norman P., Sridhar Lakshmanamurthy, Cliff Young, and David Patterson. “Google’s Training Supercomputers from TPU v2 to Ironwood: Architectural Stability, Scale, Resilience, Power Efficiency, and Sustainability Across Five Generations.” IEEE Micro, July/August 2026. https://doi.org/10.1109/MM.2026.3699647.
Leave a Reply