Shaking Up The Green500

Barry Pangrle
Last September, I wrote about the efficiency of IBM’s Power7+ architecture in my blog. IBM’s Sequoia supercomputer (a BlueGene/Q system) this past June had just shot to the top of the Supercomputing Top500 chart, clocking in at 16.32 petaflop/s on the Linpack benchmark. Other systems built around the IBM BlueGene/Q, Power BQC 16C 1.60GHz, Custom were also dominating the top of the Green500 list with the Sequoia placing 20th in the megaflop/s/watt category.

Well, that was September and the November 2012 Top500 chart now shows a new leader at the top with a score of 17.59 petaflop/s on Linpack. It is the Oak Ridge National Laboratory’s Titan, a Cray XK7 with 18,688 nodes with each node containing an AMD 16-Core Opteron 6274 and an NVIDIA Tesla K20X.

Oak Ridge National Labs’ Titan Supercomputer (Photo courtesy of ORNL)

I alluded to the potential energy efficiency of GPU-based computing in the September blog and heterogeneous systems have suddenly jumped to the top of the November 2012 Green500 list. The top 4 on the Green500 list are now all heterogeneous systems. Titan currently sits in 3rd place, bettering Sequoia’s opening position at 20th (which has now fallen to 29th). The next fastest supercomputer, RIKEN’s K Computer is now in 85th place on the Green500 list, so out of the top of the supercomputer class, the heterogeneous Cray XK7 Titan is a good number of spots higher on the Green500 list.

How about the other two systems above Titan on the green list? In first place is a mixed Intel Xeon E5-2670- and Intel Xeon Phi 5110P-based system that is good for 0.112 petaflop/s and in second is a mixed Intel Xeon E5-2650 and AMD FirePro S10000 system that checks in at 0.421 petaflop/s. It will be interesting to see if and how well these systems scale up to performance an order of magnitude or two higher to possibly compete for the top of the Top500 list. We might just get some insight into that soon with the Texas Advanced Computing Center’s Stampede supercomputer that just went online after two years of development. It is supposed to hit 10 petaflop/s (peak) and is a Dell PowerEdge C8220 cluster filled with Intel Xeon Phi co-processors. It is currently estimated to need more than 6 MegaWatts, including cooling. Based on the preliminary numbers, it looks unlikely to challenge for the top of the Green500 list, but we should wait for official numbers before drawing any firm conclusions.

Texas Advanced Computing Center’s Stampede Supercomputer (Photo Courtesy TACC)

The fourth and fifth place positions on the Green500 list belong to a Cray XK7 and an IBM BlueGene system, respectively. It will be interesting to see how the list changes again this year. The push to hit the exascale mark is on and the competition promises to be fierce.

—Barry Pangrle is a senior power methodology engineer at NVIDIA. The views expressed in this article are his own and not necessarily those of NVIDIA.