The L-CSC cluster: Optimizing power efficiency to become the greenest supercomputer in the world in the Green500 list of November 2014

David Rohr; Gvozden Neskovic; Volker Lindenstruth

doi:10.14529/jsfi150304

Authors

David Rohr Frankfurt Institute for Advanced Studies, Frankfurt
Gvozden Neskovic Frankfurt Institute for Advanced Studies, Frankfurt
Volker Lindenstruth Frankfurt Institute for Advanced Studies, Frankfurt Goethe University Frankfurt, Frankfurt

DOI:

https://doi.org/10.14529/jsfi150304

Abstract

The L-CSC (Lattice Computer for Scientific Computing) is a general purpose compute cluster built with commodity hardware installed at GSI. Its main operational purpose is Lattice QCD (LQCD) calculations for physics simulations. Quantum Chromo Dynamics (QCD) is the physical theory describing the strong force, one of the four known fundamental interactions in the universe. L-CSC leverages a multi-GPU design accommodating the huge demand of LQCD for memory bandwidth. In recent years, heterogeneous clusters with accelerators such as GPUs have become more and more powerful while supercomputers in general have shown enormous increases in power consumption making electricity costs and cooling a significant factor in the total cost of ownership. Using mainly GPUs for processing, L-CSC is very power-efficient, and its architecture was optimized to provide the greatest possible power efficiency. This paper presents the cluster design as well as optimizations to improve the power efficiency. It examines the power measurements performed for the Green500 list of the most power-efficient supercomputers in the world which led to the number 1 position as the greenest supercomputer in November 2014.

References

Matthias Bach, Matthias Kretz, Volker Lindenstruth, and David Rohr. Optimized HPL for AMD GPU and Multi-Core CPU Usage. Computer Science - Research and Development, 26(3-4):153–164, 2011. DOI: 10.1007/s00450-011-0161-5.

Matthias Bach, Volker Lindenstruth, Christopher Pinke, and Owe Philipsen. Twisted-Mass Lattice QCD using OpenCL. In Proceedings of Science - 31st International Symposium on Lattice Field Theory - LATTICE 2013.

EEHPC Working Group. Energy Efficient High Performance Computing Power Measurement Methodology v1.2 RC2.

EEHPC Working Group. Energy Efficient High Performance Computing Power Measurement Methodology v2.0 RC1, 2015.

Toshio Endo, Akira Nukada, and Satoshi Matsuoka. TSUBAME-KFC: Ultra green supercomputing testbed. Presented at International Conference for High Performance Computing, Networking, Storage and Analysis (SuperComputing, SC13), 2013.

David Rohr, Matthias Bach, Gvozden Neskovic, Volker Lindenstruth, Christopher Pinke, and Owe Philipsen. Lattice-CSC: Optimizing and building an efficient supercomputer for Lattice-QCD and to achieve first place in Green500. In Proceedings of the International Supercomputing Conference, 2015. DOI: 10.1007/978-3-319-20119-1_14.

David Rohr, Sebastian Kalcher, Matthias Bach, A. Alaqeeli, H. Alzaid, Dominic Eschweiler, Volker Lindenstruth, A. Sakhar, A. Alharthi, A. Almubarak, I. Alqwaiz, and R. Bin Suliman. An Energy-Efficient Multi-GPU Supercomputer. In Proceedings of the 16th IEEE International Conference on High Performance Computing and Communications, HPCC 2014, Paris, France. IEEE, 2014. DOI: 10.1109/hpcc.2014.14.

Sushant Sharma, Chung-Hsing Hsu, and Wu-Chun Feng. Making a case for a Green500 list. In Proceedings 20th IEEE International Parallel Distributed Processing Symposium, page 343. IEEE, 2006. DOI: 10.1109/ipdps.2006.1639600.