Acceleration of MPI mechanisms for sustainable HPC applications

Jesus Carretero; Javier Garcia-Blas; David E. Singh; Florin Isaila; Alexey Lastovetsky; Thomas Fahringer; Radu Prodan; Peter Zangerl; Christi Symeonidou; Afshin Fassihi; Horacio Pérez-Sánchez

doi:10.14529/jsfi150202

Authors

Jesus Carretero Computer Science and Engineering Dep. Engineering School University Carlos III of Madrid, Madrid
Javier Garcia-Blas Computer Science and Engineering Dep. Engineering School University Carlos III of Madrid, Madrid
David E. Singh Computer Science and Engineering Dep. Engineering School University Carlos III of Madrid, Madrid
Florin Isaila Computer Science and Engineering Dep. Engineering School University Carlos III of Madrid, Madrid
Alexey Lastovetsky University College Dublin, Dublin
Thomas Fahringer University of Innsbruck, Innsbruck
Radu Prodan University of Innsbruck, Innsbruck
Peter Zangerl University of Innsbruck, Innsbruck
Christi Symeonidou ICS, FORTH, Heraclion
Afshin Fassihi Universidad Católica San Antonio de Murcia (UCAM), Murcia
Horacio Pérez-Sánchez Universidad Católica San Antonio de Murcia (UCAM), Murcia

DOI:

https://doi.org/10.14529/jsfi150202

Abstract

Ultrascale computing systems are meant to reach a growth of two or three orders of magnitude of today computing systems. However, to achieve the performances required, we will need to design and implement more sustainable solutionsfor ultra-scale computing systems, understanding sustainability in a holistic manner to address challenges as economy-of-scale, agile elastic scalability, heterogeneity, programmability, fault resilience, energy efficiency, and scalable storage. Some of those solutions could be provided into MPI, but other should be devised as higher level concepts, less generalists, but adapted to applicative domains, possibly as programming patterns or or libraries. In this paper, we show some proposals to extend MPI trying to cover major domains that are relevant towards sustainability: MPI programming optimizations and programming models, resilience, data management, and their usage from applications.

References

Leonardo Bautista-Gomez, Seiji Tsuboi, Dimitri Komatitsch, Franck Cappello, Naoya Maruyama, and Satoshi Matsuoka. Fti: High performance fault tolerance interface for hybrid systems. In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’11, pages 32:1–32:32, New York, NY, USA, 2011. ACM.

W. Bland, A. Bouteiller, T. Herault, J. Hursey, G. Bosilca, and J.J. Dongarra. An evaluation of user-level failure mitigation support in mpi. DOI 10.1007/s00607-013-0331-3:1–14, May 2013.

Javier Garcia Blas, Florin Isaila, David E. Singh, , and Jesus Carretero. View-based collective i/o for mpi-io. In 8th IEEE International Symposium on Cluster Computing and the Grid, 2008. CCGRID ’08., pages 409–416, May 2008.

G. Bosilca, A. Bouteiller, T. Herault, Y. Robert, and J. Dongarra. Assessing the impact of abft and checkpoint composite strategies. In 16th Workshop on Advances in Parallel and Distributed Computational Models, IPDPS 2014, Phoenix, AZ, May 2014.

M. Burtscher and Paruj R. fpc: A high-speed compressor for double-precision floating-point data. IEEE Transactions on Computers, 58(1):1831, 2009.

A. Bilas C. Symeonidou, P. Pratikakis and DS. Nikolopoulos. Drasync: Distributed regionbased memory allocation and synchronization. In Proceedings of the 20th European MPI Users Group Meeting. EuroMPI 13, page 4954. ACM, 2013.

Martin J Chorley and David W Walker. Performance analysis of a hybrid MPI/OpenMP application on multi-core clusters. Journal of Computational Science, 1(3):168–174, August 2010.

Jinsuk Chung, Ikhwan Lee, Michael Sullivan, Jee Ho Ryoo, Dong Wan Kim, Doe Hyun Yoon, Larry Kaplan, and Mattan Erez. Containment domains: A scalable, efficient, and flexible resilience scheme for exascale systems. In the Proceedings of SC12, November 2012.

S. Coco, DArrigo V., and Giunta D. A rice-based lossless data compression system for space. In 2000 IEEE Nordic Signal Processing Symposium, pages 133–142, May 2000.

Shulin Cui and Shuqing Zhang. Parallel processing of topological operations by using a hybrid MPI/OpenMP approach. In Natural Computation (ICNC), 2013 Ninth International Conference on, pages 1738–1742, 2013.

Kiril Dichev, Vladimir Rychkov, and Alexey Lastovetsky. Two algorithms of irregular scatter/gather operations for heterogeneous platforms. In Recent Advances in the Message Passing Interface, pages 289–293. Springer Berlin Heidelberg, 2010.

Rosa Filgueira, Jesus Carretero, David E. Singh, Alejandro Calderon, and Alberto Nuez. Dynamic-compi: Dynamic optimization techniques for mpi parallel applications. The Journal of Supercomputing, 59(1):361–391, April 2012.

Gin´es D Guerrero, Horacio P´erez-S´anchez, Jos´e M Cecilia, and Jos´e M Garc´ıa. Parallelization of virtual screening in drug discovery on massively parallel architectures. In Parallel, Distributed and Network-Based Processing (PDP), 2012 20th Euromicro International Conference on, pages 588–595. IEEE, 2012.

Khalid Hasanov, Jean-Noel Quintin, and Alexey Lastovetsky. Hierarchical approach to optimization of parallel matrix multiplication on large-scale platforms. The Journal of Supercomputing, pages 1–24, 2014.

Florin Isaila, Francisco Javier Garcia Blas, Jesus Carretero, Wei-Keng Liao, and Alok Choudhary. A Scalable Message Passing Interface Implementation of an Ad-Hoc Parallel I/O System. Int. J. High Perform. Comput. Appl., 24(2):164–184, May 2010.

H. Jordan, P. Thoman, J. Durillo, S. Pellegrini, P. Gschwandtner, T. Fahringer, and H. Moritsch. A multi-objective auto-tuning framework for parallel codes. In Proc. of the Intl. Conference for High Performance Computing, Networking, Storage and Analysis (SC 2012). IEEE Computer Society Press, 2012.

Donald E. Knuth. Dynamic huffman coding. Journal of Algorithms, 6(2):163180, 1985.

S. Lyberis, P. Pratikakis, DS. Nikolopoulos, M. Schulz, T. Gamblin, and BR. de Supinski. The myrmics memory allocator: hierarchical,message-passing allocation for global address spaces. In Proceedings of the International Symposium on Memory Management. 2012.

Gonzalo Martin, Maria-Cristina Marinescu, David E. Singh, and Jesus Carretero. FLEXMPI: an MPI extension for supporting dynamic load balancing on heterogeneous nondedicated systems. In International European Conference on Parallel and Distributed Computing, EuroPar, 2013.

M.F.X.J. Oberhumer. LZO real-time data compression library. 2012.

Anirban Pal, Abhishek Agarwala, Soumyendu Raha, and Baidurya Bhattacharya. Performance metrics in a hybrid MPI–OpenMP based molecular dynamics simulation with short-range interactions. Journal of Parallel and Distributed Computing, 74(3):2203–2214, March 2014.

M. Pedram and Inkwon Hwang. Power and performance modeling in a virtualized server system. In 2010 39th International Conference on Parallel Processing Workshops (ICPPW),, pages 520–526, Sept 2010.

K. Sato, A. Moody, K. Mohror, T. Gamblin, B.R. de Supinski, N. Maruyama, and S. Matsuoka. Fmi: Fault tolerant messaging interface for fast and transparent recovery. In Parallel and Distributed Processing Symposium, 2014 IEEE 28th International, pages 1225–1234, May 2014.

V.M. Weaver, M. Johnson, K. Kasichayanula, J. Ralph, P. Luszczek, D. Terpstra, and S. Moore. Measuring energy and power with papi. In 2012 41st International Conference on Parallel Processing Workshops (ICPPW), pages 262–268, Sept 2012.

Fangzhou Wei and Ali E Yilmaz. Parallel Computing. Parallel Computing, 37(6-7):279–301, July 2011.

Robert Zigon. run length encoding. Dr. Dobb’s Journal, 14(2):126128, 1989.