Analysis and Optimization of Output Operations in the INM RAS Earth System Model
DOI:
https://doi.org/10.14529/jsfi230405Keywords:
Earth system model, INMCM6, MPI, data gathering, derived types, manual packing, CrayAbstract
The modern development of complex Earth system models forces developers to take into account not only the computational efficiency, but also the performance of the data input and output. This work evaluates the data output performance of the INM RAS Earth system model and optimizes its weak points. The output operations were found to be surprisingly slow on the Cray XC40-LC supercomputer compared to the results obtained on the INM RAS cluster. To identify the bottleneck, the computational time, the distributed data gathering time, and the file system output time were measured separately. The distributed data gathering time was the cause of the slowdown on the Cray XC40-LC, so optimizations were made to the gathering routines without any additional rework of the existing output code. The optimizations resulted in a significant reduction in the overall model running time on the Cray XC40-LC, while the gathering time itself was reduced by a factor of 102–103. The results highlight the importance of optimizing the output performance in Earth system models.
References
Acosta, M.C., Palomas, S., Paronuzzi Ticco, S.V., et al.: The computational and energy cost of simulation and storage for climate science: lessons from CMIP6. Geoscientific Model Development Discussions 2023, 1–21 (2023). https://doi.org/10.5194/gmd-2023-188
Balaji, V., Maisonnave, E., Zadeh, N., et al.: CPMIP: measurements of real computational performance of Earth system models in CMIP6. Geoscientific Model Development 10(1), 19–34 (2017). https://doi.org/10.5194/gmd-10-19-2017
Balay, S., Gropp, W.D., McInnes, L.C., et al.: Efficient Management of Parallelism in Object-Oriented Numerical Software Libraries. In: Arge, E., Bruaset, A.M., Langtangen, H.P. (eds.) Modern Software Tools for Scientific Computing, pp. 163–202. Birkhäuser Boston, Boston, MA (1997). https://doi.org/10.1007/978-1-4612-1986-6_8
Blagodatskikh, D.V.: Comparison of computational efficiency of two versions of a terrainfollowing ocean climate model. Numerical Methods and Programming 24, 440–449 (2023). https://doi.org/10.26089/NumMet.v24r430
Blagodatskikh, D.V., Iakovlev, N.G., Volodin, E.M., et al.: Non-local discretization of the isoneutral diffusion operator in a terrain-following climate ocean model. Russian Journal of Numerical Analysis and Mathematical Modelling 38(6), 1–8 (2023). https://doi.org/10.1515/rnam-2023-0026
Bock, L., Lauer, A., Schlund, M., et al.: Quantifying Progress Across Different CMIP Phases With the ESMValTool. Journal of Geophysical Research: Atmospheres 125(21), e2019JD032321 (2020). https://doi.org/10.1029/2019JD032321
Chen, D., Rojas, M., Samset, B.H., et al.: Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. In: Masson-Delmotte, V., Zhai, P., Pirani, A., et al. (eds.) Climate Change 2021: The Physical Science Basis, pp. 147–286. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA (2021). https://doi.org/10.1017/9781009157896.003
Emelina, S.V., Khan, V.M., Semenov, V.A., et al.: Seasonal Hydrodynamic Forecasts Using the INM-CM5 Model for Estimating the Beginning of Birch Pollen Dispersion. Izv. Atmos. Ocean. Phys. 59, 351359 (2023). https://doi.org/10.1134/S0001433823040059
Eyring, V., Bony, S., Meehl, G.A., et al.: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geoscientific Model Development 9(5), 1937–1958 (2016). https://doi.org/10.5194/gmd-9-1937-2016
Gloukhov, V.: Parallel implementation of the INM atmospheric general circulation model on distributed memory multiprocessors. In: Sloot, P.M.A., Hoekstra, A.G., Tan, C.J.K., Dongarra, J.J. (eds.) Computational Science – ICCS 2002. Lecture Notes in Computer Science, vol. 2329, pp. 753–762. Springer Berlin, Heidelberg (2002). https://doi.org/10.1007/3-540-46043-8
Haarsma, R.J., Roberts, M.J., Vidale, P.L., et al.: High Resolution Model Intercomparison Project (HighResMIP v1.0) for CMIP6. Geoscientific Model Development 9(11), 4185–4208 (2016). https://doi.org/10.5194/gmd-9-4185-2016
Hoskins, B.: The potential for skill across the range of the seamless weather-climate prediction problem: a stimulus for our science. Quarterly Journal of the Royal Meteorological Society 139(672), 573–584 (2013). https://doi.org/10.1002/qj.1991
Khan, V.M., Vilfand, R., Tishchenko, V., et al.: Assessment of Changes in the Temperature Regime of Northern Eurasia for the Next Five Years According to the INM RAS Earth System Model Forecasts and Their Possible Consequences for Agriculture. Russ. Meteorol. Hydrol. 48, 745754 (2023). https://doi.org/10.3103/S1068373923090029
Kim, Y.H., Min, S.K., Zhang, X., et al.: Evaluation of the CMIP6 multi-model ensemble for climate extreme indices. Weather and Climate Extremes 29, 100269 (2020). https://doi.org/10.1016/j.wace.2020.100269
Kulikova, I.A., Nabokova, E.V., Khan, V.M., et al.: MaddenJulian Oscillation in the Context of Subseasonal Variability, Teleconnections, and Predictability. Russ. Meteorol. Hydrol. 48, 645657 (2023). https://doi.org/10.3103/S1068373923080010
Stocker, T.F., Qin, D., Plattner, G.K., et al. (eds.): IPCC, 2013: Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, p. 1535. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA (2013), https://www.ipcc.ch/site/assets/uploads/2018/02/WG1AR5_all_final.pdf
Tarasevich, M., Sakhno, A., Blagodatskikh, D., et al.: Scalability oftheINM RAS Earth System Model. In: Voevodin, V., Sobolev, S., Yakobovskiy, M., et al. (eds.) Supercomputing. pp. 202–216. Lecture Notes in Computer Science, Springer Nature Switzerland (2023). https://doi.org/10.1007/978-3-031-49432-1_16
Tarasevich, M.A., Tsybulin, I.V., Onoprienko, V.A., et al.: Ensemble-based statistical verification of INM RAS Earth system model. Russian Journal of Numerical Analysis and Mathematical Modelling 38(3), 173–186 (2023). https://doi.org/10.1515/rnam-2023-0014
Tarasevich, M.A., Volodin, E.M.: Influence of various parameters of INM RAS climate model on the results of extreme precipitation simulation. In: International Young Scientists School and Conference on Computational Information Technologies for Environmental Sciences, May 27 – June 6, 2019. IOP Conference Series: Earth and Environmental Science, vol. 386, p. 012012. IOP Publishing (2019). https://doi.org/10.1088/1755-1315/386/1/012012
Tarasevich, M.A., Volodin, E.M.: The Influence of Autumn Eurasian Snow Cover on the Atmospheric Dynamics Anomalies during the Next Winter in INMCM5 Model Data. Supercomputing Frontiers and Innovations 8(4), 24–39 (2021). https://doi.org/10.14529/jsfi210403
Terekhov, K.M., Volodin, E.M., Gusev, A.V.: Methods and efficiency estimation of parallel implementation of the σ-model of general ocean circulation. Russ. J. Numer. Anal. Math. Modelling 26(2), 189–208 (2011). https://doi.org/10.1515/rjnamm.2011.011
Volodin, E.M.: Simulation of Present-Day Climate with the INMCM60 Model. Izvestiya, Atmospheric and Oceanic Physics 59(1), 16–22 (2023). https://doi.org/10.1134/S0001433823010139
Volodin, E.M., Gritsun, A.S.: Simulation of observed climate changes in 1850–2014 with climate model INM-CM5. Earth System Dynamics 9(4), 1235–1242 (2018). https://doi.org/10.5194/esd-9-1235-2018
Volodin, E.M., Kostrykin, S.V.: The aerosol module in the INM RAS climate model. Russian Meteorology and Hydrology 41(8), 519–528 (2016). https://doi.org/10.3103/S106837391608001X
Volodin, E.M., Lykosov, V.N.: Parametrization of Heat and Moisture Transfer in the Soil-Vegetation System for Use in Atmospheric General Circulation Models: 1. Formulation and Simulations Based on Local Observational Data. Izvestiya, Atmospheric and Oceanic Physics 34(4), 405–416 (1998), https://www.researchgate.net/publication/270586916_Parameterization_of_Heat_and_Moisture_Transfer_in_the_Soil-Vegetation_System_for_Use_in_Atmospheric_General_Circulation_Models_1_Formulation_and_Simulations_Based_on_Local_Observational_Data
Volodin, E.M., Lykosov, V.N.: Parametrization of Heat and Moisture Transfer in the Soil-Vegetation System for Use in Atmospheric General Circulation Models: 2. Numerical Experiments in Climate Modeling. Izvestiya, Atmospheric and Oceanic Physics 34(5), 559–569 (1998), https://www.researchgate.net/publication/270586932_Parameterization_of_Heat_and_Moisture_Transfer_in_the_Soil-Vegetation_System_for_Use_in_Atmospheric_General_Circulation_Models_2_Numerical_Experiments_in_Climate_Modeling
Volodin, E.M., Mortikov, E.V., Kostrykin, S.V., et al.: Simulation of modern climate with the new version of the INM RAS climate model. Izvestiya, Atmospheric and Oceanic Physics 53(2), 142–155 (2017). https://doi.org/10.1134/S0001433817020128
Volodin, E.M., Mortikov, E.V., Kostrykin, S.V., et al.: Simulation of the present-day climate with the climate model INMCM5. Climate Dynamics 49(11), 3715–3734 (2017). https://doi.org/10.1007/s00382-017-3539-7
Volodin, E.M., Mortikov, E.V., Kostrykin, S.V., et al.: Simulation of the modern climate using the INM-CM48 climate model. Russian Journal of Numerical Analysis and Mathematical Modelling 33(6), 367–374 (2018). https://doi.org/10.1515/rnam-2018-0032
Volodin, E.M., Tarasevich, M.A.: Simulation of Climate and Weather Extreme Indices with the INM-CM5 Climate Model. Russian Meteorology and Hydrology 43(11), 756–762 (2018). https://doi.org/10.3103/S1068373918110067
Volodin, E.M., Vorobyeva, V.V.: On the multi-annual potential predictability of the Arctic Ocean climate state in the INM RAS climate model. Russian Journal of Numerical Analysis and Mathematical Modelling 37(2), 119–129 (2022). https://doi.org/10.1515/rnam-2022-0010
Vorobeva, V.V., Volodin, E.M., Gritsun, A.S., et al.: Analysis of the Atmosphere and the Ocean Upper Layer State Predictability for up to 5 Years Ahead Using the INMCM5 Climate Model Hindcasts. Russian Meteorology and Hydrology 48(7), 581–589 (2023). https://doi.org/10.3103/S106837392307004X
Vorobyeva, V., Volodin, E.: Analysis of the predictability of stratospheric variability and climate indices based on seasonal retrospective forecasts of the INM RAS climate model. Russian Journal of Numerical Analysis and Mathematical Modelling 36(2), 117–126 (2021). https://doi.org/10.1515/rnam-2021-0010
Vorobyeva, V., Volodin, E.: Evaluation of the INM RAS climate model skill in climate indices and stratospheric anomalies on seasonal timescale. Tellus A: Dynamic Meteorology and Oceanography 73(1), 1–12 (2021). https://doi.org/10.1080/16000870.2021.1892435
Vorobyeva, V.V., Volodin, E.M.: Experimental Studies of Seasonal Weather Predictability Based on the INM RAS Climate Model. Mathematical Models and Computer Simulations 13(4), 571–578 (2021). https://doi.org/10.1134/S2070048221040232
Yakovlev, N.G.: Reproduction of the large-scale state of water and sea ice in the Arctic Ocean from 1948 to 2002: Part II. The state of ice and snow cover. Izvestiya, Atmospheric and Oceanic Physics 45(4), 478–494 (2009). https://doi.org/10.1134/S0001433809040082
Yakovlev, N.G.: Reproduction of the large-scale state of water and sea ice in the Arctic Ocean in 1948–2002: Part I. Numerical model. Izvestiya, Atmospheric and Oceanic Physics 45(3), 357–371 (2009). https://doi.org/10.1134/S0001433809030098
Downloads
Published
How to Cite
License
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-Non Commercial 3.0 License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.