Geodesic
Neural network approach for mapping of parallel applications
Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ Vyčislitelʹnaâ matematika i informatika, Tome 9 (2020) no. 1, pp. 36-49 Cet article a éte moissonné depuis la source Math-Net.Ru

Voir la notice de l'article

The article is about the problem of improving parallel applications efficiency. It proposes an approach to solve this problem. The approach aims to reduce communication overheads of data exchange between parallel program processes during its execution on high-performance computing system. Growing number of computer nodes leads increasing impact of the communication overhead on application performance. As a sequence the problem of parallel processes allocation to hardware nodes (mapping problem) becomes very actual. The new approach for mapping problem is proposed in this work. The key characteristic of the approach is to extract a communication pattern by phase analysis of application using a convolutional neural network to fast choosing the most relevant mapping algorithm for extracted pattern. Investigation of results of point-to-point communications between parallel program processes is used to build a communication pattern. The application timeline is broken into equal intervals. Communication patterns are created for each interval. Then Haar 2D wavelet transform is applied to these patterns for generating features. Features are clustered, after that, timeline is splitted into phases. Each phase has its own communication pattern. The selection of the most relevant mapping algorithm is performed by using convolutional neural network. It supposes some knowledge about different types of parallel applications and most suitable mapping algorithms. This knowledge must be represented by a set of pattern classes (classes of matrices), each class has the mapping algorithm, which fits best for application with this type of pattern. This set can be a training sample for the neural network. Thus the neural network classifies an input application communication pattern and finds a mapping algorithm for the application. The stages of proposed approach are implemented in this work. This implementation is demonstrated for some tests.
Keywords: high performance computing systems, topology-aware mapping, communication pattern, convolution neural network. 
@article{VYURV_2020_9_1_a2,
     author = {N. N. Popova and M. V. Kozlov and M. V. Shubin},
     title = {Neural network approach for mapping of parallel applications},
     journal = {Vestnik \^U\v{z}no-Uralʹskogo gosudarstvennogo universiteta. Seri\^a Vy\v{c}islitelʹna\^a matematika i informatika},
     pages = {36--49},
     year = {2020},
     volume = {9},
     number = {1},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/VYURV_2020_9_1_a2/}
}
TY  - JOUR
AU  - N. N. Popova
AU  - M. V. Kozlov
AU  - M. V. Shubin
TI  - Neural network approach for mapping of parallel applications
JO  - Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ Vyčislitelʹnaâ matematika i informatika
PY  - 2020
SP  - 36
EP  - 49
VL  - 9
IS  - 1
UR  - http://geodesic.mathdoc.fr/item/VYURV_2020_9_1_a2/
LA  - ru
ID  - VYURV_2020_9_1_a2
ER  - 
%0 Journal Article
%A N. N. Popova
%A M. V. Kozlov
%A M. V. Shubin
%T Neural network approach for mapping of parallel applications
%J Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ Vyčislitelʹnaâ matematika i informatika
%D 2020
%P 36-49
%V 9
%N 1
%U http://geodesic.mathdoc.fr/item/VYURV_2020_9_1_a2/
%G ru
%F VYURV_2020_9_1_a2
N. N. Popova; M. V. Kozlov; M. V. Shubin. Neural network approach for mapping of parallel applications. Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ Vyčislitelʹnaâ matematika i informatika, Tome 9 (2020) no. 1, pp. 36-49. http://geodesic.mathdoc.fr/item/VYURV_2020_9_1_a2/

[1] TOP500 List } {\tt https://www.top500.org/

[2] T. Hoefler, M. Snir, “Generic topology mapping strategies for large-scale parallel architectures”, Proceedings of international conference on Supercomputing, International conference on Supercomputing (ICS ’11) (Tucson, Arizona, USA, May, 31–June, 04, 2011), ACM, 2011, 75–84 | DOI

[3] S. Sreepathi, E. D’Azevedo, B. Philip, P. Worley, “Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers”, Proceedings of the 7th ACM/SPEC on International Conference on Performance Engineering, 7th ACM/SPEC on International Conference on Performance Engineering (ICPE ’16) (Delft, The Netherlands, March, 12–16, 2016), ACM, 2016, 225–236 | DOI

[4] J. Wu, X. Xiong, Z. Lan, “Hierarchical task mapping for parallel applications on supercomputers”, The Journal of Supercomputing, 71:5 (2015), 1776–1802 | DOI

[5] T. Hoefler, E. Jeannot, G. Mercier, “An overview of topology mapping algorithms and techniques in high-performance computing”, High-Performance Computing on Complex Environment, 2014, 75–94 | DOI

[6] M. V. Shubin, N. N. Popova, “Study of the behavior of parallel MPI-programs based on phases of interprocess communication”, Russian Supercomputing Days, Proceedings of the international conference (Moscow, Russia, September, 24–25, 2018), Moscow State University, Moscow, 2018, 662–672

[7] C. Jacobs, A. Finkelstein, D. Salesin, “Fast multiresolution image querying”, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, 22nd annual conference on Computer graphics and interactive techniques (SIGGRAPH ’95) (Los Angeles, CA, August, 1995), ACM, 1995, 277–286 | DOI

[8] K. Simonyan, A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv: 1409.1556

[9] Keras Documentation } {\tt https://keras.io/

[10] NAS Parallel Benchmarks. } {\tt https://www.nas.nasa.gov/publications/npb. html

[11] K. Asanovic, R. Bodik, B. Catanzaro, J. Gebis, P. Husbands, K. Keutzer, D. Patterson, W. Plishker, J. Shalf, S. Williams, K. Yelick, “A View of the Parallel Computing Landscape”, Communications of the ACM, 2009, no. 10, 56–67 | DOI