Geodesic
Virtualization of heterogeneous HPC-clusters based on OpenStack platform
Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ Vyčislitelʹnaâ matematika i informatika, Tome 6 (2017) no. 2, pp. 37-48 Cet article a éte moissonné depuis la source Math-Net.Ru

Voir la notice de l'article

The paper addresses to the problem of integration of heterogeneous computing clusters to the united environment based on a virtualization technology. OpenStack software is selected as a platform for managing the virtual environment. The OpenStack platform provides a wide range of components and solutions to a functional interaction with different hypervisors. These include KVM, XEN, ESXi, QEMU and other systems. In addition to the OpenStack platform, we developed a specialized hypervisor shell. It helps to start virtual machines using queues of the traditional resource management systems, such as PBS, SLURM, LSF, or SGE, that are used on clusters of a center of collective usage. The developed model of the resource allocation for virtual machines allowed us to use the knowledge about job requests, resource characteristics and current state of the environment, and the expertise of it administrators. The realized tools provide the capability for the “painless” integration of heterogeneous clusters with the preinstalled local resource managers for creating the virtual cluster with the required configuration. Extensive modeling shows that the hypervisor shell can improve efficiency of integrated environment nodes through reallocating virtual machines to queues of the traditional resource management systems.
Keywords: computer clusters, HPC, virtualization technologies, OpenStack, simulation modeling, OpenStack. 
@article{VYURV_2017_6_2_a2,
     author = {A. G. Feoktistov and I. A. Sidorov and V. V. Sergeev and R. O. Kostromin and V. G. Bogdanova},
     title = {Virtualization of heterogeneous {HPC-clusters} based on {OpenStack} platform},
     journal = {Vestnik \^U\v{z}no-Uralʹskogo gosudarstvennogo universiteta. Seri\^a Vy\v{c}islitelʹna\^a matematika i informatika},
     pages = {37--48},
     year = {2017},
     volume = {6},
     number = {2},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/VYURV_2017_6_2_a2/}
}
TY  - JOUR
AU  - A. G. Feoktistov
AU  - I. A. Sidorov
AU  - V. V. Sergeev
AU  - R. O. Kostromin
AU  - V. G. Bogdanova
TI  - Virtualization of heterogeneous HPC-clusters based on OpenStack platform
JO  - Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ Vyčislitelʹnaâ matematika i informatika
PY  - 2017
SP  - 37
EP  - 48
VL  - 6
IS  - 2
UR  - http://geodesic.mathdoc.fr/item/VYURV_2017_6_2_a2/
LA  - en
ID  - VYURV_2017_6_2_a2
ER  - 
%0 Journal Article
%A A. G. Feoktistov
%A I. A. Sidorov
%A V. V. Sergeev
%A R. O. Kostromin
%A V. G. Bogdanova
%T Virtualization of heterogeneous HPC-clusters based on OpenStack platform
%J Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ Vyčislitelʹnaâ matematika i informatika
%D 2017
%P 37-48
%V 6
%N 2
%U http://geodesic.mathdoc.fr/item/VYURV_2017_6_2_a2/
%G en
%F VYURV_2017_6_2_a2
A. G. Feoktistov; I. A. Sidorov; V. V. Sergeev; R. O. Kostromin; V. G. Bogdanova. Virtualization of heterogeneous HPC-clusters based on OpenStack platform. Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ Vyčislitelʹnaâ matematika i informatika, Tome 6 (2017) no. 2, pp. 37-48. http://geodesic.mathdoc.fr/item/VYURV_2017_6_2_a2/

[1] V. Gergel, A. Senin, “Metacluster System for Managing the HPC Integrated Environment”, Methods and Tools of Parallel Programming Multicomputers, Second Russia-Taiwan Symposium, MTPP 2010 (Vladivostok, Russia, May 16–19 2010), LNCS, 2010, 86–94 | DOI

[2] A. Mladen, S. Eric, D. Patrick, “Integration of High-Performance Computing into Cloud Computing Services”, Handbook of Cloud Computing, 2010, 255–276 | DOI

[3] I.,V. Bychkov, G. A. Oparin, A. P. Novopashin, A.,G. Feoktistov, A. S. Korsukov, I. A. Sidorov, “High-performance computing resources of ISDCT SB RAS: State-of-the-art, prospects and future trends.”, Comput. Tech., 15 (2010), 69–81

[4] V. G. Bogdanova, I. V. Bychkov, A. S. Korsukov, G. A. Oparin, A. G. Feoktistov, “Multiagent Approach to Controlling Distributed Computing in a Cluster Grid System. J.”, Comput. Syst. Sci. Int, 53 (2014), 713–722 | DOI

[5] I. V. Bychkov, G. A. Oparin, A. G. Feoktistov, V. G. Bogdanova, A. A. Pashinin, “Serviceoriented Multiagent Control of Distributed Computations”, Automat. Rem. Contr, 76 (2015), 2000–2010 | DOI

[6] I. V. Bychkov, G. A. Oparin, A. G. Feoktistov, I. A. Sidorov, V. G. Bogdanova, S. A. Gorsky, “Multiagent Control of Computational Systems on the Basis of Meta-monitoring and Imitational Simulation”, Optoelectron., Instr. and Data Process., 52 (2016), 107–112 | DOI

[7] Irkutsk Supercomputer Center of SB RAS } {\tt http://hpc.icc.ru

[8] R. Buyya, J. Broberg, A. M. Goscinski, Cloud Computing: Principles and Paradigms, Wiley, 2011, 637 pp. | DOI

[9] S. Sridharan, A Performance Comparison of Hypervisors for Cloud Computing, University of North Florida, 2012, 269 pp.

[10] Docker } {\tt http://docker.com

[11] QEMU } {\tt http://qemu.org

[12] KVM. } {\tt http://www.linux-kvm.org

[13] Xen } {\tt http://cam.ac.uk/research/srg/netos/projects/archive/xen

[14] vSphere ESXi } {\tt https://vmware.com/support/vsphere-hypervisor.html

[15] V. K. Bumgardner, OpenStack in Action, Manning Publications, 2016, 358 pp.

[16] Apache CloudStack } {\tt https://cloudstack.apache.org/

[17] Euacalyptus } {\tt http://www.eucalyptus.com/

[18] OpenNebula } {\tt https://opennebula.org

[19] I. V. Bichkov, G. A. Oparin, A. P. Novopashin, I. A. Sidorov, “Agent-Based Approach to Monitoring and Control of Distributed Computing Environment”, Parallel Computing Technologies: 13th International Conference, PaCT 2015 (Petrozavodsk, Russia, August 31–September 4), LNCS, 2015, 253–257 | DOI

[20] I. A. Sidorov, “Methods and Tools to Increase Fault Tolerance of High-performance Computing Systems”, In proc. of the 39th International Convention on information and communication technology, electronics and microelectronics, MIPRO-2016 (Opatija, Croatia, 30 May–3 June 2016), CSICTEM, Riejka, 2016, 242–246 | DOI

[21] A. G. Feoktistov, I. A. Sidorov, “Logical-Probabilistic Analysis of Distributed Computing”, In proc. of the 39th International Convention on information and communication technology, electronics and microelectronics, MIPRO-2016 (Opatija, Croatia, 30 May–3 June 2016), CSICTEM, Riejka, 2016, 247–252 | DOI

[22] T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2001, 533 pp.

[23] L. A. Sholomov, Logicheskie metodi issledovaniya diskretnih modelei vibora [Logical Research Methods of Discrete Choice Models], Nauka, Moscow, 1989, 288 pp. http://www.minutemansoftware.com