Development and application of the computational model for skeleton solutions. Case study – using ``bag-of-task'' for HRBF neural network learning

V. G. Litvinov

Journal of Samara State Technical University, Ser. Physical and Mathematical Sciences, no. 3 (2014), pp. 183-195.

Résumé

The article proposes a solution to the problem of mapping an algorithm from the field of Computational Mathematics on the target computing environment. The solution is based on a formal method for constructing parallel skeletons. The method comprises a specification of concurrency with the directed graphs and a formula for interpretation of dynamic behavior of such graphs. This interpretation is based on Temporal Logic of Actions approach proposed by Leslie Lamport. To illustrate the use of the method the "bag-of-tasks’’ parallel skeleton is discussed hereinafter. We present graphically basic skeleton operations with the proposed computational model. After that we specify a learning algorithm of hyper-radial basis function neural network in the terms of skeleton operations as a case study. This made it possible to parallelize the leaning algorithm and map it on desired computing environments with predefined run-time libraries. Computational experiments confirming that our approach does not reduce the performance of the resulting programs are presented. The approach is suitable for researchers not familiar with parallel computing. It helps to get a reliable and effective supercomputer application both for SMP and distributed architectures

Détail

Keywords: cluster, supercomputing, Templet language, pattern, bag-of-tasks, skeleton programming, model of computation, HRBF neural network.

Bibliographie

[1] Sutter H., “The free lunch is over: A fundamental turn toward concurrency in software”, Dr. Dobb’s journal, 30:3 (2005), 202–210

[2] Marchuk G. I., Kotov V. E., “Problems of computer technology and fundamental research”, Avtom. i vychisl. tekhn., 1979, no. 2, 3–14 (In Russian) | MR

[3] Gamma E., Helm R., Johnson R., Vlissides J., Design patterns: elements of reusable object-oriented software, Addison-Wesley, Westford, 1995, 395 pp.

[4] Cole M. I., Algorithmic skeletons: structured management of parallel computation, MIT Press, Massachusetts, Cambridge, Boston, 1989, v+132 pp. | MR

[5] MacDonald S., Szafron D., Schaeffer J., Bromling S., “Generating parallel program frameworks from parallel design patterns”, Euro-Par 2000 Parallel Processing, Lecture Notes in Computer Science, 1900, Springer, Berlin, Heidelberg, 2000, 95–104 | DOI | MR

[6] Schmidt D. C., Huston S. D., C++ Network Programming, v. I, Mastering Complexity with ACE and Patterns, FT Press, 2001, 336 pp. | Zbl

[7] Berzigiyarov P. K., “Programming using standardized algorithmic structures with massive parallelism”, Vychislitel'nye metody i programmirovanie, 2:2 (2001), 1–16 (In Russian)

[8] Kraeva M. A., Malyshkin V. E., “Assembly technology for parallel realization of numerical models on mimd-multicomputers”, Future Generation Computer Systems, 17:6 (2001), 755–765 | DOI | Zbl

[9] Botorog G. H., Kuchen H., “Skil: An imperative language with algorithmic skeletons for efficient distributed programming”, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing, HPDC-96, 1996, 243–252 pp. | DOI

[10] Dorta A. J., González J. A., Rodríguez C., De Sande F., “Llc: A parallel skeletal language”, Parallel Processing Letters, 13:03 (2003), 437–448 | DOI | MR

[11] Iosup A., Sonmez O., Anoep S., Epema D., “The performance of bags-of-tasks in large-scale distributed systems”, Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08, 2008, 97–108 | DOI

[12] Reinders J., Intel Threading Building Blocks: Outfitting C++ for Multi-core Processor Parallelism, O'Reilly Media, Beijing, Cambridg, etc., 2007, 336 pp.

[13] Blumofe R. D., Joerg C. F., Kuszmaul B. C., Leiserson C. E., Randall K. H., Zhou Y., “Cilk: An Efficient Multithreaded Runtime System”, Journal of Parallel and Distributed Computing, 37:1 (1996), 55–69 | DOI

[14] “OpenMP: an industry standard API for shared-memory programming”, IEEE Computational Science and Engineering, 5:1 (1998), 46–55 | DOI | Zbl

[15] Konovalov N. A., Krukov V. A., Mihailov S. N., Pogrebtsov A. A., “Fortran dvm — a language for portable parallel program development”, Proceedings of Software For Multiprocessors Supercomputers: Theory, Practice, Experience, 1994, 124–133

[16] Lamport L., “The temporal logic of actions”, ACM Transactions on Programming Languages and Systems, 16:3 (1994), 872–923 | DOI

[17] Lamport L., Specifying systems: The TLA+ Language and Tools for Hardware and Software Engineers, Addison-Wesley, Boston, San Francisco, etc., 2002, xviii+364 pp.

[18] Vostokin S. V., Graficheskaia ob"ektnaia model' parallel'nykh protsessov i ee primenenie v zadachakh chislennogo modelirovaniia [Graphical object model of parallel processes and its appication in the numerical modeling problem], Samara Research Center of RAS, Samara, 2007, 286 pp. (In Russian)

[19] Vostokin S. V., “Parallel programming automation system Graphplus Templet”, Parallel'nye vychisleniia i zadachi upravleniia, PACO’2010 \rm [Parallel Computing and Control Problems], Proc. Fifth International Conf., Institute of Control Sciences, Moscow, 2010, 1143–1156 (In Russian)

[20] Vostokin S. V., “Templet – A method of process oriented simulation of parallelism”, Programmnye produkty i sistemy, 2012, no. 3, 11–14 (In Russian)

[21] Vostokin S. V., Litvinov V. G., Makagonova D. D., Khairutdinov A. R., “Visual modeling of parallel algorithms in process oriented notation Templet”, Parallel'nye vychisleniia i zadachi upravleniia, PACO’2012 \rm [Parallel Computing and Control Problems], Proc. Sixth International Conf., v. 1, Institute of Control Sciences, Moscow, 2012, 260–269 (In Russian)

[22] Osovskii S., Neironnye seti dlia obrabotki informatsii [Neural network for processing information], Finansy i statistika, Moscow, 2004, 344 pp.

[23] Haykin S. O., Neural Networks and Learning Machines (3rd Edition), Prentice Hall, 2008, 936 pp.

[24] Litvinov V. G., Soldatova O. P., “Methods of constructing optimal structure of the HRBF neural network for solving forecasting problem”, Perspektivnye informatsionnye tekhnologii dlia aviatsii i kosmosa, PIT-2010 [Advanced information technologies for aviation and cosmos], Samara, Samara State Aerospace Univ., 2010, 252–254

[25] Vostokin S. V., Khayrutdinov A. R., Litvinov V. G., “Parallel programming software package Graphplus templet”, Vestn. Samar. Gos. Tekhn. Univ. Ser. Fiz.-Mat. Nauki, 2011, no. 4(25), 146–153 (In Russian) | DOI

[26] Artamonov Yu. S., Vostokin S. V., Nazarov Yu. P., “Templet – A service of continuous integration to the development of high-performance applications”, Vysokoproizvoditel'nye parallel'nye vychisleniia na klasternykh sistemakh [High-performance parallel computations on cluster systems], Proceedings of XII All-Russian Conference, Nizhni Novgorod State Univ. Publ., Nizhni Novgorod, 2012, 82 (In Russian)