Pattern layer reduction for a generalized regression neural network by using a self-organizing map

Kartal, S. ; Oral, M. ; Ozyildirim, B. M.

International Journal of Applied Mathematics and Computer Science, Tome 28 (2018) no. 2, pp. 411-424.

Résumé

In a general regression neural network (GRNN), the number of neurons in the pattern layer is proportional to the number of training samples in the dataset. The use of a GRNN in applications that have relatively large datasets becomes troublesome due to the architecture and speed required. The great number of neurons in the pattern layer requires a substantial increase in memory usage and causes a substantial decrease in calculation speed. Therefore, there is a strong need for pattern layer size reduction. In this study, a self-organizing map (SOM) structure is introduced as a pre-processor for the GRNN. First, an SOM is generated for the training dataset. Second, each training record is labelled with the most similar map unit. Lastly, when a new test record is applied to the network, the most similar map units are detected, and the training data that have the same labels as the detected units are fed into the network instead of the entire training dataset. This scheme enables a considerable reduction in the pattern layer size. The proposed hybrid model was evaluated by using fifteen benchmark test functions and eight different UCI datasets. According to the simulation results, the proposed model significantly simplifies the GRNN’s structure without any performance loss.

Détail

Keywords: generalized regression neural network, artificial neural network, self organizing map, nearest neighbour, reduced dataset
Mots-clés : sztuczna sieć neuronowa, mapa samoorganizująca, metoda najbliższych sąsiadów, redukcja zbioru danych

Bibliographie

[1] Bache, K. and Lichman, M. (2013). UCI Machine Learning Repository, University of California, Irvine, CA.

[2] Berkhin, P. (2002). Survey of clustering data mining techniques, Technical report, Accrue Software, https://www.cc.gatech.edu/˜isbell/reading/papers/berkhin02survey.pdf.

[3] Bezdek, J.C., Ehrlich, R. and Full, W. (1984). FCM: The fuzzy c-means clustering algorithm, Computers Geosciences 10(2–3): 191–203.

[4] Bowden, G.J., Dandy, G.C. and Maier, H.R. (2005). Input determination for neural network models in water resources applications. Part 1—Background and methodology, Journal of Hydrology 301(1): 75–92.

[5] Caliński, T. and Harabasz, J. (1974). A dendrite method for cluster analysis, Communications in Statistics—Theory and Methods 3(1): 1–27.

[6] Carrasco Kind, M. and Brunner, R.J. (2014). SOMs: Photometric redshift PDFs with self-organizing maps and random atlas, Monthly Notices of the Royal Astronomical Society 438(4): 3409–3421.

[7] Cover, T. and Hart, P. (1967). Nearest neighbor pattern classification, IEEE Transactions on Information Theory 13(1): 21–27.

[8] Davies, D.L. and Bouldin, D.W. (1979). A cluster separation measure, IEEE Transactions on Pattern Analysis and Machine Intelligence 1(2): 224–227.

[9] Hamzacebi, C. (2008). Improving genetic algorithms performance by local search for continuous function optimization, Journal of Applied Mathematics and Computation 196(1): 309–317.

[10] Harkanth, S. and Phulpagar, B.D. (2013). A survey on clustering methods and algorithms, International Journal of Computer Science and Information Technologies 4(5): 687–691.

[11] Hartigan, J.A. and Wong, M.A. (1979). Algorithm AS 136: A k-means clustering algorithm, Journal of the Royal Statistical Society C: Applied Statistics 28(1): 100–108.

[12] Husain, H., Khalid, M. and R., Y. (2004). Automatic clustering of generalized regression neural network by similarity index based fuzzy c-means clustering, IEEE Region 10 Conference, Chiang Mai, Thailand, pp. 302–305.

[13] Jain, A.K., Mao, J. and Mohiuddin, K.M. (1998). Artificial neural networks: A tutorial, IEEE Computer 29(3): 31–44.

[14] Kohonen, T. (1982). Self-organized formation of topologically correct feature maps, Biological Cybernetics 43(1): 59–69.

[15] Kokkinos, Y. and Margaritis, K.G. (2015). A fast progressive local learning regression ensemble of generalized regression neural networks, Proceedings of the 19th Panhellenic Conference on Informatics, Athens, Greece, pp. 109–114.

[16] Kolesnikov, A., Trichina, E. and Kauranne, T. (2015). Estimating the number of clusters in a numerical data set via quantization error modeling, Pattern Recognition 48(3): 941–952.

[17] Kotsiantis, S.B. and Pintelas, P.E. (2004). Recent advances in clustering: A brief survey, WSEAS Transactions on Information Science and Applications 1(1): 73–81.

[18] Krenker, A., Bester, J. and Kos, A. (2011). Introduction to the artificial neural networks, in K. Suzuki (Ed.), Artificial Neural Networks—Methodological Advances and Biomedical Applications, Intech, Rijeka, pp. 3–18.

[19] Maier, H. and Dandy, G. (1997). Determining inputs for neural network models of multivariate time series, Microcomputers in Civil Engineering 12(5): 353368.

[20] Rama, B., Jayashree, P. and Jiwani, S. (2010). A survey on clustering, current status and challenging issues, International Journal on Computer Science and Engineering 2(9): 2976–2980.

[21] Rousseeuw, P.J. (1987). Silhouettes: A graphical aid to the interpretation and validation of cluster analysis, Journal of Computational and Applied Mathematics 20: 53–65.

[22] Sabo, K. (2014). Center-based l1-clustering method, International Journal of Applied Mathematics and Computer Science 24(1): 151–163, DOI: 10.2478/amcs-2014-0012.

[23] Specht, D.F. (1991). A general regression neural network, IEEE Transactions on Neural Networks 2(6): 568–576.

[24] Szemenyei, M. and Vajda, F. (2017). Dimension reduction for objects composed of vector sets, International Journal of Applied Mathematics and Computer Science 27(1): 169–180, DOI: 10.1515/amcs-2017-0012.

[25] Tang, K., Li, X., Suganthan, P.N., Yang, Z. and Weise, T. (2009). Benchmark functions for the CEC’2010 special session and competition on large scale global optimization, Technical report, Nature Inspired Computation and Applications Laboratory, USTC, Hefei.

[26] Tibshirani, R., Walther, G. and Hastie, T. (2001). Estimating the number of clusters in a data set via the gap statistic, Journal of the Royal Statistical Society B: Statistical Methodology 63(2): 411–423.

[27] Yuen, R.K.K., Lee, E.W.M., Lim, C.P. and Cheng, G.W.Y. (2004). Fusion of GRNN and FA for online noisy data regression, Neural Processing Letters (19): 227–241.

[28] Zhao, S.J., Zhang, J.L., Li, X. and Song,W. (2007). Generalized regression neural network based on fuzzy means clustering and its application in system identification, Proceedings of the International Symposium on Information Technology Convergence, Joenju, South Korea, pp. 13–16.

[29] Zheng, L.G., Yu, M.G., Yu, S.J. and Wang,W. (2008). Improved prediction of nitrogen oxides using GRNN with k-means clustering and EDA, Proceedings of the 4th International Conference on Natural Computation, Jinan, China, pp. 91–95.