A hybrid cascade neuro-fuzzy network with pools of extended neo-fuzzy neurons and its deep learning

Bodyanskiy, Yevgeniy V. ; Tyshchenko, Oleksii K.

International Journal of Applied Mathematics and Computer Science, Tome 29 (2019) no. 3, pp. 477-488.

Résumé

This research contribution instantiates a framework of a hybrid cascade neural network based on the application of a specific sort of neo-fuzzy elements and a new peculiar adaptive training rule. The main trait of the offered system is its competence to continue intensifying its cascades until the required accuracy is gained. A distinctive rapid training procedure is also covered for this case that offers the possibility to operate with non-stationary data streams in an attempt to provide online training of multiple parametric variables. A new training criterion is examined for handling non-stationary objects. Additionally, there is always an occasion to set up (increase) the inference order and the number of membership relations inside the extended neo-fuzzy neuron.

Détail

Keywords: data stream, membership function, training procedure, adaptive neuro-fuzzy system, extended neo-fuzzy neuron
Mots-clés : strumień danych, funkcja przynależności, neuronowo rozmyty układ adaptacyjny

Bibliographie

[1] Aggarwal, C. (2015). A Data Mining: The Textbook, Springer, New York, NY.

[2] Arrow, K., Hurwicz, L. and Uzawa, H. (1958). Iterative methods for concave programming, Studies in Linear and Nonlinear Programming 6: 154–165.

[3] Bifet, A., Gavald, R., Holmes, G. and Pfahringer, B. (2018). Machine Learning for Data Streams with Practical Examples in MOA, MIT Press, Cambridge, MA.

[4] Bodyanskiy, Y.V. and Boryachok, M.D. (1993). Optimal Control of Stochastic Objects Under Conditions of Uncertainty, ISDO, Kyiv.

[5] Bodyanskiy, Y.V., Tyshchenko, A. and Deineko, A. (2015a). An evolving radial basis neural network with adaptive learning of its parameters and architecture, Automatic Control and Computer Sciences 49(5): 255–260.

[6] Bodyanskiy, Y., Tyshchenko, O. and Kopaliani, D. (2015b). A hybrid cascade neural network with an optimized pool in each cascade, Soft Computing 19(12): 3445–3454.

[7] Bodyanskiy, Y.V., Tyshchenko, O.K. and Kopaliani, D.S. (2016). Adaptive learning of an evolving cascade neo-fuzzy system in data stream mining tasks, Evolving Systems 7(2): 107–116.

[8] Bodyanskiy, Y.V. and Tyshchenko, O.K. (2018). A hybrid cascade neural network with ensembles of extended neo-fuzzy neurons and its deep learning, in P. Kulczycki et al. (Eds), Contemporary Computational Science, AGH-UCT Press, Cracow, p. 76.

[9] Bodyanskiy, Y.V. and Tyshchenko, O.K. (2020). A hybrid cascade neural network with ensembles of extended neo-fuzzy neurons and its deep learning, in P. Kulczycki et al. (Eds), Information Technology, Systems Research and Computational Physics, Springer International Publishing, Cham, pp. 164–174.

[10] Caminhas, W.M., Lemos, A.P. and Gomide, F. (2011). Multivariable Gaussian evolving fuzzy modeling system, IEEE Transactions on Fuzzy Systems 19(1): 91–104.

[11] Delen, D. (2015). Real-World Data Mining: Applied Business Analytics and Decision Making, Pearson FT Press, New York, NY.

[12] Fahlman, S.E. and Lebiere, C. (1990). The cascade-correlation learning architecture, in D.S. Touretzky (Ed.), Advances in Neural Information Processing Systems, Morgan Kaufman, San Mateo, CA, pp. 524–532.

[13] Gama, J. (2010). Knowledge Discovery from Data Streams, Chapman and Hall/CRC, Boca Raton, FL.

[14] Goodfellow, I., Bengio, Y. and Courville, A. (2016). Deep Learning, MIT Press, Cambridge, MA.

[15] Hanrahan, G. (2011). Artificial Neural Networks in Biological and Environmental Analysis, CRC Press, Boca Raton, FL.

[16] Haykin, S. (2009). Neural Networks and Learning Machines, Prentice-Hall, Upper Saddle River, NJ.

[17] Hu, Z., Bodyanskiy, Y.V. and Tyshchenko, O.K. (2017). A deep cascade neural network based on extended neo-fuzzy neurons and its adaptive learning algorithm, Proceedings of 2017 IEEE 1st Ukraine Conference on Electrical and Computer Engineering (UKRCON), Kyiv, Ukraine, pp. 801–805.

[18] Hu, Z., Bodyanskiy, Y.V., Tyshchenko, O.K. and Boiko, O.O. (2016). An evolving cascade system based on a set of neo-fuzzy nodes, International Journal of Intelligent Systems and Applications 8(9): 1–7.

[19] Jang, J.-S.R., Sun, C.T. and Mizutani, E. (1997). Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence, Prentice-Hall, Upper Saddle River, NJ.

[20] Jaworski, M. (2018). Regression function and noise variance tracking methods for data streams with concept drift, International Journal of Applied Mathematics and Computer Science 28(3): 559–567, DOI: 10.2478/amcs-2018-0043.

[21] Kaczmarz, S. (1937). Angenherte auflsung von systemen linearer gleichungen, Bulletin International de l’Acadmie Polonaise des Sciences et des Lettres Serie A(35): 355–357.

[22] Kruse, R., Borgelt, C., Klawonn, F., Moewes, C., Steinbrecher, M. and Held, P. (2013). Computational Intelligence: A Methodological Introduction, Springer-Verlag, Berlin.

[23] Larose, D. (2014). Discovering Knowledge in Data: An Introduction to Data Mining, Wiley, New York, NY.

[24] Menshawy, A. (2018). Deep Learning By Example: A Hands-on Guide to Implementing Advanced Machine Learning Algorithms and Neural Networks, Packt Publishing Limited, Birmingham.

[25] Miki, T. and Yamakawa, T. (1999). Analog implementation of neo-fuzzy neuron and its on-board learning, in N.E. Mastorakis (Ed.), Computational Intelligence and Applications, WSES Press, Piraeus, pp. 144–149.

[26] Mumford, C. and Jain, L. (2009). Computational Intelligence, Springer-Verlag, Berlin.

[27] Otto, P., Bodyanskiy, Y. and Kolodyazhniy, V. (2003). A new learning algorithm for a forecasting neuro-fuzzy network, Integrated Computer-Aided Engineering 10(4): 399–409.

[28] Silva, A.M., Caminhas, W.M., Lemos, A.P. and Gomide, F. (2013). Evolving neo-fuzzy neural network with adaptive feature selection, 2013 BRICS Congress on Computational Intelligence/11th Brazilian Congress on Computational Intelligence, Ipojuca, Brazil, pp. 341–349.

[29] Stefanowski, J., Krawiec, K. and Wrembel, R. (2017). Exploring complex and big data, International Journal of Applied Mathematics and Computer Science 27(4): 669–679, DOI: 10.1515/amcs-2017-0046.

[30] Suzuki, K. (2013). Artificial Neural Networks: Architectures and Applications, InTech, New York, NY.

[31] Takagi, T. and Sugeno, M. (1985). Fuzzy identification of systems and its application to modeling and control, IEEE Transactions on Systems, Man, and Cybernetics SMC-15(1): 116–132.

[32] Uchino, E. and Yamakawa, T. (1997). Soft computing based signal prediction, restoration and filtering, in N.E. Mastorakis (Ed.), Intelligent Hybrid Systems: Fuzzy Logic, Neural Networks and Genetic Algorithms, Kluwer Academic Publisher, Boston, MA, pp. 331–349.

[33] Wang, L. (1994). Adaptive Fuzzy Systems and Control. Design and Stability Analysis, Prentice-Hall, Upper Saddle River, NJ.

[34] Wang, L. and Mendel, J.M. (1993). Fuzzy basis functions, universal approximation and orthogonal least squares learning, IEEE Transactions on Neural Networks 3(5): 807–814.

[35] Yamakawa, T., Uchino, E., Miki, T. and Kusanagi, H. (1992). A neo fuzzy neuron and its applications to system identification and prediction of the system behavior, Proceedings of the 2nd International Conference on Fuzzy Logic and Neural Networks, Iizuka, Japan, pp. 477–483.