An outlier-robust neuro-fuzzy system for classification and regression

Siminski, Krzysztof

International Journal of Applied Mathematics and Computer Science, Tome 31 (2021) no. 2, pp. 303-319.

Résumé

Real life data often suffer from non-informative objects—outliers. These are objects that are not typical in a dataset and can significantly decline the efficacy of fuzzy models. In the paper we analyse neuro-fuzzy systems robust to outliers in classification and regression tasks. We use the fuzzy c-ordered means (FCOM) clustering algorithm for scatter domain partition to identify premises of fuzzy rules. The clustering algorithm elaborates typicality of each object. Data items with low typicalities are removed from further analysis. The paper is accompanied by experiments that show the efficacy of our modified neuro-fuzzy system to identify fuzzy models robust to high ratios of outliers.

Détail

Keywords: outliers, neuro-fuzzy system, clustering algorithm, regression
Mots-clés : wyjątki, system neurorozmyty, algorytm grupowania

Bibliographie

[1] [1] Acı, M. and Avcı, M. (2016). Artificial neural network approach for atomic coordinate prediction of carbon nanotubes, Applied Physics A 122(631).

[2] [2] Acı, M. and Avcı, M. (2017). Reducing simulation duration of carbon nanotube using support vector regression method, Journal of the Faculty of Engineering and Architecture of Gazi University 32(3): 901–907.

[3] [3] Alcalá, R., Alcalá-Fdez, J., Casillas, J., Cordón, O. and Herrera, F. (2006). Hybrid learning models to get the interpretability-accuracy trade-off in fuzzy modeling, Soft Computing 10(9): 717–734.

[4] [4] Alonso, J.M. and Magdalena, L. (2011). Special issue on interpretable fuzzy systems, Information Sciences 181(20): 4331–4339.

[5] [5] Bartczuk, L., Przybyl, A. and Cpalka, K. (2016). A new approach to nonlinear modelling of dynamic systems based on fuzzy rules, International Journal of Applied Mathematics and Computer Science 26(3): 603–621, DOI: 10.1515/amcs-2016-0042.

[6] [6] Cpałka, K., Łapa, K., Przybył, A. and Zalasiński, M. (2014). A new method for designing neuro-fuzzy systems for nonlinear modelling with interpretability aspects, Neurocomputing 135: 203–217.

[7] [7] Czogała, E. and Łęski, J. (2000). Fuzzy and Neuro-Fuzzy Intelligent Systems, Physica-Verlag, Heidelberg/New York.

[8] [8] Dave, R. and Krishnapuram, R. (1997). Robust clustering methods: A unified view, Fuzzy Systems, IEEE Transactions on 5(2): 270–293.

[9] [9] de Souza, P.V.C. (2020). Fuzzy neural networks and neuro-fuzzy networks: A review the main techniques and applications used in the literature, Applied Soft Computing 92: 106275.

[10] [10] Dovžan, D. and Škrjanc, I. (2011). Recursive fuzzy c-means clustering for recursive fuzzy identification of time-varying processes, ISA Transactions 50(2): 159–169.

[11] [11] Dunn, J.C. (1973). A fuzzy relative of the ISODATA process and its use in detecting compact, well separated clusters, Journal Cybernetics 3(3): 32–57.

[12] [12] D’Urso, P. and Leski, J.M. (2020). Fuzzy clustering of fuzzy data based on robust loss functions and ordered weighted averaging, Fuzzy Sets and Systems 389: 1–28.

[13] [13] Evsukoff, A.G., Galichet, S., de Lima, B.S. and Ebecken, N.F. (2009). Design of interpretable fuzzy rule-based classifiers using spectral analysis with structure and parameters optimization, Fuzzy Sets and Systems 160(7): 857–881.

[14] [14] Frank, A. and Asuncion, A. (2019). UCI machine learning repository, http://archive.ics.uci.edu/ml.

[15] [15] Geng, Y., Li, Q., Zheng, R., Zhuang, F. and He, R. (2018). RECOME: A new density-based clustering algorithm using relative KNN kernel density, Information Sciences 436–437: 13–30.

[16] [16] Grzegorzewski, P., Hryniewicz, O. and Romaniuk, M. (2020). Flexible resampling for fuzzy data, International Journal of Applied Mathematics and Computer Science 30(2): 281–297, DOI: 10.34768/amcs-2020-0022.

[17] [17] Haberman, S.J. (1976). Generalized residuals for log-linear models, Proceedings of the 9th International Biometrics Conference, Boston, USA, pp. 104–122.

[18] [18] Harifi, S., Khalilian, M., Mohammadzadeh, J. and Ebrahimnejad, S. (2020). Optimizing a neuro-fuzzy system based on nature-inspired emperor penguins colony optimization algorithm, IEEE Transactions on Fuzzy Systems 28(6): 1110–1124.

[19] [19] Hathaway, R.J., Bezdek, J.C. and Hu, Y. (2000). Generalized fuzzy c-means clustering strategies using lp norm distances, IEEE Transactions on Fuzzy Systems 8(5): 576–582.

[20] [20] Hekimoglu, S., Erdogan, B. and Erenoglu, R. (2015). A new outlier detection method considering outliers as model errors, Experimental Techniques 39(1): 57–68.

[21] [21] Hekimoglu, S. and Koch, K.R. (2000). How can reliability of the test for outliers be measured?, Allgemeine Vermessungsnachrichten 7: 247–253.

[22] [22] Jakubek, S. and Keuth, N. (2006). A local neuro-fuzzy network for high-dimensional models and optimalization, Engineering Applications of Artificial Intelligence 19(6): 705–717.

[23] [23] Jang, J.-S.R. (1993). ANFIS: Adaptive-network-based fuzzy inference system, IEEE Transactions on Systems, Man, and Cybernetics 23(3): 665–684.

[24] [24] Jiang, Y. and Yin, S. (2019). Recent advances in key-performance-indicator oriented prognosis and diagnosis with a Matlab toolbox: DB-KIT, IEEE Transactions on Industrial Informatics 15(5): 2849–2858.

[25] [25] Jiang, Y., Yin, S. and Kaynak, O. (2018). Data-driven monitoring and safety control of industrial cyber-physical systems: Basics and beyond, IEEE Access 6: 47374–47384.

[26] [26] Johnson, B., Tateishi, R. and Hoan, N. (2013). A hybrid pansharpening approach and multiscale object-based image analysis for mapping diseased pine and oak trees, International Journal of Remote Sensing 34(20): 6969–6982.

[27] [27] Kaya, H., Tüfekci, P. and Gürgen, S.F. (2012). Local and global learning methods for predicting power of a combined gas and steam turbine, Proceedings of the International Conference on Emerging Trends in Computer and Electronics Engineering (ICETCEE 2012), Dubai, UAE, pp. 13–18.

[28] [28] Keith, M.J., Jameson, A., van Straten, W., Bailes, M., Johnston, S., Kramer, M., Possenti, A., Bates, S.D., Bhat, N.D.R., Burgay, M., Burke-Spolaor, S., D’Amico, N., Levin, L., McMahon, P.L., Milia, S. and Stappers, B.W. (2010). The high time resolution universe pulsar survey. I: System configuration and initial discoveries, Monthly Notices of the Royal Astronomical Society 409(2): 619–627.

[29] [29] Kłopotek, R., Kłopotek, M. and Wierzchoń, S. (2020). A feasible k-means kernel trick under non–Euclidean feature space, International Journal of Applied Mathematics and Computer Science 30(4): 703–715, DOI: 10.34768/amcs-2020-0052.

[30] [30] Krishnapuram, R. and Keller, J. (1993). A possibilistic approach to clustering, IEEE Transactions on Fuzzy Systems 1(2): 98–110.

[31] [31] Latecki, L.J., Lazarevic, A. and Pokrajac, D. (2007). Outlier detection with kernel density functions, in P. Perner (Ed.), Machine Learning and Data Mining in Pattern Recognition, Springer, Berlin/Heidelberg, pp. 61–75.

[32] [32] Lehmann, R. (2013). 3σ-Rule for outlier detection from the viewpoint of geodetic adjustment, Journal of Surveying Engineering 139(4): 157–165.

[33] [33] Leski, J. and Kotas, M. (2015). On robust fuzzy c-regression models, Fuzzy Sets and Systems 279: 112–129.

[34] [34] Leski, J.M. (2014). Fuzzy c-ordered-means clustering, Fuzzy Sets and Systems 286: 114–133.

[35] [35] Leski, J.M. (2015). Fuzzy (c + p)-means clustering and its application to a fuzzy rule-based classifier: Towards good generalization and good interpretability, IEEE Transactions on Fuzzy Systems 23(4): 802–812.

[36] [36] Leski, J.M. and Kotas, M.P. (2018). Linguistically defined clustering of data, International Journal of Applied Mathematics and Computer Science 28(3): 545–557, DOI: 10.2478/amcs-2018-0042.

[37] [37] Leys, C., Ley, C., Klein, O., Bernard, P. and Licata, L. (2013). Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median, Journal of Experimental Social Psychology 49(4): 764–766.

[38] [38] Liang, X., Zou, T., Guo, B., Li, S., Zhang, H., Zhang, S., Huang, H. and Chen, S.X. (2015). Assessing Beijing’s PM2.5 pollution: Severity, weather impact, apec and winter heating, Proceedings of the Royal Society A 471(2182): 257–276.

[39] [39] Lyon, R.J., Stappers, B.W., Cooper, S., Brooke, J.M. and Knowles, J.D. (2016). Fifty years of pulsar candidate selection: From simple filters to a new principled real-time classification approach, Monthly Notices of the Royal Astronomical Society 459(1): 1104–1123.

[40] [40] Mamdani, E.H. and Assilian, S. (1975). An experiment in linguistic synthesis with a fuzzy logic controller, International Journal of Man-Machine Studies 7(1): 1–13.

[41] [41] Matthews, S.G., Gongora, M.A. and Hopgood, A.A. (2013). Evolutionary algorithms and fuzzy sets for discovering temporal rules, International Journal of Applied Mathematics and Computer Science 23(4): 855–868, DOI: 10.2478/amcs-2013-0064.

[42] [42] Nowicki, R. (2006). Rough-neuro-fuzzy system with MICOG defuzzification, 2006 IEEE International Conference on Fuzzy Systems, Vancouver, Canada, pp. 1958–1965.

[43] [43] Olson, C.C., Judd, K.P. and Nichols, J.M. (2018). Manifold learning techniques for unsupervised anomaly detection, Expert Systems with Applications 91: 374–385.

[44] [44] Otte, C. (Ed.) (2013). Safe and interpretable machine learning: A methodological review, in C. Moewes and A. Nürnberger (Eds), Computational Intelligence in Intelligent Data Analysis, Springer, Berlin/Heidelberg, pp. 111–122.

[45] [45] Piegat, A. and Dobryakova, L. (2020). A decomposition approach to type 2 interval arithmetic, International Journal of Applied Mathematics and Computer Science 30(1): 185–201, DOI: 10.34768/amcs-2020-0015.

[46] [46] Reichenbach, H. (1935). Wahrscheinlichkeitslogik, Erkenntnis 5: 37–43.

[47] [47] Riid, A. (2002). Transparent Fuzzy Systems: Modelling and Control, PhD dissertation, Tallinn Technical University, Tallinn.

[48] [48] Rocha Neto, A.R. and Barreto, G.A. (2009). On the application of ensembles of classifiers to the diagnosis of pathologies of the vertebral column: A comparative analysis, IEEE Latin America Transactions 7(4): 487–496.

[49] [49] Seresht, N.G. and Fayek, A.R. (2020). Neuro-fuzzy system dynamics technique for modeling construction systems, Applied Soft Computing 93: 106400.

[50] [50] Sholla, S., Mir, R.N. and Chishti, M.A. (2020). A neuro fuzzy system for incorporating ethics in the Internet of things, Journal of Ambient Intelligence and Humanized Computing 12: 1487–1501.

[51] [51] Sikora, M. and Krzykawski, D. (2005). Application of data exploration methods in analysis of carbon dioxide emission in hard-coal mines, dewater pump stations, Mechanizacja i Automatyzacja Górnictwa 413(6): 57–67.

[52] [52] Siminski, K. (2008). Neuro-fuzzy system with hierarchical domain partition, Proceedings of the International Conference on Computational Intelligence for Modelling, Control and Automation (CIMCA 2008), Vienna, Austria, pp. 392–397.

[53] [53] Siminski, K. (2009). Patchwork neuro-fuzzy system with hierarchical domain partition, in M. Kurzyński and M. Woźniak (Eds), Computer Recognition Systems 3, Advances in Intelligent and Soft Computing, Vol. 57, Springer-Verlag, Berlin/Heidelberg, pp. 11–18.

[54] [54] Simiński, K. (2010). Rule weights in a neuro-fuzzy system with a hierarchical domain partition, International Journal of Applied Mathematics and Computer Science 20(2): 337–347, DOI: 10.2478/v10006-010-0025-3.

[55] [55] Simiński, K. (2012). Neuro-rough-fuzzy approach for regression modelling from missing data, International Journal of Applied Mathematics and Computer Science 22(2): 461–476, DOI: 10.2478/v10006-012-0035-4.

[56] [56] Siminski, K. (2014). Neuro-fuzzy system based kernel for classification with support vector machines, in A. Gruca et al. (Eds), Man–Machine Interactions 3, Springer International Publishing, Cham, pp. 415–422.

[57] [57] Siminski, K. (2015). Rough subspace neuro-fuzzy system, Fuzzy Sets and Systems 269: 30–46.

[58] [58] Siminski, K. (2016). Memetic neuro-fuzzy system with Big-Bang-Big-Crunch optimisation, in A. Gruca et al. (Eds), Man–Machine Interactions 4, Springer International Publishing, Cham, pp. 583–592.

[59] [59] Siminski, K. (2017a). Fuzzy weighted c-ordered means clustering algorithm, Fuzzy Sets and Systems 318: 1–33.

[60] [60] Siminski, K. (2017b). Interval type-2 neuro-fuzzy system with implication-based inference mechanism, Expert Systems with Applications 79C: 140–152.

[61] [61] Siminski, K. (2017c). Robust subspace neuro-fuzzy system with data ordering, Neurocomputing 238: 33–43.

[62] [62] Siminski, K. (2019). NFL—Free library for fuzzy and neuro-fuzzy systems, in S. Kozielski et al. (Eds), Beyond Databases, Architectures and Structures: Paving the Road to Smart Data Processing and Analysis, Springer International Publishing, Cham, pp. 139–150.

[63] [63] Siminski, K. (2020). FIT2COMIn—Robust clustering algorithm for incomplete data, in A. Gruca et al. (Eds), Man–Machine Interactions 6, Springer International Publishing, Cham, pp. 99–110.

[64] [64] Škrjanc, I., Iglesias, J.A., Sanchis, A., Leite, D., Lughofer, E. and Gomide, F. (2019). Evolving fuzzy and neuro-fuzzy approaches in clustering, regression, identification, and classification: A survey, Information Sciences 490: 344–368.

[65] [65] Słowik, A., Cpałka, K. and Łapa, K. (2020). Multipopulation nature-inspired algorithm (MNIA) for the designing of interpretable fuzzy systems, IEEE Transactions on Fuzzy Systems 28(6): 1125–1139.

[66] [66] Sugeno, M. and Kang, G.T. (1988). Structure identification of fuzzy model, Fuzzy Sets and Systems 28(1): 15–33.

[67] [67] Takagi, T. and Sugeno, M. (1985). Fuzzy identification of systems and its application to modeling and control, IEEE Transactions on Systems, Man and Cybernetics 15(1): 116–132.

[68] [68] Tang, B. and He, H. (2017). A local density-based approach for outlier detection, Neurocomputing 241: 171–180.

[69] [69] Timm, H., Borgelt, C., Döring, C. and Kruse, R. (2004). An extension to possibilistic fuzzy cluster analysis, Fuzzy Sets and Systems 147: 3–16.

[70] [70] Tüfekci, P. (2014). Prediction of full load electrical power output of a base load operated combined cycle power plant using machine learning methods, International Journal of Electrical Power Energy Systems 60: 126–140.

[71] [71] Yager, R.R. (1988). On ordered weighted averaging aggregation operators in multicriteria decisionmaking, IEEE Transactions on Systems, Man, and Cybernetics 18(1): 183–190.

[72] [72] Yang, P., Zhu, Q. and Zhong, X. (2009). Subtractive clustering based RBF neural network model for outlier detection, Journal of Computers 4(8): 755–762.

[73] [73] Yeh, I.-C., Yang, K.-J. and Ting, T.-M. (2009). Knowledge discovery on RFM model using Bernoulli sequence, Expert Systems with Applications 36(3): 5866–5871.

[74] [74] Youden, W.J. (1950). Index for rating diagnostic tests, Cancer 3(1): 32–35.

[75] [75] Zadeh, L.A. (1973). Outline of a new approach to the analysis of complex systems and decision processes, IEEE Transactions on Systems, Man, and Cybernetics SMC-3(1): 28–44.