Extraction of fuzzy rules using deterministic annealing integrated with ε-insensitive learning

Czabański, R.

International Journal of Applied Mathematics and Computer Science, Tome 16 (2006) no. 3, pp. 357-372.

Résumé

A new method of parameter estimation for an artificial neural network inference system based on a logical interpretation of fuzzy if-then rules (ANBLIR) is presented. The novelty of the learning algorithm consists in the application of a deterministic annealing method integrated with ε-insensitive learning. In order to decrease the computational burden of the learning procedure, a deterministic annealing method with a “freezing” phase and ε-insensitive learning by solving a system of linear inequalities are applied. This method yields an improved neuro-fuzzy modeling quality in the sense of an increase in the generalization ability and robustness to outliers. To show the advantages of the proposed algorithm, two examples of its application concerning benchmark problems of identification and prediction are considered.

Détail

Keywords: fuzzy systems, neural networks, neuro-fuzzy systems, rules extraction, deterministic annealing
Mots-clés : system rozmyty, sieć neuronowa, sieć neuronowa rozmyta, ekstrakcja reguł

Bibliographie

[1] Bezdek J.C. (1982): Pattern Recognition with Fuzzy Objective Function Algorithms.- New York: Plenum Press.

[2] Box G.E.P. and Jenkins G.M. (1976): Time Series Analysis. Forecasting and Control. -San Francisco: Holden-Day.

[3] Czabański R. (2003): Automatic Fuzzy If-Then Rules Extraction from Numerical Data. - Ph.D. thesis, Silesian University of Technology, Gliwice, (in Polish).

[4] Czabański R. (2005): Neuro-fuzzy modeling based on deterministic annealing approach. - Int. J. Appl. Math. Comput. Sci., Vol. 15, No. 4, pp. 125-134.

[5] Czogała E. and Łęski J. (1996): A new fuzzy inference system with moving consequents in if-then rules. Application to pattern recognition.-Bull. Polish Acad. Science, Vol. 45, No. 4, pp. 643-655.

[6] Czogała E. and Łęski J. (1999): Fuzzy and Neuro-Fuzzy Intelligent Systems. -Heidelberg: Physica-Verlag.

[7] Czogała E. and Łęski J. (2001): On equivalence of approximate reasoning results using different interpretations of if-then rules. -Fuzzy Sets Syst., Vol. 117, No. 2, pp. 279-296.

[8] German S. and German D. (1984): Stochastic relaxation, Gibbs distribution and the Bayesian restoration in images. - IEEE Trans. Pattern Anal. Mach. Intell., Vol. 6, pp. 721-741.

[9] Ho D. and Kashyap R.L. (1965): An algorithm for linear inequalities and its applications. - IEEE Trans. Elec. Comp., Vol. 14, No. 5, pp. 683-688.

[10] Ho Y.C. and Kashyap R.L. (1966): A class of iterative procedures for linear inequalities. - SIAM J. Contr., Vol. 4, No. 2, pp. 112-115.

[11] Jang J.S.R. (1993): ANFIS: Adaptive-network-based fuzzy inference system. - IEEE Trans. Syst. Man Cybern., Vol. 23, No. 3, pp. 665-685.

[12] Jang J.S.R. and Sun J.S.R. (1993): Functional equivalence between radial basis function networks and fuzzy inference systems. - IEEE Trans. Neural Netw., Vol. 4, No. 1, pp. 156-159.

[13] Jang J.S.R., Sun C.T. and Mizutani E. (1997): Neuro-Fuzzy and Soft Computing. A Computational Approach to Learning and Machine Intelligence. - Upper Saddle River: Prentice-Hall.

[14] Kirkpatrick S., Gelatt C. and Vecchi M. (1983): Optimization by simulated annealing. - Science, Vol. 220, No. 4598, pp. 671-680.

[15] Łęski J. (2002): Improving generalization ability of neuro-fuzzy systems by ε-insensitive learning. - Int. J. Appl. Math. Comput. Sci., Vol. 12, No. 3, pp. 437-447.

[16] Łęski J. (2003a): Neuro-fuzzy system with learning tolerant to imprecision.-Fuzzy Sets Syst., Vol. 138, No. 2, pp. 427- 439.

[17] Łęski J. (2003b): ε-Insensitive learning techniques for approximate reasoning systems.-Int. J. Comput. Cognit., Vol. 1, No. 1, pp. 21-77.

[18] Metropolis N., Rosenbluth A.W., Rosenbluth M.N., Teller A.H. and Teller E. (1953): Equation of state calculation by fast computing machines. - J. Chem. Phys., Vol. 21, No. 6, pp. 1087-1092.

[19] Mitra S. and Hayashi Y. (2000): Neuro-fuzzy rule generation: Survey in soft computing framework. - IEEE Trans. Neural Netw., Vol. 11, No. 3, pp. 748-768.

[20] Rao A.V., Miller D., Rose K. and Gersho A. (1997): Mixture of experts regression modeling by deterministic annealing. -IEEE Trans. Signal Process., Vol. 45, No. 11, pp. 2811-2820.

[21] Rao A.V., Miller D., Rose K. and Gersho A. (1999): A deterministic annealing approach for parsimonious design of piecewise regression models.-IEEE Trans. Pattern Anal. Mach. Intell., Vol. 21, No. 2, pp. 159-173.

[22] Rose K. (1991): Deterministic Annealing, Clustering and Optimization. -Ph.D. thesis, California Inst. Tech., Pasadena.

[23] Rose K. (1998): Deterministic annealing for clustering, compression, classification, regression and related optimization problems. - Proc. IEEE, Vol. 86, No. 11, pp. 2210-2239.

[24] Vapnik V. (1998): Statistical Learning Theory. - New York: Wiley.

[25] Vapnik V. (1999): An overview of statistical learning theory. - IEEE Trans. Neural Netw., Vol. 10, No. 5, pp. 988-999.

[26] Weigend A.S., Huberman B.A. and Rumelhart D.E. (1990): Predicting the future: A connectionist approach. - Int. J. Neural Syst., Vol. 1, No. 2, pp. 193-209.

[27] Xie X.L. and Beni G. (1991): A validity measure for fuzzy clustering.- IEEE Trans. Pattern Anal. Mach. Intell., Vol. 13, No. 8, pp. 841-847.

[28] Yen J.,Wang L. and Gillespie C.W. (1998): Improving the interpretability of TSK fuzzy models by combining global learning and local learning.-IEEE Trans. Fuzzy Syst., Vol. 6, No. 4, pp. 530-537.