Neuro-fuzzy modelling based on a deterministic annealing approach

Czabański, R.

International Journal of Applied Mathematics and Computer Science, Tome 15 (2005) no. 4, pp. 561-576.

Résumé

This paper introduces a new learning algorithm for artificial neural networks, based on a fuzzy inference system ANBLIR. It is a computationally effective neuro-fuzzy system with parametrized fuzzy sets in the consequent parts of fuzzy if-then rules, which uses a conjunctive as well as a logical interpretation of those rules. In the original approach, the estimation of unknown system parameters was made by means of a combination of both gradient and least-squares methods. The novelty of the learning algorithm consists in the application of a deterministic annealing optimization method. It leads to an improvement in the neuro-fuzzy modelling performance. To show the validity of the introduced method, two examples of application concerning chaotic time series prediction and system identification problems are provided.

Détail

Keywords: fuzzy systems, neural networks, neuro-fuzzy systems, rules extraction, deterministic annealing, prediction
Mots-clés : system rozmyty, sieć neuronowa, ekstrakcja reguł

Bibliographie

[1] Bezdek J.C. (1982): Pattern Recognition with Fuzzy Objective Function Algorithms.— New York: Plenum Press.

[2] Box G.E.P. and Jenkins G.M. (1976): Time Series Analysis. Forecasting and Control. —San Francisco: Holden–Day.

[3] Chen J.Q., Xi Y.G. and Zhang Z.J. (1998): A clustering algorithm for fuzzy model identification. — Fuzzy Sets Syst., Vol. 98, No. 3, pp. 319–329.

[4] Cho K.B. and Wang B.H. (1996): Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction. — Fuzzy Sets Syst., Vol. 83, No. 3, pp. 325–339.

[5] Chung F.L. and Duan J.C. (2000): On multistage fuzzy neural network modeling. — IEEE Trans. Fuzzy Syst., Vol. 8, No. 2, pp. 125–142.

[6] Czogała E. and Łęski J. (1996): A new fuzzy inference system with moving consequents in if-then rules. Application to pattern recognition. — Bull. Polish Acad. Sci., Vol. 45, No. 4, pp. 643–655.

[7] Czogała E. and Łęski J. (1999): Fuzzy and Neuro-Fuzzy Intelligent Systems. —Heidelberg: Physica-Verlag.

[8] Czogała E. and Łęski J. (2001): On equivalence of approximate reasoning results using different interpretations of if-then rules. — Fuzzy Sets Syst., Vol. 117, No. 2, pp. 279–296.

[9] German S. and German D. (1984): Stochastic relaxation, Gibbs distribution and the Bayesian restoration in images. — IEEE Trans. Pattern Anal. Mach. Intell., Vol. 6, No. 9, pp. 721–741.

[10] Jang J.S.R. (1993): ANFIS: Adaptive-Network-Based Fuzzy Inference System. — IEEE Trans. Syst. Man Cybern., Vol. 23, No. 3, pp. 665–685.

[11] Jang J.S.R. and Sun C.T. (1993): Functional equivalence between radial basis function networks and fuzzy inference systems. — IEEE Trans. Neural Netw., Vol. 4, No. 1, pp. 156–159.

[12] Jang J.S.R. and Sun C.T. (1995): Neuro-fuzzy modeling and control.— Proc. IEEE, Vol. 83, No. 3, pp. 378–406.

[13] Jang J.S.R., Sun C.T. and Mizutani E. (1997): Neuro-Fuzzy and Soft Computing. A Computational Approach to Learning and Machine Intelligence. — Upper Saddle River: Prentice-Hall.

[14] Juang C. and Lin C. (1998): An on-line self-constructing neural fuzzy inference network and its applications. — IEEE Trans. Fuzzy Syst., Vol. 6, No. 1, pp. 12–32.

[15] Kim E., Park M. and Ji S. (1997): A new approach to fuzzy modeling. — IEEE Trans. Fuzzy Syst., Vol. 5, No. 3, pp. 328–337.

[16] Kirkpatrick S., Gelatt C. and Vecchi M. (1983): Optimization by simulated annealing. — Science, Vol. 220, No. 4598, pp. 671–680.

[17] Kosko B. (1987): Fuzzy associative memories In: Fuzzy Expert Systems (A. Kandel, Ed.).—Boca Raton: CRC Press.

[18] Łęski J. (2003): ε –insensitive learning techniques for approximate reasoning systems.—Int. J. Comput. Cognit., Vol. 1, No. 1, pp. 21–77.

[19] Lin Y. and Cunningham G.A. (1995): A new approach to fuzzy-neural modeling. – IEEE Trans. Fuzzy Syst., Vol. 3, No. 2, pp. 190–197.

[20] Mamdani E.H. (1974): Applications of fuzzy algorithms for control of simple dynamic plant. — Proc. IEEE, Vol. 121, No. 12, pp. 1585–1588.

[21] Mamdani E.H. (1976): Advances in the linguistic synthesis of fuzzy controller.—Int. J. Man–Mach. Stud., Vol. 8, No. 6, pp. 669–678.

[22] Mamdani E.H. (1977): Applications of fuzzy logic to approximate reasoning using linguistic synthesis. — IEEE Trans. Comput., Vol. 26, No. 12, pp. 1182–1191.

[23] Mamdani E.H. and Assilian S. (1975): An experiment in linguistic synthesis with a fuzzy logic controller. — Int. J. Man-Mach. Stud., Vol. 7, No. 1, pp. 1–13.

[24] Metropolis N., Rosenbluth A.W., Rosenbluth M.N., Teller A.H. and Teller E. (1953): Equation of state calculation by fast computing machines. — J. Chem. Phys., Vol. 21, No. 6, pp. 1087–1092.

[25] Mitra S. and Pal S.K. (1995): Fuzzy multi-layer perceptron, inferencing and rule generation. — IEEE Trans. Neural Netw., Vol. 6, No. 1, pp. 51–63.

[26] Pedrycz W. (1984a): An identification algorithm in fuzzy relational systems. — Fuzzy Sets Syst., Vol. 13, No. 2, pp. 153–167.

[27] Pedrycz W. (1984b): Identification in fuzzy systems. — IEEE Trans. Syst. Man Cybern., Vol. 14, No. 2, pp. 361–366.

[28] Rao A.V. and Rose K. (1999): A deterministic annealing approach for parsimonious design of piecewise regression models. — IEEE Trans. Pattern Anal. Mach. Intell., Vol. 21, No. 2, pp. 159–173.

[29] Rao A.V., Miller D., Rose K. and Gersho A. (1997): Mixture of experts regression modeling by deterministic annealing. —IEEE Trans. Signal Process., Vol. 45, No. 11, pp. 2811–2820.

[30] Rose K. (1991): Deterministic Annealing, Clustering and Optimization. — Ph.D. Thesis, California Inst. Technol, Pasadena.

[31] Rose K. (1998): Deterministic annealing for clustering, compression, classification, regression and related optimization problems. — Proc. IEEE, Vol. 86, No. 11, pp. 2210–2239.

[32] Rose K. (1999): A deterministic annealing approach for parsimonious design of piecewise regression models. — IEEE Trans. Pattern Anal. Mach. Intell., Vol. 21, No. 2, pp. 159–173.

[33] Rutkowska D. (2001): Neuro–Fuzzy Architectures and Hybrid Learning. — Heidelberg: Physica–Verlag.

[34] Schuster H.G. (1984): Deterministic Chaos.—Weinheim: VCH Verlagsgesellschaft.

[35] Sugeno M. and Kang G.T. (1988): Structure identification of fuzzy model.—Fuzzy Sets Syst., Vol. 28, No. 1, pp. 15–33.

[36] Sugeno M. and Yasukawa T. (1993): A fuzzy-logic based approach to qualitative modeling. — IEEE Trans. Fuzzy Syst., Vol. 1, No. 1, pp. 7–31.

[37] Tong R.M. (1980): The evaluation of fuzzy models derived from experimental data. — Fuzzy Sets Syst., Vol. 4, No. 13, pp. 1–12.

[38] Wang L. and Langari R. (1995): Building Sugeno-type models using fuzzy discretization and orthogonal parameter estimation techniques. — IEEE Trans. Fuzzy Syst., Vol. 3, No. 4, pp. 454–458.

[39] Xie X.L. and Beni G. (1991): A validity measure for fuzzy clustering.— IEEE Trans. Pattern Anal. Mach. Intell., Vol. 13, No. 8, pp. 841–847.

[40] Xu C.W. and Lu Y.Z. (1987): Fuzzy model identification and self-learning for dynamic systems. — IEEE Trans. Syst. Man Cybern., Vol. 17, No. 4, pp. 683–689.

[41] Yager R.R. and Filev D.P. (1984): Essentials of Fuzzy Modeling and Control.— New York: Wiley.

[42] Yen J.,Wang L. and Gillespie C.W. (1998): Improving the interpretability of TSK fuzzy models by combining global learning and local learning.—IEEE Trans. Fuzzy Syst., Vol. 6, No. 4, pp. 530–537.

[43] Zadeh L.A. (1965): Fuzzy sets. — Inf. Contr., Vol. 8, No. 3, pp. 338–353.

[44] Zadeh L.A. (1971): Towards a theory of fuzzy systems, In: Aspects of Network and System Theory (R.E. Kalman and N. DeClaris, Ed.).—New York: Holt, Rinehart Winston.

[45] Zadeh L.A. (1973): Outline of a new approach to the analysis of complex systems and decision processes. — IEEE Trans. Syst. Man Cybern., Vol. 3, No. 1, pp. 28–44.

[46] Zikidis K.C. and Vasilakos A.V. (1996): ASAFES2: A novel, neuro-fuzzy architecture for fuzzy computing, based on functional reasoning. — Fuzzy Sets Syst., Vol. 83, No. 1, pp. 63–68.