Knowledge representation and acquisition in expert systems for pattern recognition

O. M. Vasil'ev   ; D. P. Vetrov   ; D. A. Kropotov

Žurnal vyčislitelʹnoj matematiki i matematičeskoj fiziki, Tome 47 (2007) no. 8, pp. 1428-1454

Résumé

A new approach to the design of fuzzy expert systems is proposed. The representation of knowledge and the formation of statements by fuzzy logic tools are discussed in detail. A model of fuzzy inference is described. Primary attention is given to automatic extraction of knowledge (fuzzy inference rules) from a set of precedents. Various performance criteria for rules are introduced, and an algorithm for their generation (the method of effective restrictions) is proposed. An extension of the type of admissible rules by introducing a fuzzy disjunction operation is described. The possibility of optimizing the rules found is explored. The benefits of the approaches proposed are illustrated by experiments.

Bibliographie

[1] Zade L., “Razmytye mnozhestva i ikh primenenie v raspoznavanii obrazov i klaster-analize”, Klassifikatsiya i klaster, Mir, M., 1980, 208–247

[2] Zhuravlev Yu. I., Izbrannye nauchnye trudy, Magistr, M., 1998

[3] Zhuravlev Yu. I., Ryazanov V. V., Senko O. V., RASPOZNAVANIE. Matematicheskie metody. Programmnaya sistema. Prakticheskie primeneniya, Fazis, M., 2006

[4] Bishop C. M., Neural networks for pattern recognition, Univ. Press, Oxford, 1995 | MR

[5] Breinum L., Friedman J. H., Olshen R. A., Stone C. J., Classification and regression trees, Wadsworth Internal. Group, Belmont, CA, 1984

[6] Perfileva I., “Prilozheniya teorii nechetkikh mnozhestv”, Itogi nauki i tekhn. Ser. Teoriya veroyatnostei. Matem. statistika. Teoretich. kibernetika, 29, VINITI, M., 1990, 83–151 | MR

[7] Terano T., Asan K., Sugeno M., Prikladnye nechetkie sistemy, Mir, M., 1993 | MR | Zbl

[8] Mamdani E., “Advances in the linguistic synthesis of fuzzy controllers”, Proc. 6th Internat. Symp. Multiple-Values Logic, 1976, 196–202

[9] Zade L., Ponyatie lingvisticheskoi peremennoi i ego primenenie k prinyatiyu priblizhennykh reshenii, Mir, M., 1980

[10] Fitkami S., Mizumoto M., Tanaka K., “Some considerations of fuzzy conditional inference”, Fuzzy Sets and Systems, 3 (1980), 243–273 | MR

[11] Razanov V. V., Senko O. B., “O nekotorykh modelyakh golosovaniya i metodakh ikh optimizatsii”, Raspoznavanie, klassifikatsiya, prognoz, 3 (1990), 106–145

[12] Vorontsov K. V., Lektsii po logicheskim algoritmam klassifikatsii, 2006 http://www.ccas.ru/voron/download/LogicAlgs.pdf

[13] Paklin N. B., Adaptivnye modeli nechetkogo vyvoda dlya identifikatsii nelineinykh zavisimostei v slozhnykh sistemakh, Dis. $\dots$ kand. tekhn. nauk, IzhGGU, Izhevsk, 2004

[14] Ojala T., Neuro-fuzzy systems in control, Master of science thesis, Tampere, Finland, 1994

[15] Jang J.-S. R., Sun C.-T., Mizutani E., Neuro-fuzzy and soft computing, Prentice-Hall, Upper Saddle River, NJ, 1997

[16] Ishibuchi H., Nozaki K., Yamamoto N., Tanaka C., “Construction of fuzzy classification systems with rectangular fuzzy rules using genetic algorithms”, Fuzzy Sets and System., 65:2/3 (1994), 237–253 | DOI | MR

[17] Inoue H., Kamei K., Inoue K., “Rule pairing methods for crossover in GA for automatic generation of fuzzy control rules”, Fuzzy Systems Proceedings, The 1998 IEEE International Conference on Computational Intelligence, v. 2, 1998, 1223–1228 | DOI

[18] Gomez-Scarmeta A. F., Jimenez F., “Generating and tuning fuzzy rules using hybrid systems”, Proc. 6th IEEE Internat. Conf. Fuzzy System. V. 1, 1997, 247–252

[19] Rivest R. L., “Learning decision lists”, Mach. Learn., 2:3 (1987), 229–246

[20] Cohen W. W., “Fast effective rule induction”, Proc. 12th Internat. Conf. Mach. Learn., Morsan Kaufmann, CA, 1995, 151–163

[21] Freund Y., Schapire R. E., “Experiments with a new boosting algorithm”, Proc. 13th Internat. Conf. Mach. Learn., 1996, 148–156

[22] Cohen W. W., Singer Y., “A simple, fast, and effective rule learner”, Proc. 16th Nat. Conf. Artificial Intelligence, 1999

[23] Vetrov D. P., Kropotov D. A., “Ispolzovanie metodov Boosting dlya generatsii znanii”, Tr. XII Vseros. konf. “Matem. metody raspoznavaniya obrazov”, Maks-Press, M., 2005, 48–51

[24] Quinlan J. R., C4.5: Programs for machine learning, Morgan Kaufmann, San Mateo, CA, 1993

[25] Breslow L. A., Aha D. W., “Simplifying decision trees: a survey”, Knowledge Eng. Rev., 12:1 (1997), 1–40 http://siteseer.ist.psu.edu/breslow96simplifying.html | DOI | MR

[26] Dyulicheva Yu. Yu., Modeli korrektsii redutsirovannykh binarnykh reshayuschikh derevev, Dis. kand. fiz.-matem. nauk, TavRGU, Simferopol, 2004

[27] Duda R., Khart P., Raspoznavanie obrazov i analiz stsen, Mir, M., 1976

[28] Corduneanu A., Bishop C., “Variational model selection for mixture distributions”, Artificial Intelligence and Statistics, Morgan Kaufmann, 2001, 27–34

[29] Shumskii S. A., “Baiesova regulyarizatsiya obucheniya”, Lektsii po neiroinformatike, Ch. 2, MIFI, M., 2002, 30–93

[30] Bishop C. M., Svensen M., Robust Bayesian mixture modeling, Proc. ESANN, 2004

[31] Ben-Hur A., Elisseeff A., Guyon I., “A stability based method for discovering structure in clustered data”, Proc. Symposium on Biocomputing, Lihue, Hawaii, 2002, 6–17

[32] Kuncheva L., Combining pattern classifiers: methods and algorithms, Wiley, 2004 | MR | Zbl

[33] Gurov S. I., Otsenka nadezhnosti klassifitsiruyuschikh algoritmov, Izdat. otd. VMiK MGU, M., 2002

[34] MacKay D. J. C., Information theory, inference, and learning algorithms, Cambridge Univ. Press, 2003 | MR | Zbl

[35] Kropotov D. A., Tolstov I. V., Vetrov D. P., “Decision trees regularization based on stability principle”, Pattern Recognition and Image Analysis, 15:1 (2005), 107–109

[36] Berger J. O., Statistical decision theory and bayesian analysis, Springer, Berlin etc, 1985 | MR

[37] MacKay D. J. C., “Bayesian interpolation”, Neural Computation, 4:3 (1992), 415–447 | DOI

[38] Tipping M. E., “Sparse bayesian learning and the relevance vector machine”, J. Mach. Learn. Res., 1 (2001), 211–244 | DOI | MR | Zbl

[39] Barges C., “A tutorial on support vector machines for pattern recognition”, Data Mining and Knowledge Discovery, 2 (1998), 121–167 | DOI

[40] Vapnik V. N., Statistical learning theory, Wiley, 1998 | MR | Zbl

[41] Williams C. K. I., “Prediction with gaussian processes: from linear regression to linear prediction and beyond”, Learning in Graphical Models, MIT, 1999, 599–621

[42] Williams P. M., “Bayesian regularization and pruning using a laplace prior”, Neural Computation, 7:1 (1995), 117–143 | DOI

[43] Chu W., Bayesian approach to support vector machines, PhD thesis, Nat. Univ. of Singapore, 2003

[44] Dyakonov V., Kruglov V., Matematicheskie pakety rasshireniya MATLAB. Spetsialnyi spravochnik, Piter, M., 2001

[45] Murphy P., Aha D., UCI repository of machine learning databases, Univ. California, Dept. Informat. and Comput. Sci., California: Irvine, 1996 http://www.ics.uci.edu/~mlearn/MLRepository.html | Zbl

[46] Kropotov D. A., Ptashko N. O., Vetrov D. P., “The use of bayesian framework for kernel selection in vector machines classifiers”, Progress in Pattern Recognition, Image Analysis and Applic., LNCS, 3773, Springer, Berlin etc, 2005, 252–261

[47] Kropotov D. A., Vetrov D. P., Ptashko N. O., Vasiliev O. M., “The use of stability principle for kernel determination in relevance vector machines”, ICONIP2006, Part I, LNCS, 4232, Springer, Berlin etc, 2006, 727–736