Models of adaptive behavior and problem of origin of intelligence

V. G. Red'ko

Matematičeskaâ biologiâ i bioinformatika, Tome 2 (2007) no. 1, pp. 160-180.

Résumé

An approach to the problem of evolutionary origin of intelligence is discussed; the approach is based on modeling of evolution of adaptive behavior. Works of leading laboratories in the field of simulation of adaptive behavior are characterized. A special attention is paid to reinforcement learning and adaptive critic designs. The project “Animat Brain” directed to development of a general platform for systematic designing of models of adaptive behavior is described. The results of investigation of the concrete model of evolution of self-learning agents that are based on adaptive critic designs are represented. The sketch program for future research of evolution of adaptive behavior is proposed.

Bibliographie

[1] Eigen M., Samoorganizatsiya materii i evolyutsiya biologicheskikh makromolekul, Mir, M., 1973, 216 pp.

[2] Eigen M., Shuster P., Gipertsikl. Printsipy samoorganizatsii makromolekul, Mir, M., 1982, 270 pp.

[3] Eigen M., Gardiner W., Schuster P., Winkler-Oswatich R., “The origin of genetic information”, Scientific American, 244:4 (1981), 88–118 | DOI

[4] Ratner V. A., Shamin V. V., “Saizery: modelirovanie fundamentalnykh osobennostei molekulyarno-biologicheskoi organizatsii. Sootvetstvie obschikh svoistv i konstruktivnykh osobennostei kollektivov makromolekul”, Zhurn. obsch. biologii, 44:1 (1983), 51–61

[5] Redko V. G., Evolyutsionnaya kibernetika, Nauka, M., 2001, 156 pp.

[6] Redko V. G., Evolyutsiya, neironnye seti, intellekt: Modeli i kontseptsii evolyutsionnoi kibernetiki. Seriya “Sinergetika: ot proshlogo k buduschemu”, URSS, M., 2005, 224 pp.

[7] Voronin L. G., Evolyutsiya vysshei nervnoi deyatelnosti, Nauka, M., 1977, 128 pp. | Zbl

[8] Meyer J.-A., Wilson S. W. (eds.), Proceedings of the First International Conference on Simulation of Adaptive Behavior, The MIT Press, Cambridge, Massachusetts, London, England, 1990

[9] Meyer J.-A., Guillot A., “From SAB90 to SAB94: Four years of Animat research”, Proceedings of the Third International Conference on Simulation of Adaptive Behavior, eds. Cliff, Husbands, Meyer J.-A., Wilson S. W., The MIT Press, Cambridge, 1994 http://animatlab.lip6.fr/index.en.html

[10] Guillot A., Meyer J.-A., “From SAB94 to SAB2000: What's new, Animat?”, From Animals to Animats 6, Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior, eds. Meyer et al., The MIT Press, 2000 http://animatlab.lip6.fr/index.en.html

[11] Nepomnyaschikh V. A., “Animaty kak model povedeniya zhivotnykh”, IV Vserossiiskaya nauchno-tekhnicheskaya konferentsiya “Neiroinformatika-2002”, Materialy diskussii “Problemy intellektualnogo upravleniya – obschesistemnye, evolyutsionnye i neirosetevye aspekty”, MIFI, M., 2003, 58–76

[12] Nepomnyaschikh V. A., “Poisk obschikh printsipov adaptivnogo povedeniya zhivykh organizmov i animatov”, Novosti iskusstvennogo intellekta, 2 (2002), 48–53

[13] Donnart J. Y., Meyer J. A., “Learning reactive and planning rules in a motivationally autonomous animat”, IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics, 26:3 (1996), 381–395 http://animatlab.lip6.fr/index.en.html | DOI

[14] Wilson S. W., “The animat path to AI”, From animals to animats, Proceedings of the First International Conference on Simulation of Adaptive Behavior, eds. Meyer J.-A., Wilson S. W., The MIT Press, Cambridge, Massachusetts, London, England, 1990, 15–21

[15] Tsetlin M. L., Issledovaniya po teorii avtomatov i modelirovanie biologicheskikh sistem, Nauka, M., 1969, 316 pp. | MR | Zbl

[16] Varshavskii V. I., Pospelov D. A., Orkestr igraet bez dirizhera, Nauka, M., 1984 | MR

[17] Bongard M. M., Losev I. S., Smirnov M. S., “Proekt modeli organizatsii povedeniya – “Zhivotnoe””, Modelirovanie obucheniya i povedeniya, Nauka, M., 1975, 152–171

[18] Gaaze-Rapoport M. G., Pospelov D. A., Ot ameby do robota: modeli povedeniya, Nauka, M., 1987 | MR

[19] Holland J. H., Adaptation in Natural and Artificial Systems, MIT Press, Boston, MA, 1992 | MR

[20] Kureichik V. M., Geneticheskie algoritmy i ikh primenenie, TRTU, Taganrog, 2002

[21] Emelyanov V. V., Kureichik V. M., Kureichik V. V., Teoriya i praktika evolyutsionnogo modelirovaniya, Fizmatlit, M., 2003

[22] Holland J. H., Holyoak K. J., Nisbett R. E., Thagard P., Induction: Processes of Inference, Learning, and Discovery, MIT Press, Cambridge, MA, 1986

[23] Sutton R., Barto A., Reinforcement Learning: An Introduction, MIT Press, Cambridge, 1998 http://www.cs.ualberta.ca/~sutton/book/the-book.html

[24] Nolfi S., Baldassarre G., Calabretta R., Hallam J., Marocco D., Miglino O., Meyer J.-A., Parisi D. (eds.), Proceedings of the Ninth International Conference on Simulation of Adaptive Behaviour, Springer Verlag, Berlin, Germany, 2006, 4095 pp.

[25] Sait AnimatLab: http://animatlab.lip6.fr/index.en.html

[26] Sait AI Laboratory of Zurich University: http://www.ifi.unizh.ch/groups/ailab/

[27] Pfeifer R., Scheier C., Understanding Intelligence, MIT Press, 1999

[28] Sait Laboratory of Artificial Life and Robotics: http://gral.ip.rm.cnr.it/

[29] Nolfi S., Floreano D., Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines, MIT Press/Bradford Books, Cambridge, MA, 2000, 384 pp.

[30] Sait MIT Computer Science and Artificial Intelligence Laboratory: http://www.csail.mit.edu/index.php

[31] Brooks R. A., Cambrian Intelligence: The Early History of the New AI, MIT Press, 1999 | Zbl

[32] Sait Neuroscience Institute: http://www.nsi.edu/

[33] Krichmar J. L., Edelman G. M., “Machine psychology: autonomous behavior, perceptual categorization and conditioning in a brain-based device”, Cerebral Cortex, 12 (2002), 818–830 | DOI

[34] Krichmar J. L., Edelman G. M., “Brain-based devices: intelligent systems based on principles of the nervous system”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, 2003, 940–945

[35] Krichmar J. L., Seth A. K., Nitz D. A., Fleischer J. G., Edelman G. M., “Spatial navigation and causal analysis in a brain-based device modeling cortical-hippocampal interactions”, Neuroinformatics, 3:3 (2005), 197–221 ; http://www.nsi.edu/nomad/pubs.htmlhttp://www.nsi.edu/nomad/krichmar_neuroinf_2005.pdf | DOI | Zbl

[36] Nepomnyaschikh V. A., “Kak zhivotnye reshayut plokho formalizuemye zadachi poiska”, Sinergetika i psikhologiya: Teksty. Vypusk 3: Kognitivnye protsessy, eds. Arshinov V. I., Trofimova I. N., Shendyapin V. M., Kogito-Tsentr, M., 2004, 197–209

[37] Nepomnyashchikh V. A., Podgornyj K. A., “Emergence of adaptive searching rules from the dynamics of a simple nonlinear system”, Adaptive Behavior., 11:4 (2003), 245–265 | DOI | MR

[38] Zhdanov A. A., “Metod avtonomnogo adaptivnogo upravleniya”, Izv. RAN. Seriya Teoriya i sistemy upravleniya, 1999, no. 5, 127–134 | MR | Zbl

[39] Zhdanov A. A., “O metode avtonomnogo adaptivnogo upravleniya”, VI Vserossiiskaya nauchno-tekhnicheskaya konferentsiya “Neiroinformatika-2004”, Lektsii po neiroinformatike. Chast 2, MIFI, M., 2004, 15–56

[40] Stankevich L. A., “Neirologicheskie sredstva sistem upravleniya intellektualnykh robotov”, VI Vserossiiskaya nauchno-tekhnicheskaya konferentsiya “Neiroinformatika-2004”, Lektsii po neiroinformatike. Chast 2, MIFI, M., 2004, 57–110

[41] Samarin A. I., “Model adaptivnogo povedeniya mobilnogo robota, realizovannaya s ispolzovaniem idei samoorganizatsii neironnykh struktur”, IV Vserossiiskaya nauchno-tekhnicheskaya konferentsiya “Neiroinformatika-2002”, Materialy diskussii “Problemy intellektualnogo upravleniya – obschesistemnye, evolyutsionnye i neirosetevye aspekty”, MIFI, M., 2003, 106–120

[42] Burtsev M. S., Gusarev R. V., Redko V. G., “Issledovanie mekhanizmov tselenapravlennogo adaptivnogo upravleniya”, Izv. RAN. Seriya Teoriya i sistemy upravleniya, 2002, no. 6, 55–62 | MR

[43] Burtsev M. S., Model evolyutsionnogo vozniknoveniya tselenapravlennogo adaptivnogo povedeniya. 2. Issledovanie razvitiya ierarkhii tselei, Preprint IPM RAN, 2002, 69 pp.

[44] Burtsev M. S., Turchin P. V., “Evolution of cooperative strategies from first principles”, Nature, 440:7087 (2006), 1041–1044 | DOI

[45] Mosalov O. P., Redko V. G., Nepomnyaschikh V. A., Model poiskovogo povedeniya animata, Preprint IPM RAN, 2003, 19 pp.

[46] Red'ko V. G., Mosalov O. P., Prokhorov D. V., “A Model of Evolution and Learning”, Neural Networks, 18:5–6 (2005), 738–745 | DOI

[47] Redko V. G. (red.), Ot modelei povedeniya k iskusstvennomu intellektu. Seriya “Nauki ob iskusstvennom”, URSS, M., 2006

[48] Jennie Si, Andrew Barto, Warren Powell, and Donald Wunsch (eds.), Learning and Approximate Dynamic Programming: Scaling Up to the Real World, IEEE Press and John Wiley Sons, 2004

[49] Rumelhart D. E., Hinton G. E., Williams R. G., “Learning representation by back-propagating error”, Nature, 323:6088 (1986), 533–536 | DOI

[50] Redko V. G., Prokhorov D. V., “Neirosetevye adaptivnye kritiki”, VI Vserossiiskaya nauchno-tekhnicheskaya konferentsiya ` “Neiroinformatika-2004”, Sbornik nauchnykh trudov. Chast 2, MIFI, M., 2004, 77–84

[51] Prokhorov D. V., Wunsch D. C., “Adaptive critic designs”, IEEE Trans. Neural Networks, 8:5 (1997), 997–1007 | DOI

[52] Prokhorov D. V., “Backpropagation through time and derivative adaptive critics: a common framework for comparison”, Learning and Approximate Dynamic Programming: Scaling Up to the Real World, eds. Jennie Si, Andrew Barto, Warren Powell, and Donald Wunsch, IEEE Press and John Wiley Sons, 2004 http://mywebpages.comcast.net/dvp/

[53] Anokhin K. V., Burtsev M. S., Zaraiskaya I. Yu., Lukashev A. O., Redko V. G., “Proekt “Mozg animata”: razrabotka modeli adaptivnogo povedeniya na osnove teorii funktsionalnykh sistem”, Vosmaya natsionalnaya konferentsiya po iskusstvennomu intellektu s mezhdunarodnym uchastiem, Trudy konferentsii, v. 2, Fizmatlit, M., 2002, 781–789

[54] Red'ko V. G., Prokhorov D. V., Burtsev M. S., “Theory of functional systems, adaptive critics and neural networks”, International Joint Conference on Neural Networks, Budapest, 2004, 1787–1792

[55] Anokhin P. K., Sistemnye mekhanizmy vysshei nervnoi deyatelnosti, Nauka, M., 1979, 453 pp.

[56] Red'ko V. G., Mosalov O. P., Prokhorov D. V., “A model of Baldwin effect in populations of self-learning agents”, International Joint Conference on Neural Networks, Montreal, 2005 | Zbl

[57] Prokhorov D., Puskorius G., Feldkamp L., “Dynamical neural networks for control”, A field guide to dynamical recurrent networks, eds. J. Kolen and S. Kremer, IEEE Press, NY, 2001, 257–289

[58] Moody J., Wu L., Liao Y., Saffel M., “Performance function and reinforcement learning for trading systems and portfolios”, Journal of Forecasting, 17 (1998), 441–470 | 3.0.CO;2-%23 class='badge bg-secondary rounded-pill ref-badge extid-badge'>DOI

[59] Baldwin J. M., “A new factor in evolution”, American Naturalist, 30 (1896), 441–451 | DOI

[60] Turney P., Whitley D., Anderson R. (eds.), “Evolution, Learning, and Instinct: 100 Years of the Baldwin Effect”, Special Issue of Evolutionary Computation on the Baldwin Effect, 4:3 (1996)

[61] Red'ko V. G., Anokhin K. V., Burtsev M. S., Manolov A. I., Mosalov O. P., Nepomnyashchikh V. A., Prokhorov D. V., The Ninth International Conference on the Simulation of Adaptive Behavior (SAB'06), 25–29 September 2006, CNR, Roma, Italy, Third Workshop on Anticipatory Behavior in Adaptive Learning Systems (ABiALS 2006), Proceedings

[62] Hebb D. O., The Organization of Behavior. A Neuropsychological Theory, Wiley Sons, N.Y., 1949, 355 pp.