Analysis of expedient behavior of various types of automata

D. P. Dimitrichenko

News of the Kabardin-Balkar scientific center of RAS, Tome 26 (2024) no. 6, pp. 165-174

Résumé

Collective behavior of automata is one of the directions of development of machine learning methods. Such machines fulfil the function of goal-oriented behavior. The machine performs an action, in response to which the environment sends its output signal to the input of the machine. The machine, in accordance with its design, responds to this input signal with the next action. Thus, a closed loop of interaction is built between a certain environment and the machine operating in it. This environment itself in many cases allows for machine implementation. Effectiveness evaluation of the machine is defined as an optimization problem of maximizing the sum of positive signals (rewards), or minimizing negative signals (penalties), received from the environment, over the considered period of time. Formalization of both the properties of the environment and the actions of the machines, as well as processing of the obtained results is performed using the apparatus of game theory. In this case, signals from the environment are conveniently represented as the sums of the winnings and losses of the players-machines. In this paper, a comparison of machines of different designs is carried out, since the efficiency of machine reactions is determined not only by the properties of the environment, but also by such parameters as the type and depth of memory.

Détail

Keywords: automaton, expedient behavior, optimal strategy, memory depth, game theory, formalization of the environment, dynamic environment

Bibliographie

[1] V. L. Stefanyuk, Local organization of intelligent systems, FIZMATLIT, Moscow, 2004, 328 pp. (In Russian)

[2] D. P. Dimitrichenko, “Optimization of a recurrent neural network using automata with variable structure”, Software systems and computational methods, 2023, no. 4, 30–43 (In Russian) | DOI

[3] D. A. Pospelov, Games, automata, Energiya, Moscow, 1966, 136 pp. (In Russian)

[4] M. L. Tsetlin, Studies in the theory of automata and modeling of biological systems, Moscow, 1969, 316 pp. (In Russian)

[5] D. A. Pospelov, Probabilistic automata, Energiya, Moscow, 1970, 88 pp. (In Russian)

[6] V. I. Varshavskii, Collective behavior of automata, Nauka, Moscow, 1973, 408 pp. (In Russian)

[7] V. I. Varshavskii, D. A. Pospelov, The orchestra plays without a conductor: reflections on the evolution of some technical systems and their control, Nauka, Moscow, 1984, 208 pp. (In Russian)