Determining the distribution of the duration of stationary games for zero-order markov processes with final sequence of states

Alexandru Lazari

Buletinul Academiei de Ştiinţe a Republicii Moldova. Matematica, no. 3 (2015), pp. 72-78

Résumé

A zero-order Markov process with final sequence of states represents a stochastic system with independent transitions that stops its evolution as soon as given final sequence of states is reached. The transition time of the system is unitary and the transition probability depends only on the destination state. We consider the following game. Initially, each player defines his distribution of the states. The initial distribution of the states is established according to the distribution given by the first player. After that, the stochastic system passes consecutively to the next state according to the distribution given by the next player. After the last player, the first player acts on the system evolution and the game continues in this way until the given final sequence of states is achieved. Our goal is to study the duration of this game, knowing the distribution established by each player and the final sequence of states of the stochastic system. It is proved that the distribution of the duration of the game is a homogeneous linear recurrent sequence and it is developed a polynomial algorithm to determine the initial state and the generating vector of this recurrence. Using the generating function, the main probabilistic characteristics are determined.