On Derived and Nonstationary Markov Chains

J. W. Cohen

Teoriâ veroâtnostej i ee primeneniâ, Tome 7 (1962) no. 4, pp. 410-432

Résumé

Given a stationary Markov chain ${}_1M$, with a countable set of states $\mathscr{E}$, two new, nonstationary Markov chains ${}_2M$ and ${}_3M$ are formed from $\mathscr{E}$, according to the following rules (henceforth, ${}_i P_h,i=1,2,3$, denotes the transition-probability matrix at the $m$-th step of the chain ${}_i M$, and ${}_1P_h = {}_1 P$): $${}_2 P_h=\sum\limits_{n=0}^\infty {a_{nh_1}P^n},$$ where $$0\leq a_{nh}\leq 1,\quad\sum\limits_{n=1}^\infty{a_{nh}=1},\quad\mathop{\sup}\limits_h a_{0h}1$$ and ${}_2M$ is called the derived chain, while $${}_3P_h={}_2P_h+R_h,$$ where $$\sum\limits_{h=1}^\infty{\left\|{R_h} \right\|}\infty$$ and ${}_3M$ is called the perturbated chain. We study the problem of how various characteristics of the same state (e.g., return properties, periodicity, ergodicity), as well as certain other qualitative and quantitative indices of the chains, are interrelated in the chains ${}_1M$, ${}_2M$ and ${}_3M$. The results obtained can be generalized to the case of Markov chains with a continuous set of states, and similar constructions can be carried out for the case off continuous time.