Geodesic
Algorithms for determining the state-time probabilities and the limit matrix in Markov chains
Buletinul Academiei de Ştiinţe a Republicii Moldova. Matematica, no. 1 (2011), pp. 66-82 
Dmitrii Lozovanu; Stefan Pickl. Algorithms for determining the state-time probabilities and the limit matrix in Markov chains. Buletinul Academiei de Ştiinţe a Republicii Moldova. Matematica, no. 1 (2011), pp. 66-82. http://geodesic.mathdoc.fr/item/BASM_2011_1_a6/
@article{BASM_2011_1_a6,
     author = {Dmitrii Lozovanu and Stefan Pickl},
     title = {Algorithms for determining the state-time probabilities and the limit matrix in {Markov} chains},
     journal = {Buletinul Academiei de \c{S}tiin\c{t}e a Republicii Moldova. Matematica},
     pages = {66--82},
     year = {2011},
     number = {1},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/BASM_2011_1_a6/}
}
TY  - JOUR
AU  - Dmitrii Lozovanu
AU  - Stefan Pickl
TI  - Algorithms for determining the state-time probabilities and the limit matrix in Markov chains
JO  - Buletinul Academiei de Ştiinţe a Republicii Moldova. Matematica
PY  - 2011
SP  - 66
EP  - 82
IS  - 1
UR  - http://geodesic.mathdoc.fr/item/BASM_2011_1_a6/
LA  - en
ID  - BASM_2011_1_a6
ER  - 
%0 Journal Article
%A Dmitrii Lozovanu
%A Stefan Pickl
%T Algorithms for determining the state-time probabilities and the limit matrix in Markov chains
%J Buletinul Academiei de Ştiinţe a Republicii Moldova. Matematica
%D 2011
%P 66-82
%N 1
%U http://geodesic.mathdoc.fr/item/BASM_2011_1_a6/
%G en
%F BASM_2011_1_a6

Voir la notice de l'article provenant de la source Math-Net.Ru

New calculation procedures for finding the probabilities of state transitions of the system in Markov chains based on dynamic programming are developed and polynomial time algorithms for determining the limit state matrix in such processes are proposed. Computational complexity aspects and possible applications of the proposed algorithms for the stochastic optimization problems are characterized.

[1] Bollobas B., Random Graphs, Second Edition, Cambridge University Press, 2001 | MR | Zbl

[2] Bellman R., Kalaba R., Dynamic programming and modern control theory, Academic Press, New York–London, 1965 | MR

[3] Howard R. A., Dynamic Programming and Markov Processes, Wiley, 1960 | MR | Zbl

[4] Karlin S., Howard E. M., A First Course in Stochastic Processes, Academic Press, New York, 1975 | MR | Zbl

[5] Kemeny J. G, Snell J. L., Knapp A. W., Denumarable Markov Chains, Springer Verlag, New York, 1976 | MR | Zbl

[6] Lozovanu D., Pickl. S., Optimization and Multiobjective Control of Time-Discrete Systems, Springer Verlag, 2009 | MR | Zbl

[7] Lozovanu D., Pickl. S., “Dynamic Programming Algorithms for Solving Stochastic Discrete Control Problems”, Bulletin of the Academy of Sciences of Moldova. Mathematics, 2009, no. 2(60), 73–90 | MR | Zbl

[8] Lozovanu D., Pickl S., “Algorithmic Solutions of Discrete Control Problems on Stochastic Networks”, Procceedings of CTW09 Workshop on Graphs and Combinatorial Optimization, Paris, 2009, 221–224 | MR

[9] Puterman M., Markov Decision Processes: Stochastic Dynamic Programming, John Wiley, New Jersey, 2005 | Zbl

[10] Stewart W. J., Introduction to the Numerical Solution of Markov Chains, Princeton University Press, 1995 | MR