Geodesic
Determining the optimal evolution time for Markov processes with final sequence of states
Buletinul Academiei de Ştiinţe a Republicii Moldova. Matematica, no. 1 (2015), pp. 115-126 
Alexandru Lazari. Determining the optimal evolution time for Markov processes with final sequence of states. Buletinul Academiei de Ştiinţe a Republicii Moldova. Matematica, no. 1 (2015), pp. 115-126. http://geodesic.mathdoc.fr/item/BASM_2015_1_a7/
@article{BASM_2015_1_a7,
     author = {Alexandru Lazari},
     title = {Determining the optimal evolution time for {Markov} processes with final sequence of states},
     journal = {Buletinul Academiei de \c{S}tiin\c{t}e a Republicii Moldova. Matematica},
     pages = {115--126},
     year = {2015},
     number = {1},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/BASM_2015_1_a7/}
}
TY  - JOUR
AU  - Alexandru Lazari
TI  - Determining the optimal evolution time for Markov processes with final sequence of states
JO  - Buletinul Academiei de Ştiinţe a Republicii Moldova. Matematica
PY  - 2015
SP  - 115
EP  - 126
IS  - 1
UR  - http://geodesic.mathdoc.fr/item/BASM_2015_1_a7/
LA  - en
ID  - BASM_2015_1_a7
ER  - 
%0 Journal Article
%A Alexandru Lazari
%T Determining the optimal evolution time for Markov processes with final sequence of states
%J Buletinul Academiei de Ştiinţe a Republicii Moldova. Matematica
%D 2015
%P 115-126
%N 1
%U http://geodesic.mathdoc.fr/item/BASM_2015_1_a7/
%G en
%F BASM_2015_1_a7

Voir la notice de l'article provenant de la source Math-Net.Ru

This paper describes a class of dynamical stochastic systems that represents an extension of classical Markov decision processes. The Markov stochastic systems with given final sequence of states and unitary transition time, over a finite or infinite state space, are studied. Such dynamical system stops its evolution as soon as given sequence of states in given order is reached. The evolution time of the stochastic system with fixed final sequence of states depends on initial distribution of the states and probability transition matrix. The considered class of processes represents a generalization of zero-order Markov processes, studied in [3]. We are seeking for the optimal initial distribution and optimal probability transition matrix that provide the minimal evolution time for the dynamical system. We show that this problem can be solved using the signomial and geometric programming approaches.

[1] Lazari A., Lozovanu D., Capcelea M., Dynamical deterministic and stochastic systems: Evolution, optimization and discrete optimal control, CEP USM, Chişinău, 2015, 310 pp. (in Romanian)

[2] Lazari A., “Evolution Time of Composed Stochastic Systems with Final Sequence of States and Independent Transitions”, Annals of the Alexandru Ioan Cuza University – Mathematics, 2014, December, 18 pp., ISSN (Online) 1221-8421 | DOI

[3] Lazari A., “Optimization of Zero-Order Markov Processes with Final Sequence of States”, Universal Journal of Applied Mathematics (Horizon Research Publishing), 1:3 (2013), 198–206

[4] Lazari A., “Compositions of stochastic systems with final sequence states and interdependent transitions”, Annals of the University of Bucharest, 4(LXII):1 (2013), 289–303 | MR | Zbl

[5] Lazari A., Numerical methods for solving deterministic and stochastic problems in dynamic decision systems, PhD thesis in mathematics and physics, Chişinău, 2011 (in Romanian)

[6] Boyd S., Kim S., Vandenberghe L., Hassibi A., “A tutorial on geometric programming”, Optim. Eng., Educational Section, 8:1 (2007), 67–127 | MR

[7] Boyd S., Vandenberghe L., Convex optimization, Cambridge University Press, Cambridge, 2004 | MR | Zbl

[8] Zbaganu G., “Waiting for an ape to type a poem”, Bul. Acad. Ştiinţe Repub. Moldova, Mat., 1992, no. 2(8), 66–74 | MR | Zbl

[9] Guibas Leo J., Odlyzko Andrew M., “Periods in Strings”, Journal of Combinatorial Theory, Series A, 30 (1981), 19–42 | MR | Zbl