Control of Markov processes and $W$-spaces

N. V. Krylov

Izvestiya. Mathematics , Tome 5 (1971) no. 1, pp. 233-266.

Résumé

Problems in the control of continuous Markov processes on a semicompactum by two players with conflicting interests are studied. The basic content of the paper is a derivation of Bellman's equations in the case where control is exercised for an infinite time (Theorem 3), and in the case of a problem of optimal stopping (Theorem 6). The results are illustrated by two examples (Theorems 1 and 2).

Bibliographie

[1] Blackwell D., “Discrete dynamic programming”, Ann. Math. Statistics, 33:2 (1962), 719–726 | DOI | MR | Zbl

[2] Howard R. A., Dynamic programming and Markov processes, Wiley, New York, London, 1960 | MR | Zbl

[3] Shaply L. S., “Stochastic games”, Proc. Nat. Acad. Sci. USA, 39:10 (1953), 1095–1100 | DOI | MR

[4] Dubins L. E., Savage L. I., How to gamble if you must, 1963

[5] Dynkin E. B., “Optimalnyi vybor momenta ostanovki markovskogo protsessa”, Dokl. AN SSSR, 150:2 (1963), 238–240 | MR | Zbl

[6] Shiryaev A. N., Viskov O. V., “Ob upravleniyakh, privodyaschikh k optimalnym statsionarnym rezhimam”, Tr. Matem. in-ta im. V. A. Steklova AN SSSR, 71, 1964, 35–45 | MR | Zbl

[7] Krylov N. V., “Postroenie optimalnoi strategii dlya konechnoi upravlyaemoi tsepi”, Teoriya veroyatn. i ee primen., 10:1 (1965), 51–60 | MR | Zbl

[8] Girsanov I. V., “Minimaksnye zadachi v teorii diffuzionnykh protsessov”, Dokl. AN SSSR, 136:4 (1961), 761–764 | MR | Zbl

[9] Fleming W. H., “Some Markovian optimization problems”, J. Math. and Mech., 12:1 (1963), 131–140 | MR | Zbl

[10] Mandl P., “On optimal control of a non-stopped diffusion processe”, Z. Warscheinlichkeitstheorie und verw. Gebiete, 4:1 (1965), 1–9 | DOI | MR | Zbl

[11] Kogan Ya. A., “Ob optimalnom upravlenii sluchainoi zamenoi vremeni v nepreryvnom markovskom protsesse”, Teoriya veroyatn. i ee primen., 13:2 (1968), 362–365

[12] Grigelionis B. I., Shiryaev A. N., “O zadache Stefana i optimalnykh pravilakh ostanovki markovskikh protsessov”, Teoriya veroyatn. i ee primen., 11:4 (1966), 612–631 | MR | Zbl

[13] Danford N., Shvarts Dzh., Lineinye operatory, obschaya teoriya, IL, M., 1962

[14] Krylov N. V., “Ob odnom klasse nelineinykh uravnenii v prostranstve izmerimykh funktsii”, Matem. sb., 80(122) (1969), 253–265 | Zbl

[15] Krylov N. V., “O stokhasticheskom integrale Ito”, Teoriya veroyatn. i ee primen., 14:2 (1969), 340–348 | Zbl

[16] Sobolev S. L., Nekotorye primeneniya funktsionalnogo analiza v matematicheskoi fizike, Izd-vo Leningr. un-ta, L., 1950 | MR

[17] Dynkin E. B., Markovskie protsessy, Fizmatgiz, M., 1963 | MR

[18] Krylov N. V., “Ob uravneniyakh minimaksnogo tipa v teorii ellipticheskikh i parabolicheskikh uravnenii na ploskosti”, Matem. sb., 81(123) (1970), 3–22 | Zbl

[19] Gusein-zade S. M., “Ob odnoi igre, svyazannoi s vinerovskim protsessom”, Teoriya veroyatn. i ee primen., 14:4 (1969), 732–735 | MR | Zbl

[20] Krylov N. V., “Diffuziya na ploskosti s otrazheniem. Kraevaya zadacha”, Sib. matem. zh., 10:2 (1969), 355–372 | Zbl

[21] Krylov N. V., “O kvazidiffuzionnykh protsessakh”, Teoriya veroyatn. i ee primen., 11:3 (1966), 424–443 | MR | Zbl

[22] Krylov N. V., “O pervoi kraevoi zadache dlya ellipticheskikh uravnenii vtorogo poryadka”, Differentsialnye uravneniya, 3:2 (1967), 315–326 | Zbl

[23] Agmon S., “On the eigenfunctions and on the eigenvalues of general elliptic boundary value problems”, Cornm. Pure Appl. Math., 15:2 (1962), 119–147 | DOI | MR | Zbl