Geodesic
Fatou's lemma in its classical form and Lebesgue's convergence theorems for varying measures with applications to Markov decision processes
Teoriâ veroâtnostej i ee primeneniâ, Tome 65 (2020) no. 2, pp. 338-367 Cet article a éte moissonné depuis la source Math-Net.Ru

Voir la notice de l'article

The classical Fatou lemma states that the lower limit of a sequence of integrals of functions is greater than or equal to the integral of the lower limit. It is known that Fatou's lemma for a sequence of weakly converging measures states a weaker inequality because the integral of the lower limit is replaced with the integral of the lower limit in two parameters, where the second parameter is the argument of the functions. In the present paper, we provide sufficient conditions when Fatou's lemma holds in its classical form for a sequence of weakly converging measures. The functions can take both positive and negative values. Similar results for sequences of setwise converging measures are also proved. We also put forward analogies of Lebesgue's and the monotone convergence theorems for sequences of weakly and setwise converging measures. The results obtained are used to prove broad sufficient conditions for the validity of optimality equations for average-cost Markov decision processes.
Mots-clés : Fatou's lemma, setwise convergence
Keywords: measure, weak convergence, Markov decision process. 
@article{TVP_2020_65_2_a4,
     author = {E. A. Feinberg and P. O. Kas'yanov and Y. Liang},
     title = {Fatou's lemma in its classical form and {Lebesgue's} convergence theorems for varying measures with applications to {Markov} decision processes},
     journal = {Teori\^a vero\^atnostej i ee primeneni\^a},
     pages = {338--367},
     year = {2020},
     volume = {65},
     number = {2},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/TVP_2020_65_2_a4/}
}
TY  - JOUR
AU  - E. A. Feinberg
AU  - P. O. Kas'yanov
AU  - Y. Liang
TI  - Fatou's lemma in its classical form and Lebesgue's convergence theorems for varying measures with applications to Markov decision processes
JO  - Teoriâ veroâtnostej i ee primeneniâ
PY  - 2020
SP  - 338
EP  - 367
VL  - 65
IS  - 2
UR  - http://geodesic.mathdoc.fr/item/TVP_2020_65_2_a4/
LA  - ru
ID  - TVP_2020_65_2_a4
ER  - 
%0 Journal Article
%A E. A. Feinberg
%A P. O. Kas'yanov
%A Y. Liang
%T Fatou's lemma in its classical form and Lebesgue's convergence theorems for varying measures with applications to Markov decision processes
%J Teoriâ veroâtnostej i ee primeneniâ
%D 2020
%P 338-367
%V 65
%N 2
%U http://geodesic.mathdoc.fr/item/TVP_2020_65_2_a4/
%G ru
%F TVP_2020_65_2_a4
E. A. Feinberg; P. O. Kas'yanov; Y. Liang. Fatou's lemma in its classical form and Lebesgue's convergence theorems for varying measures with applications to Markov decision processes. Teoriâ veroâtnostej i ee primeneniâ, Tome 65 (2020) no. 2, pp. 338-367. http://geodesic.mathdoc.fr/item/TVP_2020_65_2_a4/

[1] D. P. Bertsekas, S. E. Shreve, Stochastic optimal control. The discrete time case, Reprint of 1978 original, Athena Scientific, Belmont, MA, 1996, 330 pp. | MR | MR | Zbl | Zbl

[2] P. Billingsley, Convergence of probability measures, Wiley Ser. Probab. Statist. Probab. Statist., 2nd ed., John Wiley Sons, Inc., New York, 1999, x+277 pp. | DOI | MR | MR | Zbl

[3] E. A. Feinberg, “Optimality conditions for inventory control”, Optimization challenges in complex, networked, and risky systems, Tutor. Oper. Res., INFORMS, Cantonsville, MD, 2016, 14–44 | DOI

[4] E. A. Feinberg, P. O. Kasyanov, Y. Liang, “Fatou's lemma for weakly converging measures under the uniform integrability condition”, Teoriya veroyatn. i ee primen., 64:4 (2019), 771–790 ; Theory Probab. Appl., 64:4 (2019), 615–630 ; (2018), arXiv: 1807.07931 | DOI | MR | Zbl | DOI

[5] E. A. Feinberg, P. O. Kasyanov, N. V. Zadoianchuk, “Average cost Markov decision processes with weakly continuous transition probabilities”, Math. Oper. Res., 37:4 (2012), 591–607 | DOI | MR | Zbl

[6] E. A. Feinberg, P. O. Kasyanov, N. V. Zadoianchuk, “Fatou's lemma for weakly converging probabilities”, Teoriya veroyatn. i ee primen., 58:4 (2013), 812–818 ; Theory Probab. Appl., 58:4 (2014), 683–689 | DOI | Zbl | DOI | MR

[7] E. A. Feinberg, P. O. Kasyanov, N. V. Zadoianchuk, “Berge's theorem for noncompact image sets”, J. Math. Anal. Appl., 397:1 (2013), 255–259 | DOI | MR | Zbl

[8] E. A. Feinberg, P. O. Kasyanov, M. Z. Zgurovsky, “Convergence of probability measures and Markov decision models with incomplete information”, Stokhasticheskoe ischislenie, martingaly i ikh primeneniya, Sbornik statei. K 80-letiyu so dnya rozhdeniya akademika Alberta Nikolaevicha Shiryaeva, Tr. MIAN, 287, MAIK “Nauka/Interperiodika”, M., 2014, 103–124 | DOI | MR

[9] E. A. Feinberg, P. O. Kasyanov, M. Z. Zgurovsky, “Uniform Fatou's lemma”, J. Math. Anal. Appl., 444:1 (2016), 550–567 | DOI | MR | Zbl

[10] E. A. Feinberg, P. O. Kasyanov, M. Z. Zgurovsky, “Partially observable total-cost Markov decision processes with weakly continuous transition probabilities”, Math. Oper. Res., 41:2 (2016), 656–681 | DOI | MR | Zbl

[11] E. A. Feinberg, M. E. Lewis, “On the convergence of optimal actions for Markov decision processes and the optimality of $(s,S)$ inventory policies”, Naval Res. Logist., 65:8 (2018), 619–637 | DOI | MR | Zbl

[12] E. A. Feinberg, Y. Liang, “On the optimality equation for average cost Markov decision processes and its validity for inventory control”, Ann. Oper. Res., Publ. online: 2017 | DOI

[13] O. Hernández-Lerma, “Average optimality in dynamic programming on Borel spaces — unbounded costs and controls”, Systems Control Lett., 17:3 (1991), 237–242 | DOI | MR | Zbl

[14] O. Hernández-Lerma, J. B. Lasserre, Discrete-time Markov control processes. Basic optimality criteria, Appl. Math. (N. Y.), 30, Springer-Verlag, New York, 1996, xiv+216 pp. | DOI | MR | Zbl

[15] J. Jacod, A. N. Shiryaev, Limit theorems for stochastic processes, Grundlehren Math. Wiss., 288, 2nd ed., Springer-Verlag, Berlin, 2003, xx+661 pp. | DOI | MR | MR | MR | Zbl | Zbl

[16] Yu. M. Kabanov, R. Sh. Liptser, “On convergence in variation of the distributions of multivariate point processes”, Z. Wahrsch. Verw. Gebiete, 63:4 (1983), 475–485 | DOI | MR | Zbl

[17] M. V. Kartashov, Imovirnist, protsesi, statistika, Kiïv. un-t, Kiïv, 2008, 494 pp.

[18] H. L. Royden, Real analysis, 2nd ed., The Macmillan Co., New York, 1968, xii+349 pp. | MR | Zbl

[19] M. Schäl, “Average optimality in dynamic programming with general state space”, Math. Oper. Res., 18:1 (1993), 163–172 | DOI | MR | Zbl

[20] R. Serfozo, “Convergence of Lebesgue integrals with varying measures”, Sankhyā Ser. A, 44:3 (1982), 380–402 | MR | Zbl

[21] A. N. Shiryaev, Probability, Grad. Texts in Math., 95, 2nd ed., Springer-Verlag, New York, 1996, xvi+623 pp. | DOI | MR | MR | Zbl | Zbl

[22] A. W. van der Vaart, Asymptotic statistics, Camb. Ser. Stat. Probab. Math., 3, Cambridge Univ. Press, Cambridge, 1998, xvi+443 pp. | DOI | MR | Zbl