Geodesic
$\phi $PHI-divergences, sufficiency, Bayes sufficiency, and deficiency
Kybernetika, Tome 48 (2012) no. 4, pp. 690-713 Cet article a éte moissonné depuis la source Czech Digital Mathematics Library

Voir la notice de l'article

The paper studies the relations between $\phi$-divergences and fundamental concepts of decision theory such as sufficiency, Bayes sufficiency, and LeCam's deficiency. A new and considerably simplified approach is given to the spectral representation of $\phi $-divergences already established in Österreicher and Feldman [28] under restrictive conditions and in Liese and Vajda [22], [23] in the general form. The simplification is achieved by a new integral representation of convex functions in terms of elementary convex functions which are strictly convex at one point only. Bayes sufficiency is characterized with the help of a binary model that consists of the joint distribution and the product of the marginal distributions of the observation and the parameter, respectively. LeCam's deficiency is expressed in terms of $\phi $-divergences where $\phi $ belongs to a class of convex functions whose curvature measures are finite and satisfy a normalization condition.
The paper studies the relations between $\phi$-divergences and fundamental concepts of decision theory such as sufficiency, Bayes sufficiency, and LeCam's deficiency. A new and considerably simplified approach is given to the spectral representation of $\phi $-divergences already established in Österreicher and Feldman [28] under restrictive conditions and in Liese and Vajda [22], [23] in the general form. The simplification is achieved by a new integral representation of convex functions in terms of elementary convex functions which are strictly convex at one point only. Bayes sufficiency is characterized with the help of a binary model that consists of the joint distribution and the product of the marginal distributions of the observation and the parameter, respectively. LeCam's deficiency is expressed in terms of $\phi $-divergences where $\phi $ belongs to a class of convex functions whose curvature measures are finite and satisfy a normalization condition.
Classification : 62B05, 62B10, 62B15, 62G10
Keywords: divergences; sufficiency; Bayes sufficiency; deficiency 
@article{KYB_2012_48_4_a4,
     author = {Liese, Friedrich},
     title = {$\phi ${PHI-divergences,} sufficiency, {Bayes} sufficiency, and deficiency},
     journal = {Kybernetika},
     pages = {690--713},
     year = {2012},
     volume = {48},
     number = {4},
     mrnumber = {3013395},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/KYB_2012_48_4_a4/}
}
TY  - JOUR
AU  - Liese, Friedrich
TI  - $\phi $PHI-divergences, sufficiency, Bayes sufficiency, and deficiency
JO  - Kybernetika
PY  - 2012
SP  - 690
EP  - 713
VL  - 48
IS  - 4
UR  - http://geodesic.mathdoc.fr/item/KYB_2012_48_4_a4/
LA  - en
ID  - KYB_2012_48_4_a4
ER  - 
%0 Journal Article
%A Liese, Friedrich
%T $\phi $PHI-divergences, sufficiency, Bayes sufficiency, and deficiency
%J Kybernetika
%D 2012
%P 690-713
%V 48
%N 4
%U http://geodesic.mathdoc.fr/item/KYB_2012_48_4_a4/
%G en
%F KYB_2012_48_4_a4
Liese, Friedrich. $\phi $PHI-divergences, sufficiency, Bayes sufficiency, and deficiency. Kybernetika, Tome 48 (2012) no. 4, pp. 690-713. http://geodesic.mathdoc.fr/item/KYB_2012_48_4_a4/

[1] M. S. Ali, D. Silvey: A general class of coefficients of divergence of one distribution from another. J. Roy. Statist. Soc. Ser. B 28 (1966), 131-140. | MR | Zbl

[2] S. Arimoto: Information-theoretical considerations on estimation problems. Inform. Control. 19 (1971), 181-194. | DOI | MR | Zbl

[3] A. R. Barron, L. Györfi, E. C. van der Meulen: Distribution estimates consistent in total variation and two types of information divergence. IEEE Trans. Inform. Theory 38 (1990), 1437-1454. | DOI

[4] A. Berlinet, I. Vajda, E. C. van der Meulen: About the asymptotic accuracy of Barron density estimates. IEEE Trans. Inform. Theory 44 (1990), 999-1009. | DOI | MR | Zbl

[5] A. Bhattacharyya: On some analogues to the amount of information and their uses in statistical estimation. Sankhya 8 (1946), 1-14. | MR

[6] H. Chernoff: A measure of asymptotic efficiency for test of a hypothesis based on the sum of observations. Ann. Math. Statist. 23 (1952), 493-507. | DOI | MR

[7] B. S. Clarke, A. R. Barron: Information-theoretic asymptotics of Bayes methods. IEEE Trans. Inform. Theory 36 (1990), 453-471. | DOI | MR | Zbl

[8] I. Csiszár: Eine informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffscher Ketten. Publ. Math. Inst. Hungar. Acad. Sci.8 (1963), 84-108. | MR

[9] I. Csiszár: Information-type measures of difference of probability distributions and indirect observations. Studia Sci. Math. Hungar. 2, (1967), 299-318. | MR | Zbl

[10] T. Cover, J. Thomas: Elements of Information Theory. Wiley, New York 1991. | MR | Zbl

[11] M. H. De Groot: Optimal Statistical Decisions. McGraw Hill, New York 1970. | MR

[12] D. Feldman, F. Österreicher: A note on $f$-divergences. Studia Sci. Math. Hungar. 24 (1989), 191-200. | MR | Zbl

[13] A. Guntuboyina: Lower bounds for the minimax risk using $f$-divergences, and applications. IEEE Trans. Inform. Theory 57 (2011), 2386-2399. | DOI | MR

[14] C. Guttenbrunner: On applications of the representation of $f$-divergences as averaged minimal Bayesian risk. In: Trans. 11th Prague Conf. Inform. Theory, Statist. Dec. Funct., Random Processes A, 1992, pp. 449-456.

[15] L. Jager, J. A. Wellner: Goodness-of-fit tests via phi-divergences. Ann. Statist. 35 (2007), 2018-2053. | DOI | MR | Zbl

[16] T. Kailath: The divergence and Bhattacharyya distance measures in signal selection. IEEE Trans. Commun. Technol. 15 (1990), 52-60. | DOI

[17] S. Kakutani: On equivalence of infinite product measures. Ann. Math. 49 (1948), 214-224. | DOI | MR | Zbl

[18] S. Kullback, R. Leibler: On information and sufficiency. Ann. Math. Statist. 22 (1951), 79-86. | DOI | MR | Zbl

[19] L. LeCam: Locally asymptotically normal families of distributions. Univ. Calif. Publ. 3, (1960), 37-98. | MR

[20] L. LeCam: Asymptotic Methods in Statistical Decision Theory. Springer, Berlin 1986.

[21] F. Liese, I. Vajda: Convex Statistical Distances. Teubner, Leipzig 1987. | MR | Zbl

[22] F. Liese, I. Vajda: On divergence and informations in statistics and information theory. IEEE Trans. Inform. Theory 52 (2006), 4394-4412. | DOI | MR

[23] F. Liese, I. Vajda: $f$-divergences: Sufficiency, deficiency and testing of gypotheses. In: Advances in Inequalities from Probability Theory and Statistics. (N. S. Barnett and S. S. Dragomir, eds.), Nova Science Publisher, Inc., New York 2008, pp. 113-149. | MR

[24] F. Liese, K. J. Miescke: Statistical Decision Theory, Estimation, Testing and Selection. Springer, New York 2008. | MR | Zbl

[25] K. Matusita: Decision rules based on the distance, for problems of fit, two samples and estimation. Ann. Math. Statist. 26 (1955), 613-640. | DOI | MR | Zbl

[26] D. Mussmann: Decision rules based on the distance, for problems of fit, two samples and estimation. Studia Sci. Math. Hungar. 14 (1979), 37-41.

[27] X. Nguyen, M. J. Wainwright, M. I. Jordan: On surrogate loss functions and $f$-divergences. Ann. Statist. 37 (2009), 2018-2053. | DOI | MR | Zbl

[28] F. Österreicher, D. Feldman: Divergenzen von Wahrscheinlichkeitsverteilungen - integralgeometrisch betrachtet. Acta Math. Sci. Hungar. 37 (1981), 329-337. | DOI | MR | Zbl

[29] F. Österreicher, I. Vajda: Statistical information and discrimination. IEEE Trans. Inform. Theory 39 (1993), 1036-1039. | DOI | MR | Zbl

[30] J. Pfanzagl: A characterization of sufficiency by power functions. Metrika 21 (1974), 197-199. | DOI | MR | Zbl

[31] H. V. Poor: Robust decision design using a distance criterion. IEEE Trans. Inform. Theory 26 (1980), 578-587. | MR | Zbl

[32] M. R. C. Read, N. A. C. Cressie: Goodness-of-Fit Statistics for Discrete Multivariate Data. Springer, Berlin 1988. | MR | Zbl

[33] A. Rényi: On measures of entropy and information. In: Proc. 4th Berkeley Symp. on Probab. Theory and Math. Statist. Berkeley Univ. Press, Berkeley 1961, pp. 547-561. | MR | Zbl

[34] A. W. Roberts, D. E. Varberg: Convex Functions. Academic Press, New York 1973. | MR | Zbl

[35] M. J. Schervish: Theory of Statistics. Springer, New York 1995. | MR | Zbl

[36] C. E. Shannon: A mathematical theory of communication. Bell. Syst. Tech. J. 27 (1948), 379-423, 623-656. | DOI | MR | Zbl

[37] H. Strasser: Mathematical Theory of Statistics. De Gruyter, Berlin 1985. | MR | Zbl

[38] F. Topsøe: Information-theoretical optimization techniques. Kybernetika 15 (1979), 7-17. | MR

[39] F. Topsøe: Some inequalities for information divergence and related measures of discrimination. IEEE Trans. Inform. Theory 46 (2000), 1602-1609. | DOI | MR

[40] E. Torgersen: Comparison of Statistical Experiments. Cambridge Univ. Press, Cambridge 1991. | MR | Zbl

[41] I. Vajda: On the $f$-divergence and singularity of probability measures. Periodica Math. Hungar. 2 (1972), 223-234. | DOI | MR | Zbl

[42] I. Vajda: Theory of Statistical Inference and Information. Kluwer Academic Publishers, Dordrecht - Boston - London 1989. | Zbl

[43] I. Vajda: On metric divergences of probability measures. Kybernetika 45 (2009), 885-900. | MR

[44] I. Vajda: On convergence of information contained in quantized observations. IEEE Trans. Inform. Theory. 48 (1980) 2163-2172. | DOI | MR | Zbl

[45] I. Vincze: On the concept and measure of information contained in an observation. In: Contribution to Probability. (J. Gani and V. F. Rohatgi, eds.) Academic Press, New York 1981, pp. 207-214. | MR | Zbl