Geodesic
Generalized information criteria for Bayes decisions
Kybernetika, Tome 48 (2012) no. 4, pp. 714-749 Cet article a éte moissonné depuis la source Czech Digital Mathematics Library

Voir la notice de l'article

This paper deals with Bayesian models given by statistical experiments and standard loss functions. Bayes probability of error and Bayes risk are estimated by means of classical and generalized information criteria applicable to the experiment. The accuracy of the estimation is studied. Among the information criteria studied in the paper is the class of posterior power entropies which include the Shannon entropy as special case for the power $\alpha =1$. It is shown that the most accurate estimate is in this class achieved by the quadratic posterior entropy of the power $\alpha =2$. The paper introduces and studies also a new class of alternative power entropies which in general estimate the Bayes errors and risk more tightly than the classical power entropies. Concrete examples, tables and figures illustrate the obtained results.
This paper deals with Bayesian models given by statistical experiments and standard loss functions. Bayes probability of error and Bayes risk are estimated by means of classical and generalized information criteria applicable to the experiment. The accuracy of the estimation is studied. Among the information criteria studied in the paper is the class of posterior power entropies which include the Shannon entropy as special case for the power $\alpha =1$. It is shown that the most accurate estimate is in this class achieved by the quadratic posterior entropy of the power $\alpha =2$. The paper introduces and studies also a new class of alternative power entropies which in general estimate the Bayes errors and risk more tightly than the classical power entropies. Concrete examples, tables and figures illustrate the obtained results.
Classification : 62B10, 62C10
Keywords: Shannon entropy; alternative Shannon entropy; power entropies; alternative power entropies; Bayes error; Bayes risk; sub-Bayes risk 
@article{KYB_2012_48_4_a5,
     author = {Morales, Domingo and Vajda, Igor},
     title = {Generalized information criteria for {Bayes} decisions},
     journal = {Kybernetika},
     pages = {714--749},
     year = {2012},
     volume = {48},
     number = {4},
     mrnumber = {3013396},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/KYB_2012_48_4_a5/}
}
TY  - JOUR
AU  - Morales, Domingo
AU  - Vajda, Igor
TI  - Generalized information criteria for Bayes decisions
JO  - Kybernetika
PY  - 2012
SP  - 714
EP  - 749
VL  - 48
IS  - 4
UR  - http://geodesic.mathdoc.fr/item/KYB_2012_48_4_a5/
LA  - en
ID  - KYB_2012_48_4_a5
ER  - 
%0 Journal Article
%A Morales, Domingo
%A Vajda, Igor
%T Generalized information criteria for Bayes decisions
%J Kybernetika
%D 2012
%P 714-749
%V 48
%N 4
%U http://geodesic.mathdoc.fr/item/KYB_2012_48_4_a5/
%G en
%F KYB_2012_48_4_a5
Morales, Domingo; Vajda, Igor. Generalized information criteria for Bayes decisions. Kybernetika, Tome 48 (2012) no. 4, pp. 714-749. http://geodesic.mathdoc.fr/item/KYB_2012_48_4_a5/

[1] M. Ben Bassat: $f$-entropies, probability of error, and feature selection. Inform. Control 39 (1978), 227-242. | DOI | MR | Zbl

[2] M. Ben Bassat, J. Raviv: Rényi's entropy and probability of error. IEEE Trans. Inform. Theory 24 (1978), 324-331. | DOI | MR

[3] J. O. Berger: Statistical Decision Theory and Bayesian Analysis. Second edition. Springer, Berlin 1986. | MR

[4] T. M. Cover, P. E. Hart: Nearest neighbor pattern classification. IEEE Trans. Inform. Theory 13 (1967), 21-27. | DOI | Zbl

[5] P. Devijver, J. Kittler: Pattern Recognition. A Statistical Approach. Prentice Hall, Englewood Cliffs, New Jersey 1982. | MR | Zbl

[6] L. Devroye, L. Györfi, G. Lugosi: A Probabilistic Theory of Pattern Recognition 1996. Springer, Berlin 1996. | MR

[7] D. K. Faddeev: Zum Begriff der Entropie einer endlichen Wahrscheinlichkeitsschemas. Vol. I. Deutscher Verlag der Wissenschaften, Berlin 1957.

[8] M. Feder, N. Merhav: Relations between entropy and error probability. IEEE Trans. Inform. Theory 40 (1994), 259-266. | DOI | Zbl

[9] P. Harremoës, F. Topsøe: Inequalities between entropy and index of coincidence derived from information diagrams. IEEE Trans. Inform. Theory 47 (2001), 2944-2960. | DOI | MR

[10] J. Havrda, F. Charvát: Concept of structural $a$-entropy. Kybernetika 3 (1967), 30-35. | MR | Zbl

[11] L. Kanal: Patterns in pattern recognittion. IEEE Trans. Inform. Theory 20 (1974), 697-707. | MR

[12] V. A. Kovalevsky: The problem of character recognition from the point of view of mathematical statistics. In: Reading Automata and Pattern Recognition (in Russian) (Naukova Dumka, Kyjev, ed. 1965). English translation in: Character Readers and Pattern Recognition, Spartan Books, New York 1968, pp. 3-30.

[13] D. Morales, L. Pardo, I. Vajda: Uncertainty of discrete stochastic systems: general theory and statistical inference. IEEE Trans. System, Man and Cybernetics, Part A 26 (1996), 1-17.

[14] A. Rényi: Proceedings of 4th Berkeley Symp. on Probab. Statist. University of California Press, Berkeley, California 1961. | MR

[15] N. P. Salikhov: Confirmation of a hypothesis of I. Vajda (in Russian). Problemy Peredachi Informatsii 10 (1974), 114-115. | MR

[16] D. L. Tebbe, S. J. Dwyer III: Uncertainty and probability of error. IEEE Trans. Inform. Theory 14 (1968), 516-518. | DOI

[17] G. T. Toussaint: A generalization of Shannon's equivocation and the Fano bound. IEEE Trans. System, Man and Cybernetics 7 (1977), 300-302. | DOI | MR | Zbl

[18] I. Vajda: Bounds on the minimal error probability and checking a finite or countable number of hypotheses. Inform. Transmission Problems 4 (1968), 9-17. | MR

[19] I. Vajda: A contribution to informational analysis of patterns. In: Methodologies of Pattern Recognition (M. S. Watanabe, ed.), Academic Press, New York 1969.

[20] I. Vajda, K. Vašek: Majorization, concave entropies and comparison of experiments. Problems Control Inform. Theory 14 (1985), 105-115. | MR | Zbl

[21] I. Vajda, J. Zvárová: On generalized entropies, Bayesian decisions and statistical diversity. Kybernetika 43 (2007), 675-696. | MR | Zbl