Geodesic
Selection of a~metric for the nearest neighbor entropy estimators
Fundamentalʹnaâ i prikladnaâ matematika, Tome 18 (2013) no. 2, pp. 209-227.

Voir la notice de l'article provenant de la source Math-Net.Ru

We consider the problem of improving the efficiency of the nonparametric entropy estimation for a stationary ergodic process. Our approach is based on the nearest-neighbor distances. We propose a broad class of metrics on the space of right-sided infinite sequences drawn from a finite alphabet. The new metric has a parameter which is a nonincreasing function. We prove that, under certain conditions, our estimators have a small variance and show that a special selection of the metric parameters reduces the estimator's bias. 
@article{FPM_2013_18_2_a16,
     author = {E. A. Timofeev},
     title = {Selection of a~metric for the nearest neighbor entropy estimators},
     journal = {Fundamentalʹna\^a i prikladna\^a matematika},
     pages = {209--227},
     publisher = {mathdoc},
     volume = {18},
     number = {2},
     year = {2013},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/FPM_2013_18_2_a16/}
}
TY  - JOUR
AU  - E. A. Timofeev
TI  - Selection of a~metric for the nearest neighbor entropy estimators
JO  - Fundamentalʹnaâ i prikladnaâ matematika
PY  - 2013
SP  - 209
EP  - 227
VL  - 18
IS  - 2
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/FPM_2013_18_2_a16/
LA  - ru
ID  - FPM_2013_18_2_a16
ER  - 
%0 Journal Article
%A E. A. Timofeev
%T Selection of a~metric for the nearest neighbor entropy estimators
%J Fundamentalʹnaâ i prikladnaâ matematika
%D 2013
%P 209-227
%V 18
%N 2
%I mathdoc
%U http://geodesic.mathdoc.fr/item/FPM_2013_18_2_a16/
%G ru
%F FPM_2013_18_2_a16
E. A. Timofeev. Selection of a~metric for the nearest neighbor entropy estimators. Fundamentalʹnaâ i prikladnaâ matematika, Tome 18 (2013) no. 2, pp. 209-227. http://geodesic.mathdoc.fr/item/FPM_2013_18_2_a16/

[1] Aldous D., Shields P., “A diffusion limit for a class of randomly-growing binary trees”, Probab. Th. Rel. Fields, 79 (1988), 509–542 | DOI | MR | Zbl

[2] Devroye L., “Exponential inequalities in nonparametric estimation”, Nonparametric Functional Estimation and Related Topics, ed. G. Roussas, Kluwer Academic, Dordrecht, 1991, 31–44 | DOI | MR

[3] Deza M., Deza T., Encyclopedia of Distances, Springer, Berlin, 2009 | MR

[4] Gradshtein I. S., Ryzhik I. M., Table of Integrals, Series, and Products, Academic Press, London, 1994 | MR

[5] Grassberger P., “Estimating the information content of symbol sequences and efficient codes”, IEEE Trans. Inform. Theory, 35 (1989), 669–675 | DOI | MR

[6] Kaltchenko A., Timofeeva N., “Entropy estimators with almost sure convergence and an $O(n^{-1})$ variance”, Adv. Math. Commun., 2 (2008), 1–13 | DOI | MR | Zbl

[7] Kaltchenko A., Timofeeva N., “Rate of convergence of the nearest neighbor entropy estimator”, AEU Int. J. Electron. Commun., 64 (2010), 75–79 | DOI

[8] Knuth D. E., The Art of Computer Programming, v. 3, Sorting and Searching, Addison-Wesley, Reading, 1975 | MR

[9] Kontoyiannis I., Suhov Yu. M., “Prefixes and the entropy rate for long-range sources”, Probability Statistics and Optimization, ed. F. P. Kelly, Wiley, New York, 1994, 89–98 | MR | Zbl

[10] Martin N., England J., Mathematical Theory of Entropy, Cambridge Univ. Press, Cambridge, 1984

[11] McDiarmid C., “On the method of bounded differences”, Surveys in Combinatorics, Cambridge Univ. Press, Cambridge, 1989, 148–188 | MR

[12] Shields P. C., “Entropy and prefixes”, Ann. Probab., 20 (1992), 403–409 | DOI | MR | Zbl

[13] Silvapulle M. J., Sen P. K., Constrained Statistical Inference: Inequality, Order and Shape Restrictions, Wiley, New York, 2005 | MR | Zbl

[14] Timofeev E. A., “Statistical estimation of measure invariants”, St. Petersburg Math. J., 17:3 (2006), 527–551 | DOI | MR | Zbl

[15] Timofeev E. A., “Bias of a nonparametric entropy estimator for Markov measures”, J. Math. Sci., 176:2 (2011), 255–269 | DOI | MR | Zbl

[16] Ziv J., Lempel A., “Compression of individual sequences by variable rate coding”, IEEE Trans. Inform. Theory, 24 (1978), 530–536 | DOI | MR | Zbl