Existence of an unbiased consistent entropy estimator for the special Bernoulli measure

E. A. Timofeev

Modelirovanie i analiz informacionnyh sistem, Tome 26 (2019) no. 2, pp. 267-278.

Résumé

Let $\Omega = \mathcal{A}^{\mathbb{N}}$ be a space of right-sided infinite sequences drawn from a finite alphabet $\mathcal{A} = \{0,1\}$, $\mathbb{N} = \{1,2,\dots \} $, $$ \rho(\boldsymbol{x},\boldsymbol{y}) = \sum_{k=1}^{\infty}|x_{k} - y_{k}|2^{-k} $$ — a metric on $\Omega = \mathcal{A}^{\mathbb{N}}$, and $\mu$ — a probability measure on $\Omega$. Let $\boldsymbol{\xi_0}, \boldsymbol{\xi_1}, \dots, \boldsymbol{\xi_n}$ be independent identically distributed points on $\Omega$. We study the estimator $\eta_n^{(k)}(\gamma)$ of the reciprocal of the entropy $1/h$ that are defined as $$ \eta_n^{(k)}(\gamma) = k \left(r_{n}^{(k)}(\gamma) - r_{n}^{(k+1)}(\gamma)\right), $$ where $$ r_n^{(k)}(\gamma) = \frac{1}{n+1}\sum_{j=0}^{n} \gamma\left(\min_{i:i \neq j} {^{(k)}} \rho(\boldsymbol{\xi_{i}}, \boldsymbol{\xi_{j}})\right), $$ $\min ^{(k)}\{X_1,\dots,X_N\}= X_k$, if $X_1\leq X_2\leq \dots\leq X_N$. Number $k$ and a function $\gamma(t)$ are auxiliary parameters. The main result of this paper is Theorem. Let $\mu$ be the Bernoulli measure with probabilities $p_0,p_1>0$, $p_0+p_1=1$, $p_0=p_1^2$, then $\forall \varepsilon>0$ $\exists$ some continuous function $\gamma(t)$ such that $$ \left|\mathsf{E}\eta_n^{(k)}(\gamma) - \frac1h\right| \varepsilon,\quad \mathsf{Var}\,\eta_n^{(k)}(\gamma)\to 0,\ n\to\infty. $$

Détail

Keywords: measure, metric, entropy, estimator, unbias, self-similar, Bernoulli measure.

Bibliographie

[1] Falconer K. J., Fractal geometry : Mathematical Foundation and Applications, John Wiley Sons, NY USA, 1990 | MR

[2] Hutchinson J. E., “Fractals and self-similarity”, Indiana Univ. Math. J., 30 (1981), 713–747 | DOI | MR | Zbl

[3] Mauldin R. D., Williams S. C., “Random Recursive Constructions: Asymptotic Geometric and Topological Properties”, Transactions of the American Mathematical Society, 295:1 (1986), 325–346 | DOI | MR | Zbl

[4] Timofeev E. A., “Selection of a Metric for the Nearest Neighbor Entropy Estimators”, Journal of Mathematical Sciences, 203:6 (2014), 892–906 | DOI | MR | Zbl

[5] Kaltchenko A., Timofeeva N., “Entropy Estimators with Almost Sure Convergence and an $O(n^{-1})$ Variance”, Advances in Mathematics of Communications, 2:1 (2008), 1–13 | DOI | MR | Zbl

[6] Kaltchenko A., Timofeeva N., “Rate of convergence of the nearest neighbor entropy estimator”, AEU – International Journal of Electronics and Communications, 64:1 (2010), 75–79 | DOI

[7] Timofeeva N. E., “Construction of Entropy Estimator with Special Metric and Arbitrary Function”, Modeling and Analysis of Information Systems, 20:6 (2013), 174–178 (in Russ.)

[8] Timofeev E. A., “Bias of a nonparametric entropy estimator for Markov measures”, Journal of Mathematical Sciences, 176:2 (2011), 255–269 | DOI | MR | Zbl

[9] Timofeev E. A., “Statistical Estimation of measure invariants”, St. Petersburg Math. J., 17:3 (2006), 527–551 | DOI | MR | Zbl

[10] Timofeev E. A., “Existence of an unbiased entropy estimator for the special Bernoulli measure”, Modeling and Analysis of Information Systems, 24:5 (2017), 521–536 (in Russ.) | MR