Existence of an unbiased consistent entropy estimator for the special Bernoulli measure

E. A. Timofeev

Modelirovanie i analiz informacionnyh sistem, Tome 26 (2019) no. 2, pp. 267-278

Résumé

Let $\Omega = \mathcal{A}^{\mathbb{N}}$ be a space of right-sided infinite sequences drawn from a finite alphabet $\mathcal{A} = \{0,1\}$, $\mathbb{N} = \{1,2,\dots \} $, $$ \rho(\boldsymbol{x},\boldsymbol{y}) = \sum_{k=1}^{\infty}|x_{k} - y_{k}|2^{-k} $$ — a metric on $\Omega = \mathcal{A}^{\mathbb{N}}$, and $\mu$ — a probability measure on $\Omega$. Let $\boldsymbol{\xi_0}, \boldsymbol{\xi_1}, \dots, \boldsymbol{\xi_n}$ be independent identically distributed points on $\Omega$. We study the estimator $\eta_n^{(k)}(\gamma)$ of the reciprocal of the entropy $1/h$ that are defined as $$ \eta_n^{(k)}(\gamma) = k \left(r_{n}^{(k)}(\gamma) - r_{n}^{(k+1)}(\gamma)\right), $$ where $$ r_n^{(k)}(\gamma) = \frac{1}{n+1}\sum_{j=0}^{n} \gamma\left(\min_{i:i \neq j} {^{(k)}} \rho(\boldsymbol{\xi_{i}}, \boldsymbol{\xi_{j}})\right), $$ $\min ^{(k)}\{X_1,\dots,X_N\}= X_k$, if $X_1\leq X_2\leq \dots\leq X_N$. Number $k$ and a function $\gamma(t)$ are auxiliary parameters. The main result of this paper is Theorem. Let $\mu$ be the Bernoulli measure with probabilities $p_0,p_1>0$, $p_0+p_1=1$, $p_0=p_1^2$, then $\forall \varepsilon>0$ $\exists$ some continuous function $\gamma(t)$ such that $$ \left|\mathsf{E}\eta_n^{(k)}(\gamma) - \frac1h\right| \varepsilon,\quad \mathsf{Var}\,\eta_n^{(k)}(\gamma)\to 0,\ n\to\infty. $$

Détail

Keywords: measure, metric, entropy, estimator, unbias, self-similar, Bernoulli measure.