Geodesic
On the maximum entropy of a sum of independent discrete random variables
Teoriâ veroâtnostej i ee primeneniâ, Tome 66 (2021) no. 3, pp. 601-609 Cet article a éte moissonné depuis la source Math-Net.Ru

Voir la notice de l'article

Let $X_1, \dots, X_n$ be independent random variables taking values in the alphabet $\{0, 1, \dots, r\}$, and let $S_n=\sum_{i=1}^n X_i$. The Shepp–Olkin theorem states that in the binary case (${r=1}$), the Shannon entropy of $S_n$ is maximized when all the $X_i$'s are uniformly distributed, i.e., Bernoulli(1/2). In an attempt to generalize this theorem to arbitrary finite alphabets, we obtain a lower bound on the maximum entropy of $S_n$ and prove that it is tight in several special cases. In addition to these special cases, an argument is presented supporting the conjecture that the bound represents the optimal value for all $n$, $r$, i.e., that $H(S_n)$ is maximized when $X_1, \dots, X_{n-1}$ are uniformly distributed over $\{0, r\}$, while the probability mass function of $X_n$ is a mixture (with explicitly defined nonzero weights) of the uniform distributions over $\{0, r\}$ and $\{1, \dots, r-1\}$.
Keywords: maximum entropy, Shepp–Olkin theorem, ultra-log-concavity.
Mots-clés : Bernoulli sum, binomial distribution 
@article{TVP_2021_66_3_a11,
     author = {M. Kova\v{c}evi\'c},
     title = {On the maximum entropy of a~sum of independent discrete random variables},
     journal = {Teori\^a vero\^atnostej i ee primeneni\^a},
     pages = {601--609},
     year = {2021},
     volume = {66},
     number = {3},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/TVP_2021_66_3_a11/}
}
TY  - JOUR
AU  - M. Kovačević
TI  - On the maximum entropy of a sum of independent discrete random variables
JO  - Teoriâ veroâtnostej i ee primeneniâ
PY  - 2021
SP  - 601
EP  - 609
VL  - 66
IS  - 3
UR  - http://geodesic.mathdoc.fr/item/TVP_2021_66_3_a11/
LA  - ru
ID  - TVP_2021_66_3_a11
ER  - 
%0 Journal Article
%A M. Kovačević
%T On the maximum entropy of a sum of independent discrete random variables
%J Teoriâ veroâtnostej i ee primeneniâ
%D 2021
%P 601-609
%V 66
%N 3
%U http://geodesic.mathdoc.fr/item/TVP_2021_66_3_a11/
%G ru
%F TVP_2021_66_3_a11
M. Kovačević. On the maximum entropy of a sum of independent discrete random variables. Teoriâ veroâtnostej i ee primeneniâ, Tome 66 (2021) no. 3, pp. 601-609. http://geodesic.mathdoc.fr/item/TVP_2021_66_3_a11/

[1] P. Harremoës, “Binomial and Poisson distributions as maximum entropy distributions”, IEEE Trans. Inform. Theory, 47:5 (2001), 2039–2041 | DOI | MR | Zbl

[2] E. Hillion, O. Johnson, “Discrete versions of the transport equation and the Shepp–Olkin conjecture”, Ann. Probab., 44:1 (2016), 276–306 | DOI | MR | Zbl

[3] E. Hillion, O. Johnson, “A proof of the Shepp–Olkin entropy concavity conjecture”, Bernoulli, 23:4B (2017), 3638–3649 | DOI | MR | Zbl

[4] E. Hillion, O. Johnson, “A proof of the Shepp–Olkin entropy monotonicity conjecture”, Electron. J. Probab., 24 (2019), 126, 14 pp. | DOI | MR | Zbl

[5] O. Johnson, “Log-concavity and the maximum entropy property of the Poisson distribution”, Stochastic Process. Appl., 117:6 (2007), 791–802 | DOI | MR | Zbl

[6] J. N. Kapur, Maximum-entropy models in science and engineering, John Wiley Sons, Inc., New York, 1989, xii+635 pp. | MR | Zbl

[7] P. Mateev, “On the entropy of the multinomial distribution”, Theory Probab. Appl., 23:1 (1978), 188–190 | DOI | MR | Zbl

[8] E. Ordentlich, “Maximizing the entropy of a sum of independent bounded random variables”, IEEE Trans. Inform. Theory, 52:5 (2006), 2176–2181 | DOI | MR | Zbl

[9] L. A. Shepp, I. Olkin, Entropy of the sum of independent Bernoulli random variables and of the multinomial distribution, Tech. Report 131, Stanford Univ., Stanford, CA, 1978, 10 pp. ; Contributions to probability, Academic Press, New York, 1981, 201–206 https://statistics.stanford.edu/research/entropy-sum-independent-bernoulli-random-variables-and-multinomial-distribution | MR | Zbl

[10] Yaming Yu, “Maximum entropy for sums of symmetric and bounded random variables: a short derivation”, IEEE Trans. Inform. Theory, 54:4 (2008), 1818–1819 | DOI | MR | Zbl

[11] Yaming Yu, “On the maximum entropy properties of the binomial distribution”, IEEE Trans. Inform. Theory, 54:7 (2008), 3351–3353 | DOI | MR | Zbl