Geodesic
Properties of unique information
Kybernetika, Tome 57 (2021) no. 3, pp. 383-403
Cet article a éte moissonné depuis la source Czech Digital Mathematics Library

Voir la notice de l'article

We study the unique information function $UI(T:X\setminus Y)$ defined by Bertschinger et al. within the framework of information decompositions. In particular, we study uniqueness and support of the solutions to the convex optimization problem underlying the definition of $UI$. We identify sufficient conditions for non-uniqueness of solutions with full support in terms of conditional independence constraints and in terms of the cardinalities of $T$, $X$ and $Y$. Our results are based on a reformulation of the first order conditions on the objective function as rank constraints on a matrix of conditional probabilities. These results help to speed up the computation of $UI(T:X\setminus Y)$, most notably when $T$ is binary. Optima in the relative interior of the optimization domain are solutions of linear equations if $T$ is binary. In the all binary case, we obtain a complete picture of where the optimizing probability distributions lie.
We study the unique information function $UI(T:X\setminus Y)$ defined by Bertschinger et al. within the framework of information decompositions. In particular, we study uniqueness and support of the solutions to the convex optimization problem underlying the definition of $UI$. We identify sufficient conditions for non-uniqueness of solutions with full support in terms of conditional independence constraints and in terms of the cardinalities of $T$, $X$ and $Y$. Our results are based on a reformulation of the first order conditions on the objective function as rank constraints on a matrix of conditional probabilities. These results help to speed up the computation of $UI(T:X\setminus Y)$, most notably when $T$ is binary. Optima in the relative interior of the optimization domain are solutions of linear equations if $T$ is binary. In the all binary case, we obtain a complete picture of where the optimizing probability distributions lie.
DOI : 10.14736/kyb-2021-3-0383
Classification : 94A15, 94A17
Keywords: information decomposition; unique information 
@article{10_14736_kyb_2021_3_0383,
     author = {Rauh, Johannes and Sch\"unemann, Maik and Jost, J\"urgen},
     title = {Properties of unique information},
     journal = {Kybernetika},
     pages = {383--403},
     year = {2021},
     volume = {57},
     number = {3},
     doi = {10.14736/kyb-2021-3-0383},
     mrnumber = {4299455},
     zbl = {07442516},
     language = {en},
     url = {http://geodesic.mathdoc.fr/articles/10.14736/kyb-2021-3-0383/}
}
TY  - JOUR
AU  - Rauh, Johannes
AU  - Schünemann, Maik
AU  - Jost, Jürgen
TI  - Properties of unique information
JO  - Kybernetika
PY  - 2021
SP  - 383
EP  - 403
VL  - 57
IS  - 3
UR  - http://geodesic.mathdoc.fr/articles/10.14736/kyb-2021-3-0383/
DO  - 10.14736/kyb-2021-3-0383
LA  - en
ID  - 10_14736_kyb_2021_3_0383
ER  - 
%0 Journal Article
%A Rauh, Johannes
%A Schünemann, Maik
%A Jost, Jürgen
%T Properties of unique information
%J Kybernetika
%D 2021
%P 383-403
%V 57
%N 3
%U http://geodesic.mathdoc.fr/articles/10.14736/kyb-2021-3-0383/
%R 10.14736/kyb-2021-3-0383
%G en
%F 10_14736_kyb_2021_3_0383
Rauh, Johannes; Schünemann, Maik; Jost, Jürgen. Properties of unique information. Kybernetika, Tome 57 (2021) no. 3, pp. 383-403. doi: 10.14736/kyb-2021-3-0383

[1] Amari, Shun-ichi, Nagaoka, Hiroshi: Methods of Information Geometry. American Mathematical Society 2000. | DOI

[2] Banerjee, P. Kr., Olbrich, E., Jost, J., Rauh, J.: Unique informations and deficiencies. In: Proc. Allerton, 2018. | DOI

[3] Banerjee, P. Kr., Rauh, J., Montúfar, G.: Computing the unique information. In: Proc. IEEE ISIT 2018, pp. 141-145 | DOI

[4] Bertschinger, N., Rauh, J., Olbrich, E., Jost, J., Ay, N.: Quantifying unique information. Entropy 16 (2014), 4, 2161-2183. | DOI

[5] Diaconis, Persi, Sturmfels, Bernd: Algebraic algorithms for sampling from conditional distributions. Ann. Statist. 26 (1998), 363-397.

[6] Fink, A.: The binomial ideal of the intersection axiom for conditional probabilities. J. Algebr. Combin. 33 (2011), 3, 455-463. | DOI

[7] Finn, C., Lizier, J. T.: Pointwise partial information decomposition using the specificity and ambiguity lattices. Entropy 20 (2018), 4, 297. | DOI

[8] Harder, M., Salge, Ch., Polani, D.: A bivariate measure of redundant information. Phys. Rev. E 87 (2013), 012130. | DOI

[9] Ince, R.: Measuring multivariate redundant information with pointwise common change in surprisal. Entropy 19 (2017), 7, 318. | DOI

[10] James, R., Emenheiser, J., Crutchfield, J.: Unique information via dependency constraints. J. Physics A 52 (2018), 1, 014002. | DOI

[11] Makkeh, A., Theis, D. O., Vicente, R.: Bivariate partial information decomposition: The optimization perspective. Entropy 19 (2017), 10, 530. | DOI

[12] Niu, X., Quinn, Ch.: A measure of synergy, redundancy, and unique information using information geometry. In: Proc. IEEE ISIT 2019. | DOI

[13] Rauh, J., Banerjee, P. Kr., Olbrich, E., Jost, J.: Unique information and secret key decompositions. In: 2019 IEEE International Symposium on Information Theory (ISIT), pp. 3042-3046. | DOI

[14] Rauh, J., Ay, N.: Robustness, canalyzing functions and systems design. Theory Biosciences 133 (2014), 2, 63-78. | DOI

[15] Rauh, J., Bertschinger, N., Olbrich, E., Jost, J.: Reconsidering unique information: Towards a multivariate information decomposition. In: Proc. IEEE ISIT 2014, pp. 2232-2236. | DOI

[16] Smith, N. A., R, Tromble, W.: Sampling Uniformly from the Unit Simplex. Technical Report 29, Johns Hopkins University 29, 2004.

[17] Williams, P., Beer, R.: Nonnegative decomposition of multivariate information. arXiv:1004.2515v1.

Cité par Sources :