Geodesic
Comparison of two methods for approximation of probability distributions with prescribed marginals
Kybernetika, Tome 43 (2007) no. 5, pp. 591-618 Cet article a éte moissonné depuis la source Czech Digital Mathematics Library

Voir la notice de l'article

Let $P$ be a discrete multidimensional probability distribution over a finite set of variables $N$ which is only partially specified by the requirement that it has prescribed given marginals $\lbrace P_{A};\ A\in {\cal S} \rbrace $, where ${\cal S}$ is a class of subsets of $N$ with $\bigcup {\cal S} = N$. The paper deals with the problem of approximating $P$ on the basis of those given marginals. The divergence of an approximation $\hat{P}$ from $P$ is measured by the relative entropy $H(P|\hat{P})$. Two methods for approximating $P$ are compared. One of them uses formerly introduced concept of dependence structure simplification (see Perez [Per79]). The other one is based on an explicit expression, which has to be normalized. We give examples showing that neither of these two methods is universally better than the other. If one of the considered approximations $\hat{P}$ really has the prescribed marginals then it appears to be the distribution $P$ with minimal possible multiinformation. A simple condition on the class ${\cal S}$ implying the existence of an approximation $\hat{P}$ with prescribed marginals is recalled. If the condition holds then both methods for approximating $P$ give the same result.
Let $P$ be a discrete multidimensional probability distribution over a finite set of variables $N$ which is only partially specified by the requirement that it has prescribed given marginals $\lbrace P_{A};\ A\in {\cal S} \rbrace $, where ${\cal S}$ is a class of subsets of $N$ with $\bigcup {\cal S} = N$. The paper deals with the problem of approximating $P$ on the basis of those given marginals. The divergence of an approximation $\hat{P}$ from $P$ is measured by the relative entropy $H(P|\hat{P})$. Two methods for approximating $P$ are compared. One of them uses formerly introduced concept of dependence structure simplification (see Perez [Per79]). The other one is based on an explicit expression, which has to be normalized. We give examples showing that neither of these two methods is universally better than the other. If one of the considered approximations $\hat{P}$ really has the prescribed marginals then it appears to be the distribution $P$ with minimal possible multiinformation. A simple condition on the class ${\cal S}$ implying the existence of an approximation $\hat{P}$ with prescribed marginals is recalled. If the condition holds then both methods for approximating $P$ give the same result.
Classification : 62C25, 62G05, 68T37
Keywords: marginal problem; relative entropy; dependence structure simplification; explicit expression approximation; multiinformation; decomposable model; asteroid 
@article{KYB_2007_43_5_a1,
     author = {Perez, Albert and Studen\'y, Milan},
     title = {Comparison of two methods for approximation of probability distributions with prescribed marginals},
     journal = {Kybernetika},
     pages = {591--618},
     year = {2007},
     volume = {43},
     number = {5},
     mrnumber = {2376326},
     zbl = {1144.68379},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/KYB_2007_43_5_a1/}
}
TY  - JOUR
AU  - Perez, Albert
AU  - Studený, Milan
TI  - Comparison of two methods for approximation of probability distributions with prescribed marginals
JO  - Kybernetika
PY  - 2007
SP  - 591
EP  - 618
VL  - 43
IS  - 5
UR  - http://geodesic.mathdoc.fr/item/KYB_2007_43_5_a1/
LA  - en
ID  - KYB_2007_43_5_a1
ER  - 
%0 Journal Article
%A Perez, Albert
%A Studený, Milan
%T Comparison of two methods for approximation of probability distributions with prescribed marginals
%J Kybernetika
%D 2007
%P 591-618
%V 43
%N 5
%U http://geodesic.mathdoc.fr/item/KYB_2007_43_5_a1/
%G en
%F KYB_2007_43_5_a1
Perez, Albert; Studený, Milan. Comparison of two methods for approximation of probability distributions with prescribed marginals. Kybernetika, Tome 43 (2007) no. 5, pp. 591-618. http://geodesic.mathdoc.fr/item/KYB_2007_43_5_a1/

[1] Csiszár I., Matúš F.: Information projections revisited. IEEE Trans. Inform. Theory 49 (2003), 1474–1490 | MR | Zbl

[2] Kellerer H. G.: Verteilungsfunktionen mit gegebenem Marginalverteilungen (in German, translation: Distribution functions with given marginal distributions). Z. Wahrsch. verw. Gerbiete 3 (1964), 247–270 | MR

[3] Lauritzen S. L.: Graphical Models. Clarendon Press, Oxford 1996 | MR | Zbl

[4] Perez A.: $\varepsilon $-admissible simplifications of the dependence structure of random variables. Kybernetika 13 (1979), 439–449 | MR

[5] Perez A.: The barycenter concept of a set of probability measures as a tool in statistical decision. In: The book of abstracts of the 4th Internat. Vilnius Conference on Probability Theory and Mathematical Statistics 1985, pp. 226–228

[6] Perez A.: Princip maxima entropie a princip barycentra při integraci dílčích znalostí v expertních systémech (in Czech, translation: The maximum entropy principle and the barycenter principle in partial knowledge integration in expert systems). In: Metody umělé inteligence a expertní systémy III (V. Mařík and Z. Zdráhal, eds.), ČSVT – FEL ČVUT, Prague 1987, pp. 62–74

[7] Perez A.: Explicit expression Exe – containing the same multiinformation as that in the given marginal set – for approximating probability distributions. A manuscript in Word, 2003

[8] Studený M.: Pojem multiinformace v pravděpodobnostním rozhodování (in Czech, translation: The notion of multiinformation in probabilistic decision-making). CSc Thesis, Czechoslovak Academy of Sciences, Institute of Information Theory and Automation, Prague 1987

[9] Studený M.: Probabilistic Conditional Independence Structures. Springer–Verlag, London 2005