Comparison of two methods for approximation of probability distributions with prescribed marginals

Perez, Albert ; Studený, Milan

Kybernetika, Tome 43 (2007) no. 5, pp. 591-618.

Résumé

Let $P$ be a discrete multidimensional probability distribution over a finite set of variables $N$ which is only partially specified by the requirement that it has prescribed given marginals $\lbrace P_{A};\ A\in {\cal S} \rbrace $, where ${\cal S}$ is a class of subsets of $N$ with $\bigcup {\cal S} = N$. The paper deals with the problem of approximating $P$ on the basis of those given marginals. The divergence of an approximation $\hat{P}$ from $P$ is measured by the relative entropy $H(P|\hat{P})$. Two methods for approximating $P$ are compared. One of them uses formerly introduced concept of dependence structure simplification (see Perez [Per79]). The other one is based on an explicit expression, which has to be normalized. We give examples showing that neither of these two methods is universally better than the other. If one of the considered approximations $\hat{P}$ really has the prescribed marginals then it appears to be the distribution $P$ with minimal possible multiinformation. A simple condition on the class ${\cal S}$ implying the existence of an approximation $\hat{P}$ with prescribed marginals is recalled. If the condition holds then both methods for approximating $P$ give the same result.

Détail

Classification : 62C25, 62G05, 68T37
Keywords: marginal problem; relative entropy; dependence structure simplification; explicit expression approximation; multiinformation; decomposable model; asteroid