Mixture of experts architectures for neural networks as a special case of conditional expectation formula

Grim, Jiří

Kybernetika, Tome 34 (1998) no. 4, p. [417].

Résumé

Recently a new interesting architecture of neural networks called “mixture of experts” has been proposed as a tool of real multivariate approximation or prediction. We show that the underlying problem is closely related to approximating the joint probability density of involved variables by finite mixture. Particularly, assuming normal mixtures, we can explicitly write the conditional expectation formula which can be interpreted as a mixture-of- experts network. In this way the related optimization problem can be reduced to standard estimation of normal mixtures by means of EM algorithm. The resulting prediction is optimal in the sense of minimum dispersion if the assumed mixture model is true. It is shown that some of the recently published results can be obtained by specifying the normal components of mixtures in a special form.

Détail

Classification : 68T05, 92B20
Keywords: neural networks; mixtures; multivariate approximation; prediction