Bayesian estimation and classification with incomplete data using mixture models

J. Zhang and R.M. Everson

Abstract

Reasoning from data in practical problems is frequently hampered by missing observations. Mixture models provide a powerful general semi-parametric method for modelling densities and have close links to radial basis function neural networks (RBFs). In this paper we extend the Data Augmentation (DA) technique for multiple imputation to Gaussian mixture models to permit fully Bayesian inference of the mixture model parameters and estimation of the missing values. The method is illustrated and compared to imputation using a single normal density on synthetic data and real-world data sets. In addition to a lower mean squared error, mixture models provide valuable information on the potentially multi-modal nature of imputed values. The DA formalism is extended to a classifier closely related to RBF networks to permit Bayesian classification with incomplete data; the technique is illustrated on synthetic and real-world datasets. This efficient technology enables us to perform Bayesian imputation, parameter estimation and classification simultaneously for data with missing values.

Gzipped postscript  (150 kb)     PDF  (147 kb)