Variational Bounds for Mixed-Data Factor Analysis
By Emtiyaz Mohammad Khan, Department of CS, UBC.
Abstract:
We propose a new variational EM algorithm for fitting factor analysis models
with mixed continuous and categorical observations. The algorithm is based on a
simple quadratic bound to the log-sum-exp function. In the special case of fully
observed binary data, the bound we propose is significantly faster than previous
variational methods. We show that EM is significantly more robust in the
presence
of missing data compared to treating the latent factors as parameters, which is
the
approach used by exponential family PCA and other related matrix-factorization
methods. A further benefit of the variational approach is that it can easily be
extended to the case of mixtures of factor analyzers, as we show. We present
results on synthetic and real data sets demonstrating several desirable
properties
of our proposed method.
This work is in collaboration with Benjamin Marlin (CS, UBC), Guillaume Bouchard
(XRCE), and Kevin Murphy (CS, UBC).