A Gibbs sampling approach to independent factor analysis

We present a Gibbs sampler for estimating parameters of the Independent Factor model. Independent Factor Analysis (IFA) is a generalization of Mixtures of Factor Analyzers, where instead we learn nonlinear subspaces in data. IFA can also be considered a method for blind source separation. The IFA generative model is hierarchical, with each factor modeled as an independent Mixture of Gaussians, each mixture component representing a factor state. Computing expectations over factors quickly becomes intractable with increasing number of factors as this requires summation over exponentially many state configurations, making parameter estimation via Expectation Maximization (EM) with an exact E-step infeasible. Unlike the Variational method that has been proposed, we take a simulation based approach to obtain exact parameter estimates. We define prior distributions and use a Gibbs sampler to obtain samples from the parameter posterior. Application to synthetic data demonstrates effectiveness of the method in estimating model parameters and robustness to model permutation invariance.