Efficient Computation of the Fisher Information Matrix in the EM Algorithm

The expectation-maximization (EM) algorithm is an iterative computational method to calculate the maximum likelihood estimators (MLEs) from the sample data. When the MLE is available, we naturally want the Fisher information matrix (FIM) of unknown parameters. However, one of the limitations of the EM algorithm is that the FIM is not an automatic by-product of the algorithm. In this paper, we construct a simple Monte Carlo-based method requiring only the gradient values of the function we obtain from the E step and basic operations. The key part of our method is to utilize the simultaneous perturbation stochastic approximation method to estimate the Hessian matrix from the gradient of the conditional expectation of the complete-data log-likelihood function.