TLS-EM algorithm of Mixture Density Models for exponential families

As a widely used model, Mixture Density Model (MDM) is traditionally solved by Expectation-Maximization (EM) algorithm. EM maximizes a lower bound function iteratively, especially for exponential families. This paper managed to improve EM by combining it with Total Least Squares (TLS), proposing a new algorithm called the TLS-EM algorithm. In this algorithm, parameters are divided into two groups, linear parameters and sub-model parameters. They are solved in each iteration separately. First, data set is separated in different intervals and the conditional maximizing question is transformed into the over-determined linear equations. TLS is adopted to solve these equations and calculate linear parameters, with sub-model parameters fixed. Second, sub-model parameters are solved with EM. Properties of TLS-EM have been provided with proofs. Combining the properties of TLS, EM and the properties of its own, TLS-EM not only inherits most advantages of EM but also improves it in most cases, especially in bad initial or bad model conditions. Numerical experiments confirm these properties. (c) 2021 Elsevier B.V. All rights reserved.