Supervised latent Dirichlet allocation with a mixture of sparse softmax

Real data often show that from appearance within-class similarity is relatively low and between-class similarity is relatively high, which could increase the difficulty of classification. To classify this kind of data effectively, we learn multiple classification criteria simultaneously, and make different classification criterion be applied to classify different data for the purpose of relieving difficulty of fitting this kind of data and class label only by using a single classifier. Considering that topic model can learn high-level semantic features of the original data, and that mixture of softmax model is an efficient and effective probabilistic ensemble classification method, we embed a mixture of softmax model into latent Dirichlet allocation model, and propose a supervised topic model, supervised latent Dirichlet allocation with a mixture of softmax, and its improved version, supervised latent Dirichlet allocation with a mixture of sparse softmax. Next, we give their parameter estimation algorithms based on variational Expectation Maximization (EM) method. Moreover, we give an approximation method to classify unseen data, and analyze the convergence of the parameter estimation algorithm. Finally, we demonstrate the effectiveness of the proposed models by comparing them with some recently proposed approaches on two real image datasets and one text dataset. The experimental results demonstrate the good performance of the proposed models. (C) 2018 Published by Elsevier B.V.