Structured Sparse Models

As continuing to emerge demand of high dimensional and ultra-high dimensional regression and classification in bioinformatics, psychology diagnosis, computational linguistics and phonetics, computer vision, the Portal site, e-commerce, mobile Internet, and Internet of Things, there is an urgent need to study high dimensional and ultra-high dimensional variable selection and feature dimension reduction in regression and classification model. Thus the sparse models have been quite popular in recent years, such as the Lasso, adaptive Lasso and the elastic net. However, these sparse models ignore the structural information of the variables, such as the group structure sparsity, overlapping group structure sparsity, bi-level sparse structure, Multi-layer Sparse structure, tree structure sparsity and graph structure sparsity. The structured sparse models that consider this structural prior information can improve the statistic properties of the sparse models when facing with the corresponding structure sparse datasets. The structured sparse models are the hot research direction of the sparse model learning and many research findings appear in recent years. This paper gives a systematic survey of mainstream of structured sparsity model, such as group structure sparse model, structure sparse dictionary learning, bi-level structure sparse model, and tree structure sparse model and graphical structure sparse model. As objective function of structure sparse model contains non-differential, non-convex and non-separable variable, objective function of structure sparse model first needs to be approximately transform into differentiable, convex and separable variable ones. The main approximate transformation methods are summarized, including majority-minority inequality, approximate method of Nesterov's double objective function, first order Taylor expansion and second order Taylor expansion. Optimization algorithms solving approximate objective function of structure sparse model are carried out a detailed comparative analysis on the conception, the features and performance, which involves minimum angle regression, group Least angle regression, block coordinate descent algorithm, block coordinate gradient descent algorithm, local coordinate descent algorithm, spectrum projection gradient method, active set algorithm and alternating direction method of multipliers, some future research directions are discussed in the final section. © 2017, Science Press. All right reserved.