Sparse kernel logistic regression based on L1/2 regularization

被引:0
|
作者
XU Chen [1 ]
PENG ZhiMing [2 ]
JING WenFeng [2 ]
机构
[1] Department of Statistics,University of British Columbia,Vancouver,BC V6T1Z2,Canada
[2] Institute for Information and System Science,School of Mathematics and Statistics,Xi'an Jiaotong University
基金
中国国家自然科学基金;
关键词
classification; L1/2; regularization; thresholding algorithm; kernel logistic regression; support vectors;
D O I
暂无
中图分类号
O212.1 [一般数理统计];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
The sparsity driven classification technologies have attracted much attention in recent years,due to their capability of providing more compressive representations and clear interpretation.Two most popular classification approaches are support vector machines(SVMs) and kernel logistic regression(KLR),each having its own advantages.The sparsification of SVM has been well studied,and many sparse versions of 2-norm SVM,such as 1-norm SVM(1-SVM),have been developed.But,the sparsifiation of KLR has been less studied.The existing sparsification of KLR is mainly based on L1 norm and L2 norm penalties,which leads to the sparse versions that yield solutions not so sparse as it should be.A very recent study on L1/2 regularization theory in compressive sensing shows that L1/2 sparse modeling can yield solutions more sparse than those of 1 norm and 2 norm,and,furthermore,the model can be efficiently solved by a simple iterative thresholding procedure.The objective function dealt with in L1/2 regularization theory is,however,of square form,the gradient of which is linear in its variables(such an objective function is the so-called linear gradient function).In this paper,through extending the linear gradient function of L1/2 regularization framework to the logistic function,we propose a novel sparse version of KLR,the 1/2 quasi-norm kernel logistic regression(1/2-KLR).The version integrates advantages of KLR and L1/2 regularization,and defines an efficient implementation scheme of sparse KLR.We suggest a fast iterative thresholding algorithm for 1/2-KLR and prove its convergence.We provide a series of simulations to demonstrate that 1/2-KLR can often obtain more sparse solutions than the existing sparsity driven versions of KLR,at the same or better accuracy level.The conclusion is also true even in comparison with sparse SVMs(1-SVM and 2-SVM).We show an exclusive advantage of 1/2-KLR that the regularization parameter in the algorithm can be adaptively set whenever the sparsity(correspondingly,the number of support vectors) is given,which suggests a methodology of comparing sparsity promotion capability of different sparsity driven classifiers.As an illustration of benefits of 1/2-KLR,we give two applications of 1/2-KLR in semi-supervised learning,showing that 1/2-KLR can be successfully applied to the classification tasks in which only a few data are labeled.
引用
收藏
页码:75 / 90
页数:16
相关论文
共 50 条
  • [11] Sparse logistic regression with a L1/2 penalty for gene selection in cancer classification
    Liang, Yong
    Liu, Cheng
    Luan, Xin-Ze
    Leung, Kwong-Sak
    Chan, Tak-Ming
    Xu, Zong-Ben
    Zhang, Hai
    BMC BIOINFORMATICS, 2013, 14
  • [12] Sparse logistic regression with a L1/2 penalty for gene selection in cancer classification
    Yong Liang
    Cheng Liu
    Xin-Ze Luan
    Kwong-Sak Leung
    Tak-Ming Chan
    Zong-Ben Xu
    Hai Zhang
    BMC Bioinformatics, 14
  • [13] αl1 - βl2 regularization for sparse recovery
    Ding, Liang
    Han, Weimin
    INVERSE PROBLEMS, 2019, 35 (12)
  • [14] Sparse Channel Estimation Based on L1/2 Regularization in OFDM Systems
    Duan, WenLei
    Li, Feng
    Liu, Zhe
    2014 INTERNATIONAL CONFERENCE ON INFORMATION AND COMMUNICATION TECHNOLOGY CONVERGENCE (ICTC), 2014, : 442 - 445
  • [15] NONCONVEX L1/2 REGULARIZATION FOR SPARSE PORTFOLIO SELECTION
    Xu, Fengmin
    Wang, Guan
    Gao, Yuelin
    PACIFIC JOURNAL OF OPTIMIZATION, 2014, 10 (01): : 163 - 176
  • [16] Sparse possibilistic clustering with L1 regularization
    Inokuchi, Ryo
    Miyamoto, Sadaaki
    GRC: 2007 IEEE INTERNATIONAL CONFERENCE ON GRANULAR COMPUTING, PROCEEDINGS, 2007, : 442 - 445
  • [17] SPARSE REPRESENTATION LEARNING OF DATA BY AUTOENCODERS WITH L1/2 REGULARIZATION
    Li, F.
    Zurada, J. M.
    Wu, W.
    NEURAL NETWORK WORLD, 2018, 28 (02) : 133 - 147
  • [18] Sparse Gabor Time-Frequency Representation Based on l1/2-l2 Regularization
    Li, Rui
    Zhou, Jian
    CIRCUITS SYSTEMS AND SIGNAL PROCESSING, 2019, 38 (10) : 4700 - 4722
  • [19] l1/2,1 group sparse regularization for compressive sensing
    Liu, Shengcai
    Zhang, Jiangshe
    Liu, Junmin
    Yin, Qingyan
    SIGNAL IMAGE AND VIDEO PROCESSING, 2016, 10 (05) : 861 - 868
  • [20] Parameter choices for sparse regularization with the l1 norm
    Liu, Qianru
    Wang, Rui
    Xu, Yuesheng
    Yan, Mingsong
    INVERSE PROBLEMS, 2023, 39 (02)