Generalization Error Bounds for Multiclass Sparse Linear Classifiers

被引:0
|
作者
Levy, Tomer [1 ]
Abramovich, Felix [1 ]
机构
[1] Tel Aviv Univ, Dept Stat & Operat Res, Tel Aviv, Israel
基金
以色列科学基金会;
关键词
Feature selection; high-dimensionality; minimaxity; misclassification excess risk; sparsity; HIGH-DIMENSIONAL CLASSIFICATION; MODELS; SLOPE; REGULARIZATION; CONSISTENCY; SELECTION; LASSO;
D O I
暂无
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
We consider high-dimensional multiclass classification by sparse multinomial logistic regression. Unlike binary classification, in the multiclass setup one can think about an entire spectrum of possible notions of sparsity associated with different structural assumptions on the regression coefficients matrix. We propose a computationally feasible feature selection procedure based on penalized maximum likelihood with convex penalties capturing a specific type of sparsity at hand. In particular, we consider global row-wise sparsity, double row-wise sparsity, and low-rank sparsity, and show that with the properly chosen tuning parameters the derived plug-in classifiers attain the minimax generalization error bounds (in terms of misclassification excess risk) within the corresponding classes of multiclass sparse linear classifiers. The developed approach is general and can be adapted to other types of sparsity as well.
引用
收藏
页数:35
相关论文
共 50 条
  • [31] Using Unsupervised Analysis to Constrain Generalization Bounds for Support Vector Classifiers
    Decherchi, Sergio
    Ridella, Sandro
    Zunino, Rodolfo
    Gastaldo, Paolo
    Anguita, Davide
    IEEE TRANSACTIONS ON NEURAL NETWORKS, 2010, 21 (03): : 424 - 438
  • [32] Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Models
    Wu, David X.
    Sahai, Anant
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [33] Empirical margin distributions and bounding the generalization error of combined classifiers
    Koltchinskii, V
    Panchenko, D
    ANNALS OF STATISTICS, 2002, 30 (01): : 1 - 50
  • [34] Fast Rate Generalization Error Bounds: Variations on a Theme
    Wu, Xuetong
    Manton, Jonathan H.
    Aickelin, Uwe
    Zhu, Jingge
    2022 IEEE INFORMATION THEORY WORKSHOP (ITW), 2022, : 43 - 48
  • [35] A unified generalization of some quadrature rules and error bounds
    Liu, Wenjun
    Jiang, Yong
    Tuna, Adnan
    APPLIED MATHEMATICS AND COMPUTATION, 2013, 219 (09) : 4765 - 4774
  • [36] Tightening Mutual Information Based Bounds on Generalization Error
    Bu, Yuheng
    Zou, Shaofeng
    Veeravalli, Venugopal V.
    2019 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT), 2019, : 587 - 591
  • [37] Strengthened Information-theoretic Bounds on the Generalization Error
    Issa, Ibrahim
    Esposito, Amedeo Roberto
    Gastpar, Michael
    2019 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT), 2019, : 582 - 586
  • [38] Generalization Error Bounds on Deep Learning with Markov Datasets
    Truong, Lan V.
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35, NEURIPS 2022, 2022,
  • [39] Lower Bounds on the Generalization Error of Nonlinear Learning Models
    Seroussi, Inbar
    Zeitouni, Ofer
    IEEE TRANSACTIONS ON INFORMATION THEORY, 2022, 68 (12) : 7956 - 7970
  • [40] Generalization Error Bounds for Optimization Algorithms via Stability
    Meng, Qi
    Wang, Yue
    Chen, Wei
    Wang, Taifeng
    Ma, Zhi-Ming
    Liu, Tie-Yan
    THIRTY-FIRST AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2017, : 2336 - 2342