A comprehensive survey on regularization strategies in machine learning

被引:118
|
作者
Tian, Yingjie [1 ,3 ,4 ]
Zhang, Yuqi [2 ,3 ,4 ]
机构
[1] Univ Chinese Acad Sci, Sch Econ & Management, Beijing 100190, Peoples R China
[2] Univ Chinese Acad Sci, Sch Math Sci, Beijing 100049, Peoples R China
[3] Chinese Acad Sci, Res Ctr Fictitious Econ & Data Sci, Beijing 100190, Peoples R China
[4] Chinese Acad Sci, Key Lab Big Data Min & Knowledge Management, Beijing 100190, Peoples R China
基金
中国国家自然科学基金;
关键词
Overfitting; Generalization; Regularization; Machine learning; COVARIANCE-MATRIX ESTIMATION; P-LAPLACIAN REGULARIZATION; FEATURE-SELECTION; SPARSE REGULARIZATION; VARIABLE SELECTION; NEURAL-NETWORKS; ROBUST PCA; IMAGE; REGRESSION; APPROXIMATION;
D O I
10.1016/j.inffus.2021.11.005
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In machine learning, the model is not as complicated as possible. Good generalization ability means that the model not only performs well on the training data set, but also can make good prediction on new data. Regularization imposes a penalty on model's complexity or smoothness, allowing for good generalization to unseen data even when training on a finite training set or with an inadequate iteration. Deep learning has developed rapidly in recent years. Then the regularization has a broader definition: regularization is a technology aimed at improving the generalization ability of a model. This paper gave a comprehensive study and a state-of-the-art review of the regularization strategies in machine learning. Then the characteristics and comparisons of regularizations were presented. In addition, it discussed how to choose a regularization for the specific task. For specific tasks, it is necessary for regularization technology to have good mathematical characteristics. Meanwhile, new regularization techniques can be constructed by extending and combining existing regularization techniques. Finally, it concluded current opportunities and challenges of regularization technologies, as well as many open concerns and research trends.
引用
收藏
页码:146 / 166
页数:21
相关论文
共 50 条
  • [21] A Comprehensive Survey of Artificial Intelligence and Machine Learning Application in Healthcare
    Vishwanatth, Kannan
    Satish, Savitha
    Enagandula, Prasad
    Khade, Anindita A.
    Pratibh
    Mirza, Zainab
    JOURNAL OF ELECTRICAL SYSTEMS, 2024, 20 (06) : 1425 - 1431
  • [22] Machine Learning Approaches for Film Censorship: A Comprehensive Survey of Techniques
    Hatkar, Kaustubh
    Lokhande, Sanket
    Munot, Mousami, V
    Jaiswal, R. C.
    SMART TRENDS IN COMPUTING AND COMMUNICATIONS, VOL 1, SMARTCOM 2024, 2024, 945 : 251 - 262
  • [23] Hyperbox-based machine learning algorithms: a comprehensive survey
    Khuat, Thanh Tung
    Ruta, Dymitr
    Gabrys, Bogdan
    SOFT COMPUTING, 2021, 25 (02) : 1325 - 1363
  • [24] Machine Learning Applications in Sarcopenia Detection and Management: A Comprehensive Survey
    Turimov Mustapoevich, Dilmurod
    Kim, Wooseong
    HEALTHCARE, 2023, 11 (18)
  • [25] Machine Learning for Network Slicing Resource Management:A Comprehensive Survey
    HAN Bin
    Hans D.SCHOTTEN
    ZTE Communications, 2019, 17 (04) : 27 - 32
  • [26] Machine learning for wireless communications in the Internet of Things: A comprehensive survey
    Jagannath, Jithin
    Polosky, Nicholas
    Jagannath, Anu
    Restuccia, Francesco
    Melodia, Tommaso
    AD HOC NETWORKS, 2019, 93
  • [27] Hyperbox-based machine learning algorithms: a comprehensive survey
    Thanh Tung Khuat
    Dymitr Ruta
    Bogdan Gabrys
    Soft Computing, 2021, 25 : 1325 - 1363
  • [28] A comprehensive survey on feature selection in the various fields of machine learning
    Dhal, Pradip
    Azad, Chandrashekhar
    APPLIED INTELLIGENCE, 2022, 52 (04) : 4543 - 4581
  • [29] A Comprehensive Survey of Machine Learning Techniques and Models for Object Detection
    Trigka, Maria
    Dritsas, Elias
    SENSORS, 2025, 25 (01)
  • [30] From distributed machine to distributed deep learning: a comprehensive survey
    Mohammad Dehghani
    Zahra Yazdanparast
    Journal of Big Data, 10