A new algorithm for support vector regression with automatic selection of hyperparameters

被引:27
|
作者
Wang, You-Gan
Wu, Jinran [2 ]
Hu, Zhi-Hua [3 ]
McLachlan, Geoffrey J. [1 ,4 ]
机构
[1] Australian Catholic Univ, Brisbane, Qld 4000, Australia
[2] Queensland Univ Technol, Brisbane, Qld 4001, Australia
[3] Shanghai Maritime Univ, Shanghai 201306, Peoples R China
[4] Univ Queensland, St Lucia, Qld 4072, Australia
基金
澳大利亚研究理事会;
关键词
Automatic selection; Loss functions; Noise models; Parameter estimation; Probability regularization; GAS SOLUBILITY OPTIMIZATION; PARAMETER SELECTION;
D O I
10.1016/j.patcog.2022.108989
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The hyperparameters in support vector regression (SVR) determine the effectiveness of the support vectors with fitting and predictions. However, the choice of these hyperparameters has always been challenging in both theory and practice. The v-support vector regression eliminates the need to specify an is an element of value elegantly, but at the cost of specifying or postulating a v value. We propose an extended primal objective function arising from probability regularization leading to an automatic selection of is an element of, and we can express v as an explicit function of is an element of. The resultant hyperparameter values can be interpreted as 'working' values required only in training but not testing or prediction. This regularized algorithm, namely is an element of*-SVR, automatically provides a data-dependent is an element of and is found to have a close connection to the v-support vector regression in the sense that v as a fraction is a sensible function of is an element of. The is an element of*- SVR automatically selects both v and is an element of values. We illustrate these findings with some public benchmark datasets.(C) 2022 Elsevier Ltd. All rights reserved.
引用
收藏
页数:9
相关论文
共 50 条
  • [1] Adaptive Chaotic Cultural Algorithm for Hyperparameters Selection of Support Vector Regression
    Cheng, Jian
    Qian, Jiansheng
    Guo, Yi-nan
    EMERGING INTELLIGENT COMPUTING TECHNOLOGY AND APPLICATIONS: WITH ASPECTS OF ARTIFICIAL INTELLIGENCE, 2009, 5755 : 286 - 293
  • [2] A hyperparameters selection technique for support vector regression models
    Tsirikoglou, P.
    Abraham, S.
    Contino, F.
    Lacor, C.
    Ghorbaniasl, G.
    APPLIED SOFT COMPUTING, 2017, 61 : 139 - 148
  • [3] Heuristic algorithm for tuning hyperparameters in support vector regression
    Liu, Jing-Xu
    Cai, Huai-Ping
    Tan, Yue-Jin
    Xitong Fangzhen Xuebao / Journal of System Simulation, 2007, 19 (07): : 1540 - 1543
  • [4] Improving grasshopper optimization algorithm for hyperparameters estimation and feature selection in support vector regression
    Algamal, Zakariya Yahya
    Qasim, Maimoonah Khalid
    Lee, Muhammad Hisyam
    Ali, Haithem Taha Mohammad
    CHEMOMETRICS AND INTELLIGENT LABORATORY SYSTEMS, 2021, 208
  • [5] Hyperparameters optimization of support vector regression using black hole algorithm
    Alrefaee, Saifuldeen Dheyauldeen
    Al Bakal, Salih Muayad
    Algamal, Zakariya Yahya
    INTERNATIONAL JOURNAL OF NONLINEAR ANALYSIS AND APPLICATIONS, 2022, 13 (01): : 3441 - 3450
  • [6] A novel approach for the hyperparameters of support vector regression
    Jeng, JT
    Chuang, CC
    PROCEEDING OF THE 2002 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, VOLS 1-3, 2002, : 642 - 647
  • [7] Hybrid approach of selecting hyperparameters of support vector machine for regression
    Jeng, Jin-Tsong
    IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS, 2006, 36 (03): : 699 - 709
  • [8] Feature selection for support vector regression using a genetic algorithm
    Mckearnan, Shannon B.
    Vock, David M.
    Marai, G. Elisabeta
    Canahuate, Guadalupe
    Fuller, Clifton D.
    Wolfson, Julian
    BIOSTATISTICS, 2023, 24 (02) : 295 - 308
  • [9] Fast grid search: A grid search-inspired algorithm for optimizing hyperparameters of support vector regression
    Acikkar, Mustafa
    TURKISH JOURNAL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCES, 2024, 32 (01) : 68 - 92
  • [10] Shrinking the tube:: A new support vector regression algorithm
    Schölkopf, B
    Bartlett, P
    Smola, A
    Williamson, R
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 11, 1999, 11 : 330 - 336