An Approach to Hyperparameter Optimization for the Objective Function in Machine Learning

被引:14
|
作者
Kim, Yonghoon [1 ]
Chung, Mokdong [1 ]
机构
[1] Pukyong Natl Univ, Dept Comp Engn, Pusan 48513, South Korea
基金
新加坡国家研究基金会;
关键词
bayesian optimization; gaussian process; learning rate; acauisition function; machine learning; GLOBAL OPTIMIZATION;
D O I
10.3390/electronics8111267
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In machine learning, performance is of great value. However, each learning process requires much time and effort in setting each parameter. The critical problem in machine learning is determining the hyperparameters, such as the learning rate, mini-batch size, and regularization coefficient. In particular, we focus on the learning rate, which is directly related to learning efficiency and performance. Bayesian optimization using a Gaussian Process is common for this purpose. In this paper, based on Bayesian optimization, we attempt to optimize the hyperparameters automatically by utilizing a Gamma distribution, instead of a Gaussian distribution, to improve the training performance of predicting image discrimination. As a result, our proposed method proves to be more reasonable and efficient in the estimation of learning rate when training the data, and can be useful in machine learning.
引用
收藏
页数:19
相关论文
共 50 条
  • [1] Multi-Objective Hyperparameter Optimization in Machine Learning—An Overview
    Karl F.
    Pielok T.
    Moosbauer J.
    Pfisterer F.
    Coors S.
    Binder M.
    Schneider L.
    Thomas J.
    Richter J.
    Lang M.
    Garrido-Merchán E.C.
    Branke J.
    Bischl B.
    ACM. Trans. Evol. Learn. Optim., 2023, 4
  • [2] A survey on multi-objective hyperparameter optimization algorithms for machine learning
    Alejandro Morales-Hernández
    Inneke Van Nieuwenhuyse
    Sebastian Rojas Gonzalez
    Artificial Intelligence Review, 2023, 56 : 8043 - 8093
  • [3] A survey on multi-objective hyperparameter optimization algorithms for machine learning
    Morales-Hernandez, Alejandro
    Van Nieuwenhuyse, Inneke
    Gonzalez, Sebastian Rojas
    ARTIFICIAL INTELLIGENCE REVIEW, 2023, 56 (08) : 8043 - 8093
  • [4] A Multi-objective Hyperparameter Optimization for Machine Learning Using Genetic Algorithms: A Green AI Centric Approach
    Yokoyama, Andre M.
    Ferro, Mariza
    Schulze, Bruno
    ADVANCES IN ARTIFICIAL INTELLIGENCE-IBERAMIA 2022, 2022, 13788 : 133 - 144
  • [5] Multi-objective hyperparameter optimization approach with genetic algorithms towards efficient and environmentally friendly machine learning
    Yokoyama, Andre M.
    Ferro, Mariza
    Schulze, Bruno
    AI COMMUNICATIONS, 2024, 37 (03) : 429 - 442
  • [6] Machine Learning Assisted Hyperparameter Tuning for Optimization
    Linkous, Lauren
    Lundquist, Jonathan
    Suche, Michael
    Topsakal, Erdem
    2024 IEEE INC-USNC-URSI RADIO SCIENCE MEETING (JOINT WITH AP-S SYMPOSIUM), 2024, : 107 - 108
  • [7] Sherpa: Robust hyperparameter optimization for machine learning
    Hertel, Lars
    Collado, Julian
    Sadowski, Peter
    Ott, Jordan
    Baldi, Pierre
    SOFTWAREX, 2020, 12
  • [8] Hyperparameter optimization for machine learning models based on Bayesian optimization
    Wu J.
    Chen X.-Y.
    Zhang H.
    Xiong L.-D.
    Lei H.
    Deng S.-H.
    Journal of Electronic Science and Technology, 2019, 17 (01) : 26 - 40
  • [9] Hyperparameter Optimization for Machine Learning Models Based on Bayesian Optimization
    Jia Wu
    Xiu-Yun Chen
    Hao Zhang
    Li-Dong Xiong
    Hang Lei
    Si-Hao Deng
    Journal of Electronic Science and Technology, 2019, (01) : 26 - 40
  • [10] Hyperparameter Optimization for Machine Learning Models Based on Bayesian Optimization
    Jia Wu
    XiuYun Chen
    Hao Zhang
    LiDong Xiong
    Hang Lei
    SiHao Deng
    Journal of Electronic Science and Technology, 2019, 17 (01) : 26 - 40