An Approach to Hyperparameter Optimization for the Objective Function in Machine Learning

被引:14
|
作者
Kim, Yonghoon [1 ]
Chung, Mokdong [1 ]
机构
[1] Pukyong Natl Univ, Dept Comp Engn, Pusan 48513, South Korea
基金
新加坡国家研究基金会;
关键词
bayesian optimization; gaussian process; learning rate; acauisition function; machine learning; GLOBAL OPTIMIZATION;
D O I
10.3390/electronics8111267
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In machine learning, performance is of great value. However, each learning process requires much time and effort in setting each parameter. The critical problem in machine learning is determining the hyperparameters, such as the learning rate, mini-batch size, and regularization coefficient. In particular, we focus on the learning rate, which is directly related to learning efficiency and performance. Bayesian optimization using a Gaussian Process is common for this purpose. In this paper, based on Bayesian optimization, we attempt to optimize the hyperparameters automatically by utilizing a Gamma distribution, instead of a Gaussian distribution, to improve the training performance of predicting image discrimination. As a result, our proposed method proves to be more reasonable and efficient in the estimation of learning rate when training the data, and can be useful in machine learning.
引用
收藏
页数:19
相关论文
共 50 条
  • [21] Machine Learning Approach for Identification of Objective Function in Production Scheduling Problems
    Matsuoka, Yoki
    Nishi, Tatsushi
    Tierney, Kevin
    2019 IEEE 15TH INTERNATIONAL CONFERENCE ON AUTOMATION SCIENCE AND ENGINEERING (CASE), 2019, : 679 - 684
  • [22] OptABC: an Optimal Hyperparameter Tuning Approach for Machine Learning Algorithms
    Zahedi, Leila
    Mohammadi, Farid Ghareh
    Amini, M. Hadi
    20TH IEEE INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND APPLICATIONS (ICMLA 2021), 2021, : 1138 - 1145
  • [23] Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning
    Giovanelli, Joseph
    Tornede, Alexander
    Tornede, Tanja
    Lindauer, Marius
    THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 11, 2024, : 12172 - 12180
  • [24] EMORL: Effective multi-objective reinforcement learning method for hyperparameter optimization
    Chen, SenPeng
    Wu, Jia
    Liu, XiYuan
    ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2021, 104
  • [25] Rainfall Forecasting with Hybrid and Machine Learning Models Based on Hyperparameter Optimization
    Ersoy, Mevlut
    Keskin, M. Erol
    Guerfidan, Remzi
    JOURNAL OF HYDROLOGIC ENGINEERING, 2023, 28 (11)
  • [26] MetaGen: A framework for metaheuristic development and hyperparameter optimization in machine and deep learning
    Gutierrez-aviles, David
    Jimenez-navarro, Manuel Jesus
    Torres, Jose Francisco
    Martinez-Alvarez, Francisco
    NEUROCOMPUTING, 2025, 637
  • [27] Cross-platform hyperparameter optimization for machine learning interatomic potentials
    du Toit, Daniel Thomas F.
    Deringer, Volker L.
    JOURNAL OF CHEMICAL PHYSICS, 2023, 159 (02):
  • [28] Bayesian Hyperparameter Optimization and Ensemble Learning for Machine Learning Models on Software Effort Estimation
    Marco, Robert
    Ahmad, Sakinah Sharifah Syed
    Ahmad, Sabrina
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2022, 13 (03) : 419 - 429
  • [29] Hyperparameter optimization in learning systems
    Andonie, Razvan
    JOURNAL OF MEMBRANE COMPUTING, 2019, 1 (04) : 279 - 291
  • [30] Learning Hyperparameter Optimization Initializations
    Wistuba, Martin
    Schilling, Nicolas
    Schmidt-Thieme, Lars
    PROCEEDINGS OF THE 2015 IEEE INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (IEEE DSAA 2015), 2015, : 339 - 348