Bayesian Contextual Bandits for Hyper Parameter Optimization

被引：0

作者：

Sui, Guoxin ^{[1
]}

Yu, Yong ^{[1
]}

机构：

[1] Department of Computer Science, Shanghai Jiao Tong University, Shanghai,200240, China

来源：

IEEE Access | 2020年 / 8卷

关键词：

Hyper parameter optimization (HPO) is a crucial step in modern machine learning systems. Bayesian optimization (BO) has shown great promise in HPO; where the parameter evaluation is conducted through a black-box optimization procedure. However; the main drawback of BO lies in the expensive computation cost; which limits its application in large deep models. Hence there is strong motivation to reduce the number of epochs of training required by leveraging the partial information provided by iterative training procedures. Recent advancements use probabilistic models that extrapolate learning curves explicitly to early-stop the training of poor-performing configurations. However; these approaches impose a strong prior assumption on the distribution of learning curves and involve much larger computational complexity or rely heavily on predefined rules; which is not general for different cases. To tackle the challenge of training resource allocation in infinite parameter search space and in time horizon; we study HPO problem in Bayesian contextual bandits setting and derive several fully-dynamic strategies that are information-efficient and scalable. Extensive experiments demonstrate that the proposed method can significantly speed up the HPO process. © 2013 IEEE;

D O I：

暂无

中图分类号：

学科分类号：

摘要：

引用

页码：42971 / 42979

共 50 条

[1] Bayesian Contextual Bandits for Hyper Parameter Optimization
Sui, Guoxin
Yu, Yong
IEEE ACCESS, 2020, 8 : 42971 - 42979
[2] Bayesian Optimization for Accelerating Hyper-parameter Tuning
Vu Nguyen
2019 IEEE SECOND INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND KNOWLEDGE ENGINEERING (AIKE), 2019, : 302 - 305
[3] A Bayesian Approach for Subset Selection in Contextual Bandits
Li, Jialian
Du, Chao
Zhu, Jun
THIRTY-FIFTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THIRTY-THIRD CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE AND THE ELEVENTH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2021, 35 : 8384 - 8391
[4] HyperUCB: Hyperparameter Optimization Using Contextual Bandits
Tavakol, Maryam
Mair, Sebastian
Morik, Katharina
MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES, ECML PKDD 2019, PT I, 2020, 1167 : 44 - 50
[5] Parallel Contextual Bandits in Wireless Handover Optimization
Colin, Igor
Thomas, Albert
Draief, Moez
2018 18TH IEEE INTERNATIONAL CONFERENCE ON DATA MINING WORKSHOPS (ICDMW), 2018, : 258 - 265
[6] Personalized Recommendation via Parameter-Free Contextual Bandits
Tang, Liang
Jiang, Yexi
Li, Lei
Zeng, Chunqiu
Li, Tao
SIGIR 2015: PROCEEDINGS OF THE 38TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, 2015, : 323 - 332
[7] A Parameter-Free Algorithm for Misspecified Linear Contextual Bandits
Takemura, Kei
Ito, Shinji
Hatano, Daisuke
Sumita, Hanna
Fukunaga, Takuro
Kakimura, Naonori
Kawarabayashi, Ken-ichi
24TH INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS (AISTATS), 2021, 130
[8] Offline Contextual Bayesian Optimization
Char, Ian
Chung, Youngseog
Neiswanger, Willie
Kandasamy, Kirthevasan
Nelson, Andrew Oakleigh
Boyer, Mark D.
Kolemen, Egemen
Schneider, Jeff
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019), 2019, 32
[9] Contextual Linear Bandits under Noisy Features: Towards Bayesian Oracles
Kim, Jung-hun
Yun, Se-Young
Jeong, Minchan
Nam, Junhyun
Shin, Jinwoo
Combes, Richard
INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 206, 2023, 206
[10] ONLINE HYPER-PARAMETER TUNING FOR THE CONTEXTUAL BANDIT
Bouneffouf, Djallel
Claeys, Emmanuelle
2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021), 2021, : 3445 - 3449

← 1 2 3 4 5 →