Optimizing the Parameters of Long Short-Term Memory Networks Using the Bees Algorithm

被引:7
|
作者
Alamri, Nawaf Mohammad H. [1 ]
Packianather, Michael [1 ]
Bigot, Samuel [1 ]
机构
[1] Cardiff Univ, Sch Engn, Queens Bldg,14-17 Parade, Cardiff CF24 3AA, Wales
来源
APPLIED SCIENCES-BASEL | 2023年 / 13卷 / 04期
关键词
Deep Learning (DL); Long Short-Term Memory (LSTM); Convolutional Neural Network (CNN); Bees Algorithm (BA); Bees Algorithm Convolutional Neural Network Long Short-Term Memory (BA-CNN-LSTM); NEURAL-NETWORKS;
D O I
10.3390/app13042536
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
Improving the performance of Deep Learning (DL) algorithms is a challenging problem. However, DL is applied to different types of Deep Neural Networks, and Long Short-Term Memory (LSTM) is one of them that deals with time series or sequential data. This paper attempts to overcome this problem by optimizing LSTM parameters using the Bees Algorithm (BA), which is a nature-inspired algorithm that mimics the foraging behavior of honey bees. In particular, it was used to optimize the adjustment factors of the learning rate in the forget, input, and output gates, in addition to cell candidate, in both forward and backward sides. Furthermore, the BA was used to optimize the learning rate factor in the fully connected layer. In this study, artificial porosity images were used for testing the algorithms; since the input data were images, a Convolutional Neural Network (CNN) was added in order to extract the features in the images to feed into the LSTM for predicting the percentage of porosity in the sequential layers of artificial porosity images that mimic real CT scan images of products manufactured by the Selective Laser Melting (SLM) process. Applying a Convolutional Neural Network Long Short-Term Memory (CNN-LSTM) yielded a porosity prediction accuracy of 93.17%. Although using Bayesian Optimization (BO) to optimize the LSTM parameters mentioned previously did not improve the performance of the LSTM, as the prediction accuracy was 93%, adding the BA to optimize the same LSTM parameters did improve its performance in predicting the porosity, with an accuracy of 95.17% where a hybrid Bees Algorithm Convolutional Neural Network Long Short-Term Memory (BA-CNN-LSTM) was used. Furthermore, the hybrid BA-CNN-LSTM algorithm was capable of dealing with classification problems as well. This was shown by applying it to Electrocardiogram (ECG) benchmark images, which improved the test set classification accuracy, which was 92.50% for the CNN-LSTM algorithm and 95% for both the BO-CNN-LSTM and BA-CNN-LSTM algorithms. In addition, the turbofan engine degradation simulation numerical dataset was used to predict the Remaining Useful Life (RUL) of the engines using the LSTM network. A CNN was not needed in this case, as there was no feature extraction for the images. However, adding the BA to optimize the LSTM parameters improved the prediction accuracy in the testing set for the LSTM and BO-LSTM, which increased from 74% to 77% for the hybrid BA-LSTM algorithm.
引用
收藏
页数:27
相关论文
共 50 条
  • [41] Categorisation of power quality problems using long short-term memory networks
    Abdelsalam, Abdelazeem A.
    Hassanin, Ahmed M.
    Hasanien, Hany M.
    IET GENERATION TRANSMISSION & DISTRIBUTION, 2021, 15 (10) : 1626 - 1639
  • [42] Classification of Drug Prescribing Information Using Long Short-Term Memory Networks
    Liu, Tianen
    Khuri, Natalia
    36TH ANNUAL ACM SYMPOSIUM ON APPLIED COMPUTING, SAC 2021, 2021, : 1086 - 1089
  • [43] Intelligent Islanding Detection of Microgrids Using Long Short-Term Memory Networks
    Bukhari, Syed Basit Ali
    Mehmood, Khawaja Khalid
    Wadood, Abdul
    Park, Herie
    ENERGIES, 2021, 14 (18)
  • [44] Aircraft Trajectory Prediction Using Deep Long Short-Term Memory Networks
    Zhao, Ziyu
    Zeng, Weili
    Quan, Zhibin
    Chen, Mengfei
    Yang, Zhao
    CICTP 2019: TRANSPORTATION IN CHINA-CONNECTING THE WORLD, 2019, : 124 - 135
  • [45] Sepsis Deterioration Prediction Using Channelled Long Short-Term Memory Networks
    Svenson, Peter
    Haralabopoulos, Giannis
    Torres, Mercedes Torres
    ARTIFICIAL INTELLIGENCE IN MEDICINE (AIME 2020), 2020, : 359 - 370
  • [46] Predicting Marimba Stickings Using Long Short-Term Memory Neural Networks
    Chong, Jet Kye
    Correa, Debora
    AI 2022: ADVANCES IN ARTIFICIAL INTELLIGENCE, 2022, 13728 : 339 - 352
  • [47] SPOKEN LANGUAGE UNDERSTANDING USING LONG SHORT-TERM MEMORY NEURAL NETWORKS
    Yao, Kaisheng
    Peng, Baolin
    Zhang, Yu
    Yu, Dong
    Zweig, Geoffrey
    Shi, Yangyang
    2014 IEEE WORKSHOP ON SPOKEN LANGUAGE TECHNOLOGY SLT 2014, 2014, : 189 - 194
  • [48] Damage Detection in a Benchmark Structure Using Long Short-term Memory Networks
    Lin, Zhiwei
    Liu, Yonggui
    Zhou, Linren
    2019 CHINESE AUTOMATION CONGRESS (CAC2019), 2019, : 2300 - 2305
  • [49] Short-term traffic travel time forecasting using ensemble approach based on long short-term memory networks
    Jia, Xingli
    Zhou, Wuxiao
    Yang, Hongzhi
    Li, Shuangqing
    Chen, Xingpeng
    IET INTELLIGENT TRANSPORT SYSTEMS, 2023, 17 (06) : 1262 - 1273
  • [50] ANALYSIS AND COMPARISON OF LONG SHORT-TERM MEMORY NETWORKS SHORT-TERM TRAFFIC PREDICTION PERFORMANCE
    Dogan, Erdem
    SCIENTIFIC JOURNAL OF SILESIAN UNIVERSITY OF TECHNOLOGY-SERIES TRANSPORT, 2020, 107 : 19 - 32