Optimizing the Parameters of Long Short-Term Memory Networks Using the Bees Algorithm

被引:7
|
作者
Alamri, Nawaf Mohammad H. [1 ]
Packianather, Michael [1 ]
Bigot, Samuel [1 ]
机构
[1] Cardiff Univ, Sch Engn, Queens Bldg,14-17 Parade, Cardiff CF24 3AA, Wales
来源
APPLIED SCIENCES-BASEL | 2023年 / 13卷 / 04期
关键词
Deep Learning (DL); Long Short-Term Memory (LSTM); Convolutional Neural Network (CNN); Bees Algorithm (BA); Bees Algorithm Convolutional Neural Network Long Short-Term Memory (BA-CNN-LSTM); NEURAL-NETWORKS;
D O I
10.3390/app13042536
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
Improving the performance of Deep Learning (DL) algorithms is a challenging problem. However, DL is applied to different types of Deep Neural Networks, and Long Short-Term Memory (LSTM) is one of them that deals with time series or sequential data. This paper attempts to overcome this problem by optimizing LSTM parameters using the Bees Algorithm (BA), which is a nature-inspired algorithm that mimics the foraging behavior of honey bees. In particular, it was used to optimize the adjustment factors of the learning rate in the forget, input, and output gates, in addition to cell candidate, in both forward and backward sides. Furthermore, the BA was used to optimize the learning rate factor in the fully connected layer. In this study, artificial porosity images were used for testing the algorithms; since the input data were images, a Convolutional Neural Network (CNN) was added in order to extract the features in the images to feed into the LSTM for predicting the percentage of porosity in the sequential layers of artificial porosity images that mimic real CT scan images of products manufactured by the Selective Laser Melting (SLM) process. Applying a Convolutional Neural Network Long Short-Term Memory (CNN-LSTM) yielded a porosity prediction accuracy of 93.17%. Although using Bayesian Optimization (BO) to optimize the LSTM parameters mentioned previously did not improve the performance of the LSTM, as the prediction accuracy was 93%, adding the BA to optimize the same LSTM parameters did improve its performance in predicting the porosity, with an accuracy of 95.17% where a hybrid Bees Algorithm Convolutional Neural Network Long Short-Term Memory (BA-CNN-LSTM) was used. Furthermore, the hybrid BA-CNN-LSTM algorithm was capable of dealing with classification problems as well. This was shown by applying it to Electrocardiogram (ECG) benchmark images, which improved the test set classification accuracy, which was 92.50% for the CNN-LSTM algorithm and 95% for both the BO-CNN-LSTM and BA-CNN-LSTM algorithms. In addition, the turbofan engine degradation simulation numerical dataset was used to predict the Remaining Useful Life (RUL) of the engines using the LSTM network. A CNN was not needed in this case, as there was no feature extraction for the images. However, adding the BA to optimize the LSTM parameters improved the prediction accuracy in the testing set for the LSTM and BO-LSTM, which increased from 74% to 77% for the hybrid BA-LSTM algorithm.
引用
收藏
页数:27
相关论文
共 50 条
  • [31] Anomaly Detection for Controller Area Networks Using Long Short-Term Memory
    Tanksale, Vinayak
    IEEE OPEN JOURNAL OF INTELLIGENT TRANSPORTATION SYSTEMS, 2020, 1 : 253 - 265
  • [32] Video captioning using boosted and parallel Long Short-Term Memory networks
    Nabati, Masoomeh
    Behrad, Alireza
    COMPUTER VISION AND IMAGE UNDERSTANDING, 2020, 190
  • [33] Air Quality Forecasting in Madrid Using Long Short-Term Memory Networks
    Pardo, Esteban
    Malpica, Norberto
    BIOMEDICAL APPLICATIONS BASED ON NATURAL AND ARTIFICIAL COMPUTING, PT II, 2017, 10338 : 232 - 239
  • [34] Tailings Pond Risk Prediction Using Long Short-Term Memory Networks
    Li, Jianwei
    Chen, Haoyu
    Zhou, Ting
    Li, Xiaowen
    IEEE ACCESS, 2019, 7 : 182527 - 182537
  • [35] An Incremental Learning Approach Using Long Short-Term Memory Neural Networks
    Álvaro C. Lemos Neto
    Rodrigo A. Coelho
    Cristiano L. de Castro
    Journal of Control, Automation and Electrical Systems, 2022, 33 : 1457 - 1465
  • [36] Quantifying the nativeness of antibody sequences using long short-term memory networks
    Wollacott, Andrew M.
    Xue, Chonghua
    Qin, Qiuyuan
    Hua, June
    Bohnuud, Tanggis
    Viswanathan, Karthik
    Kolachalama, Vijaya B.
    PROTEIN ENGINEERING DESIGN & SELECTION, 2019, 32 (07): : 347 - 354
  • [37] An Incremental Learning Approach Using Long Short-Term Memory Neural Networks
    Lemos Neto, Alvaro C.
    Coelho, Rodrigo A.
    de Castro, Cristiano L.
    JOURNAL OF CONTROL AUTOMATION AND ELECTRICAL SYSTEMS, 2022, 33 (05) : 1457 - 1465
  • [38] Inspection route prediction in substation using long short-term memory networks
    Yang, Yingyi
    Yang, Fan
    Wu, Hao
    COMPUTERS & ELECTRICAL ENGINEERING, 2022, 98
  • [39] Location Prediction of Sperm Cells Using Long Short-Term Memory Networks
    Noy, Lioz
    Barnea, Itay
    Dudaie, Matan
    Kamber, Dotan
    Levi, Mattan
    Shaked, Natan T.
    ADVANCED INTELLIGENT SYSTEMS, 2023, 5 (09)
  • [40] Spectrum Usage Analysis And Prediction using Long Short-Term Memory Networks
    Ghosh, Anneswa
    Van der Merwe, Jacobus
    Kasera, Sneha Kumar
    PROCEEDINGS OF THE 24TH INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING AND NETWORKING, ICDCN 2023, 2023, : 270 - 279