Automated Image Captioning Using Sparrow Search Algorithm With Improved Deep Learning Model

被引:0
|
作者
Arasi, Munya A. [1 ]
Alshahrani, Haya Mesfer [2 ]
Alruwais, Nuha [3 ]
Motwakel, Abdelwahed [4 ]
Ahmed, Noura Abdelaziz [5 ]
Mohamed, Abdullah [6 ]
机构
[1] King Khalid Univ, Coll Sci & Arts Rijal Almaa, Dept Comp Sci, Abha 62529, Saudi Arabia
[2] Princess Nourah Bint Abdulrahman Univ, Coll Comp & Informat Sci, Dept Informat Syst, POB 84428, Riyadh 11671, Saudi Arabia
[3] King Saud Univ, Coll Appl Studies & Community Serv, Dept Comp Sci & Engn, POB 22459, Riyadh 11495, Saudi Arabia
[4] Prince Sattam Bin Abdulaziz Univ, Coll Business Adm Hawtat Bani Tamim, Dept Management Informat Syst, Al Kharj 11942, Saudi Arabia
[5] Prince Sattam Bin Abdulaziz Univ, Dept Comp & Self Dev, Preparatory Year Deanship, Al Kharj 11942, Saudi Arabia
[6] Future Univ Egypt, Res Ctr, New Cairo 11845, Egypt
关键词
Convolutional neural networks; Visualization; Feature extraction; Convolution; Deep learning; Natural language processing; Computational modeling; Image capture; Search methods; Image captioning; deep learning; natural language processing; sparrow search algorithm; computer vision;
D O I
10.1109/ACCESS.2023.3317276
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Image captioning is a deep learning technique that intends to create and generate textual descriptions or captions for images. It integrates computer vision and natural language processing (NLP) to comprehend the visual content of an image and generate human-like descriptions. Deep learning (DL) based image captioning models can be trained on large-scale datasets, allowing them to generalize various types of images and generate captions that apply to a wide range of visual scenarios. By combining computer vision and natural language processing, DL-enabled image captioning models can understand both visual and textual information, which enables them to generate captions that not only describe the visual content but also incorporate contextual and semantic information. This study develops an Automated Image Captioning using Sparrow Search Algorithm with Improved Deep Learning (AIC-SSAIDL) technique. The major intention of the AIC-SSAIDL technique lies in the automated generation of textual captions for the input images. To accomplish this, the AIC-SSAIDL technique utilizes the MobileNetv2 model to generate feature descriptors of the input images and its hyperparameter tuning process takes place using SSA. For the image captioning process, the AIC-SSAIDL technique utilizes an attention mechanism with long short-term memory (AM-LSTM) network. Finally, the hyperparameter selection of the AM-LSTM model is performed by the fruit fly optimization (FFO) algorithm. A wide range of experiments has been conducted on benchmark data to depict the better performance of the AIC-SSAIDL method. The comprehensive result analysis highlighted the enhanced captioning results of the AIC-SSAIDL method with maximum CIDEr of 46.12, 61.89, and 137.45 on Flickr8k, Flickr30k, and MSCOCO datasets, respectively.
引用
收藏
页码:104633 / 104642
页数:10
相关论文
共 50 条
  • [1] Modeling of Hyperparameter Tuned Deep Learning Model for Automated Image Captioning
    Omri, Mohamed
    Abdel-Khalek, Sayed
    Khalil, Eied M.
    Bouslimi, Jamel
    Joshi, Gyanendra Prasad
    MATHEMATICS, 2022, 10 (03)
  • [2] Brain Tumor Diagnosis Using Sparrow Search Algorithm Based Deep Learning Model
    Rajathi, G. Ignisha
    Kumar, R. Ramesh
    Ravikumar, D.
    Joel, T.
    Kadry, Seifedine
    Jeong, Chang-Won
    Nam, Yunyoung
    COMPUTER SYSTEMS SCIENCE AND ENGINEERING, 2023, 44 (02): : 1793 - 1806
  • [3] Image Captioning Using Deep Learning
    Adithya, Paluvayi Veera
    Kalidindi, Mourya Viswanadh
    Swaroop, Nallani Jyothi
    Vishwas, H. N.
    ADVANCED NETWORK TECHNOLOGIES AND INTELLIGENT COMPUTING, ANTIC 2023, PT III, 2024, 2092 : 42 - 58
  • [4] Image Captioning using Deep Learning
    Jain, Yukti Sanjay
    Dhopeshwar, Tanisha
    Chadha, Supreet Kaur
    Pagire, Vrushali
    2021 INTERNATIONAL CONFERENCE ON COMPUTATIONAL PERFORMANCE EVALUATION (COMPE-2021), 2021,
  • [5] Prediction tool wear using improved deep extreme learning machines based on the sparrow search algorithm
    Zhou, Wenjun
    Xiao, Xiaoping
    Li, Zisheng
    Zhang, Kai
    He, Ruide
    MEASUREMENT SCIENCE AND TECHNOLOGY, 2024, 35 (04)
  • [6] Deep kernel extreme learning machine classifier based on the improved sparrow search algorithm
    Zhao Guangyuan
    Lei Yu
    The Journal of China Universities of Posts and Telecommunications, 2024, (03) : 15 - 29
  • [7] Deep kernel extreme learning machine classifier based on the improved sparrow search algorithm
    Guangyuan, Zhao
    Yu, Lei
    Journal of China Universities of Posts and Telecommunications, 2024, 31 (03): : 15 - 29
  • [8] Threshold image segmentation based on improved sparrow search algorithm
    Dongmei Wu
    Chengzhi Yuan
    Multimedia Tools and Applications, 2022, 81 : 33513 - 33546
  • [9] An Improved Sparrow Search Algorithm
    Song, Wei
    Liu, Song
    Wang, Xiaochun
    Wu, Weiguo
    2020 IEEE INTL SYMP ON PARALLEL & DISTRIBUTED PROCESSING WITH APPLICATIONS, INTL CONF ON BIG DATA & CLOUD COMPUTING, INTL SYMP SOCIAL COMPUTING & NETWORKING, INTL CONF ON SUSTAINABLE COMPUTING & COMMUNICATIONS (ISPA/BDCLOUD/SOCIALCOM/SUSTAINCOM 2020), 2020, : 537 - 543
  • [10] A reference-based model using deep learning for image captioning
    Tiago do Carmo Nogueira
    Cássio Dener Noronha Vinhal
    Gélson da Cruz Júnior
    Matheus Rudolfo Diedrich Ullmann
    Thyago Carvalho Marques
    Multimedia Systems, 2023, 29 : 1665 - 1681