Parallel PSO for Efficient Neural Network Training Using GPGPU and Apache Spark in Edge Computing Sets

被引:2
|
作者
Capel, Manuel I. [1 ]
Salguero-Hidalgo, Alberto [2 ]
Holgado-Terriza, Juan A. [1 ]
机构
[1] Univ Granada, Software Engn Dept, ETSIIT, Granada 18071, Spain
[2] Univ Malaga, Dept Comp Sci & Programming Languages, ETSII, Malaga 29010, Spain
关键词
Apache Spark; classification recall; deep neural networks; GPU parallelism; optimization research; particle swarm optimization (PSO); predictive accuracy; PARTICLE SWARM OPTIMIZATION;
D O I
10.3390/a17090378
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The training phase of a deep learning neural network (DLNN) is a computationally demanding process, particularly for models comprising multiple layers of intermediate neurons.This paper presents a novel approach to accelerating DLNN training using the particle swarm optimisation (PSO) algorithm, which exploits the GPGPU architecture and the Apache Spark analytics engine for large-scale data processing tasks. PSO is a bio-inspired stochastic optimisation method whose objective is to iteratively enhance the solution to a (usually complex) problem by approximating a given objective. The expensive fitness evaluation and updating of particle positions can be supported more effectively by parallel processing. Nevertheless, the parallelisation of an efficient PSO is not a simple process due to the complexity of the computations performed on the swarm of particles and the iterative execution of the algorithm until a solution close to the objective with minimal error is achieved. In this study, two forms of parallelisation have been developed for the PSO algorithm, both of which are designed for execution in a distributed execution environment. The synchronous parallel PSO implementation guarantees consistency but may result in idle time due to global synchronisation. In contrast, the asynchronous parallel PSO approach reduces the necessity for global synchronization, thereby enhancing execution time and making it more appropriate for large datasets and distributed environments such as Apache Spark. The two variants of PSO have been implemented with the objective of distributing the computational load supported by the algorithm across the different executor nodes of the Spark cluster to effectively achieve coarse-grained parallelism. The result is a significant performance improvement over current sequential variants of PSO.
引用
收藏
页数:26
相关论文
共 50 条
  • [21] Efficient FPGA-Based Convolutional Neural Network Implementation for Edge Computing
    Cuong, Pham-Quoc
    Thinh, Tran Ngoc
    JOURNAL OF ADVANCES IN INFORMATION TECHNOLOGY, 2023, 14 (03) : 479 - 487
  • [22] Fully-Parallel Area-Efficient Deep Neural Network Design Using Stochastic Computing
    Xie, Yi
    Liao, Siyu
    Yuan, Bo
    Wang, Yanzhi
    Wang, Zhongfeng
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS, 2017, 64 (12) : 1382 - 1386
  • [23] Typhoon Quantitative Rainfall Prediction from Big Data Analytics by Using the Apache Hadoop Spark Parallel Computing Framework
    Wei, Chih-Chiang
    Chou, Tzu-Hao
    ATMOSPHERE, 2020, 11 (08)
  • [24] EFFICIENT NEURAL NETWORK TRAINING USING CURVELET FEATURES
    Hafiz, Abdul Rahman
    Al-Marzouqi, Hasan
    2016 IEEE 12TH IMAGE, VIDEO, AND MULTIDIMENSIONAL SIGNAL PROCESSING WORKSHOP (IVMSP), 2016,
  • [25] RETRACTED: Efficient Feature Extraction Using Apache Spark for Network Behavior Anomaly Detection (Retracted Article)
    Ye, Xiaoming
    Chen, Xingshu
    Liu, Dunhu
    Wang, Wenxian
    Yang, Li
    Liang, Gang
    Shao, Guolin
    TSINGHUA SCIENCE AND TECHNOLOGY, 2018, 23 (05) : 561 - 573
  • [26] Efficient Collaborative Edge Computing for Vehicular Network Using Clustering Service
    Al-Allawee, Ali
    Lorenz, Pascal
    Munther, Alhamza
    NETWORK, 2024, 4 (03): : 390 - 403
  • [27] Efficient spiking neural network training and inference with reduced precision memory and computing
    Wang, Yi
    Shahbazi, Karim
    Zhang, Hao
    Oh, Kwang-Il
    Lee, Jae-Jin
    Ko, Seok-Bum
    IET COMPUTERS AND DIGITAL TECHNIQUES, 2019, 13 (05): : 397 - 404
  • [28] Poster: Selection of Optimal Neural Model using Spiking Neural Network for Edge Computing
    Sanaullah
    Roy, Kaushik
    Ruckert, Ulrich
    Jungeblut, Thorsten
    2024 IEEE 44TH INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING SYSTEMS, ICDCS 2024, 2024, : 1452 - 1453
  • [29] System performance and optimization in NOMA mobile edge computing surveillance network using GA and PSO
    Truong, Truong Van
    Nayyar, Anand
    COMPUTER NETWORKS, 2023, 223
  • [30] A Memory-Efficient Hybrid Parallel Framework for Deep Neural Network Training
    Li, Dongsheng
    Li, Shengwei
    Lai, Zhiquan
    Fu, Yongquan
    Ye, Xiangyu
    Cai, Lei
    Qiao, Linbo
    IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2024, 35 (04) : 577 - 591