Parallel PSO for Efficient Neural Network Training Using GPGPU and Apache Spark in Edge Computing Sets

被引:2
|
作者
Capel, Manuel I. [1 ]
Salguero-Hidalgo, Alberto [2 ]
Holgado-Terriza, Juan A. [1 ]
机构
[1] Univ Granada, Software Engn Dept, ETSIIT, Granada 18071, Spain
[2] Univ Malaga, Dept Comp Sci & Programming Languages, ETSII, Malaga 29010, Spain
关键词
Apache Spark; classification recall; deep neural networks; GPU parallelism; optimization research; particle swarm optimization (PSO); predictive accuracy; PARTICLE SWARM OPTIMIZATION;
D O I
10.3390/a17090378
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The training phase of a deep learning neural network (DLNN) is a computationally demanding process, particularly for models comprising multiple layers of intermediate neurons.This paper presents a novel approach to accelerating DLNN training using the particle swarm optimisation (PSO) algorithm, which exploits the GPGPU architecture and the Apache Spark analytics engine for large-scale data processing tasks. PSO is a bio-inspired stochastic optimisation method whose objective is to iteratively enhance the solution to a (usually complex) problem by approximating a given objective. The expensive fitness evaluation and updating of particle positions can be supported more effectively by parallel processing. Nevertheless, the parallelisation of an efficient PSO is not a simple process due to the complexity of the computations performed on the swarm of particles and the iterative execution of the algorithm until a solution close to the objective with minimal error is achieved. In this study, two forms of parallelisation have been developed for the PSO algorithm, both of which are designed for execution in a distributed execution environment. The synchronous parallel PSO implementation guarantees consistency but may result in idle time due to global synchronisation. In contrast, the asynchronous parallel PSO approach reduces the necessity for global synchronization, thereby enhancing execution time and making it more appropriate for large datasets and distributed environments such as Apache Spark. The two variants of PSO have been implemented with the objective of distributing the computational load supported by the algorithm across the different executor nodes of the Spark cluster to effectively achieve coarse-grained parallelism. The result is a significant performance improvement over current sequential variants of PSO.
引用
收藏
页数:26
相关论文
共 50 条
  • [1] Neural network training using stochastic PSO
    Chen, Xin
    Li, Yangmin
    NEURAL INFORMATION PROCESSING, PT 2, PROCEEDINGS, 2006, 4233 : 1051 - 1060
  • [2] Construction of an Efficient Divided/Distributed Neural Network Model Using Edge Computing
    Shingai, Ryuta
    Hiraga, Yuria
    Fukuoka, Hisakazu
    Mitani, Takamasa
    Nakada, Takashi
    Nakashima, Yasuhiko
    IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2020, E103D (10) : 2072 - 2082
  • [3] Nodule Detection with Convolutional Neural Network Using Apache Spark and GPU Frameworks
    Venkatesan, Nikitha Johnsirani
    Shin, Dong Ryeol
    Nam, Choon Sung
    APPLIED SCIENCES-BASEL, 2021, 11 (06):
  • [4] Detection DDoS Attacks Based on Neural-Network Using Apache Spark
    Hsieh, Chang-Jung
    Chan, Ting-Yuan
    PROCEEDINGS OF 2016 INTERNATIONAL CONFERENCE ON APPLIED SYSTEM INNOVATION (ICASI), 2016,
  • [5] Efficient Neural Network Acceleration on GPGPU using Content Addressable Memory
    Imani, Mohsen
    Peroni, Daniel
    Kim, Yeseong
    Rahimi, Abbas
    Rosing, Tajana
    PROCEEDINGS OF THE 2017 DESIGN, AUTOMATION & TEST IN EUROPE CONFERENCE & EXHIBITION (DATE), 2017, : 1026 - 1031
  • [6] Efficient Feature Extraction Using Apache Spark for Network Behavior Anomaly Detection
    Xiaoming Ye
    Xingshu Chen
    Dunhu Liu
    Wenxian Wang
    Li Yang
    Gang Liang
    Guolin Shao
    TsinghuaScienceandTechnology, 2018, 23 (05) : 561 - 573
  • [7] Efficient partition of learning data sets for neural network training
    Inst. of Bioorg. and Petrol. Chem., Kiev, Ukraine
    不详
    Neural Netw., 8 (1361-1374):
  • [8] Efficient partition of learning data sets for neural network training
    Tetko, IV
    Villa, AEP
    NEURAL NETWORKS, 1997, 10 (08) : 1361 - 1374
  • [9] Traffic Network Flow Prediction Using Parallel Training for Deep Convolutional Neural Networks on Spark Cloud
    Zhang, Yongnan
    Zhou, Yonghua
    Lu, Huapu
    Fujita, Hamido
    IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, 2020, 16 (12) : 7369 - 7380
  • [10] FedSNN: Training Slimmable Neural Network With Federated Learning in Edge Computing
    Xu, Yang
    Liao, Yunming
    Xu, Hongli
    Wang, Zhiyuan
    Wang, Lun
    Liu, Jianchun
    Qian, Chen
    IEEE-ACM TRANSACTIONS ON NETWORKING, 2024,