Accelerating Graph Neural Network Training on ReRAM-Based PIM Architectures via Graph and Model Pruning

被引:5
|
作者
Ogbogu, Chukwufumnanya O. [1 ]
Arka, Aqeeb Iqbal [1 ]
Pfromm, Lukas [2 ]
Joardar, Biresh Kumar [3 ]
Doppa, Janardhan Rao [1 ]
Chakrabarty, Krishnendu [4 ]
Pande, Partha Pratim [1 ]
机构
[1] Washington State Univ, Sch Elect Engn & Comp Sci, Pullman, WA 99164 USA
[2] Oregon State Univ, Dept Elect & Comp Engn, Corvallis, OR 97331 USA
[3] Univ Houston, Dept Elect & Comp Engn, Houston, TX 77004 USA
[4] Duke Univ, Dept Elect & Comp Engn, Durham, NC 27708 USA
基金
美国国家科学基金会;
关键词
Data compression; graph neural network (GNN); PIM; pruning; resistive random-access memory (ReRAM);
D O I
10.1109/TCAD.2022.3227879
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Graph neural networks (GNNs) are used for predictive analytics on graph-structured data, and they have become very popular in diverse real-world applications. Resistive random-access memory (ReRAM)-based PIM architectures can accelerate GNN training. However, GNN training on ReRAMbased architectures is both compute- and data intensive in nature. In this work, we propose a framework called SlimGNN that synergistically combines both graph and model pruning to accelerate GNN training on ReRAM-based architectures. The proposed framework reduces the amount of redundant information in both the GNN model and input graph(s) to streamline the overall training process. This enables fast and energy-efficient GNN training on ReRAM-based architectures. Experimental results demonstrate that using this framework, we can accelerate GNN training by up to 4.5x while using 6.6x less energy compared to the unpruned counterparts.
引用
收藏
页码:2703 / 2716
页数:14
相关论文
共 50 条
  • [21] A Survey of ReRAM-Based Architectures for Processing-In-Memory and Neural Networks
    Mittal, Sparsh
    MACHINE LEARNING AND KNOWLEDGE EXTRACTION, 2019, 1 (01): : 75 - 114
  • [22] ON THE CHOICE OF GRAPH NEURAL NETWORK ARCHITECTURES
    Vignac, Clement
    Ortiz-Jimenez, Guillermo
    Frossard, Pascal
    2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 2020, : 8489 - 8493
  • [23] A Unified Framework for Training, Mapping and Simulation of ReRAM-Based Convolutional Neural Network Acceleration
    Liu, He
    Han, Jianhui
    Zhang, Youhui
    IEEE COMPUTER ARCHITECTURE LETTERS, 2019, 18 (01) : 63 - 66
  • [24] CraftRGP: A Comprehensive Reliability Analysis Framework Towards ReRAM-Based Graph Processing
    Wei, Xiaohui
    Deng, Jiaguo
    Wang, Xiaonan
    Li, Zongdian
    Jiang, Nan
    Yue, Hengshan
    8TH INTERNATIONAL TEST CONFERENCE IN ASIA, ITC-ASIA 2024, 2024,
  • [25] A Framework for Accelerating Transformer-Based Language Model on ReRAM-Based Architecture
    Kang, Myeonggu
    Shin, Hyein
    Kim, Lee-Sup
    IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, 2022, 41 (09) : 3026 - 3039
  • [26] GraphA: An efficient ReRAM-based architecture to accelerate large scale graph processing
    Ghasemi, Seyed Ali
    Jahannia, Belal
    Farbeh, Hamed
    JOURNAL OF SYSTEMS ARCHITECTURE, 2022, 133
  • [27] Device Modeling Bias in ReRAM-Based Neural Network Simulations
    Yousuf, Osama
    Hossen, Imtiaz
    Daniels, Matthew W.
    Lueker-Boden, Martin
    Dienstfrey, Andrew
    Adam, Gina C.
    IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS, 2023, 13 (01) : 382 - 394
  • [28] Offline Training-Based Mitigation of IR Drop for ReRAM-Based Deep Neural Network Accelerators
    Lee, Sugil
    Fouda, Mohammed E.
    Lee, Jongeun
    Eltawil, Ahmed M.
    Kurdahi, Fadi
    IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, 2023, 42 (02) : 521 - 532
  • [29] Graph Attention Neural Network Distributed Model Training
    Esmaeilzadeh, Armin
    Kambar, Mina Esmail Zadeh Nojoo
    Heidari, Maryam
    2022 IEEE WORLD AI IOT CONGRESS (AIIOT), 2022, : 447 - 452
  • [30] High-Throughput Training of Deep CNNs on ReRAM-Based Heterogeneous Architectures via Optimized Normalization Layers
    Joardar, Biresh Kumar
    Deshwal, Aryan
    Doppa, Janardhan Rao
    Pande, Partha Pratim
    Chakrabarty, Krishnendu
    IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, 2022, 41 (05) : 1537 - 1549