A progressive surrogate gradient learning for memristive spiking neural network

被引:0
|
作者
Wang, Shu [1 ]
Chen, Tao [1 ]
Gong, Yu [1 ]
Sun, Fan [1 ]
Shen, Si-Yuan [1 ]
Duan, Shu-Kai [1 ]
Wang, Li-Dan [1 ]
机构
[1] Southwest Univ, Coll Artificial Intelligence, Chongqing 400715, Peoples R China
关键词
spiking neural network; surrogate gradient; supervised learning; memristor cross array; BACKPROPAGATION; SYNAPSE; DEVICE; MODEL;
D O I
10.1088/1674-1056/acb9f6
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
In recent years, spiking neural networks (SNNs) have received increasing attention of research in the field of artificial intelligence due to their high biological plausibility, low energy consumption, and abundant spatio-temporal information. However, the non-differential spike activity makes SNNs more difficult to train in supervised training. Most existing methods focusing on introducing an approximated derivative to replace it, while they are often based on static surrogate functions. In this paper, we propose a progressive surrogate gradient learning for backpropagation of SNNs, which is able to approximate the step function gradually and to reduce information loss. Furthermore, memristor cross arrays are used for speeding up calculation and reducing system energy consumption for their hardware advantage. The proposed algorithm is evaluated on both static and neuromorphic datasets using fully connected and convolutional network architecture, and the experimental results indicate that our approach has a high performance compared with previous research.
引用
收藏
页数:9
相关论文
共 50 条
  • [1] A progressive surrogate gradient learning for memristive spiking neural network
    王姝
    陈涛
    龚钰
    孙帆
    申思远
    段书凯
    王丽丹
    [J]. Chinese Physics B, 2023, (06) : 794 - 802
  • [2] A Fully Memristive Spiking Neural Network with Unsupervised Learning
    Zhou, Peng
    Choi, Dong-Uk
    Eshraghian, Jason K.
    Kang, Sung-Mo
    [J]. 2022 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS 22), 2022, : 634 - 638
  • [3] GRADUAL SURROGATE GRADIENT LEARNING IN DEEP SPIKING NEURAL NETWORKS
    Chen, Yi
    Zhang, Silin
    Ren, Shiyu
    Qu, Hong
    [J]. 2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2022, : 8927 - 8931
  • [4] Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks
    Neftci, Emre O.
    Mostafa, Hesham
    Zenke, Friedemann
    [J]. IEEE SIGNAL PROCESSING MAGAZINE, 2019, 36 (06) : 51 - 63
  • [5] Directly training temporal Spiking Neural Network with sparse surrogate gradient
    Li, Yang
    Zhao, Feifei
    Zhao, Dongcheng
    Zeng, Yi
    [J]. NEURAL NETWORKS, 2024, 179
  • [6] Meta-learning spiking neural networks with surrogate gradient descent
    Stewart, Kenneth M.
    Neftci, Emre O.
    [J]. NEUROMORPHIC COMPUTING AND ENGINEERING, 2022, 2 (04):
  • [7] Memristive Logic Design of Multifunctional Spiking Neural Network with Unsupervised Learning
    Andreeva, N. V.
    Ryndin, E. A.
    Gerasimova, M. I.
    [J]. BIONANOSCIENCE, 2020, 10 (04) : 824 - 833
  • [8] Memristive Logic Design of Multifunctional Spiking Neural Network with Unsupervised Learning
    N.V. Andreeva
    E.A. Ryndin
    M.I. Gerasimova
    [J]. BioNanoScience, 2020, 10 : 824 - 833
  • [9] In Situ Learning in Hardware Compatible Multilayer Memristive Spiking Neural Network
    Li, Jiwei
    Xu, Hui
    Sun, Sheng-Yang
    Li, Nan
    Li, Qingjiang
    Li, Zhiwei
    Liu, Haijun
    [J]. IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS, 2022, 14 (02) : 448 - 461
  • [10] A Forgetting Memristive Spiking Neural Network for Pavlov Experiment
    Chen, Ling
    Li, Chuandong
    Chen, Yiran
    [J]. INTERNATIONAL JOURNAL OF BIFURCATION AND CHAOS, 2018, 28 (06):