DiaNet: An elastic neural network for effectively re-configurable implementation

被引:0
|
作者
Wu, Man [1 ]
Kan, Yirong [1 ]
Erlina, Tati [1 ]
Zhang, Renyuan [1 ]
Nakashima, Yasuhiko [1 ]
机构
[1] Nara Inst Sci & Technol, Grad Sch Informat Sci, Nara, Japan
关键词
Sparse neural network; Multi-grained reconfigurable; Parameter reduction; Highly parallel; Neuromorphic architecture; PROCESSOR;
D O I
10.1016/j.neucom.2021.08.059
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
An elastic neural network is developed and evolved towards effectively re-configurable hardware in fully parallel on chip. The original prototype of DiaNet is organized as a symmetrical bisection neural network, which is feasible to be partitioned into arbitrary pieces of neural networks (NNs) without redundancy. To prevent the depth explosion in implementing complex tasks (complicated pattern recognition for instance), the evolution of DiaNets is investigated in this work. By using the I/O layer integration technology which enables all neurons in the hidden layer of DiaNet to receive inputs, the number of layers is reduced to 8.8% of DiaNet prototype. In this manner, the DiaNet topology is feasible to implement complex NNs without the risk of depth explosion. Moreover, the skip connection technology is proposed to avoid the gradient vanishing due to deep learning, which is significant to DiaNets especially. Compared with the LeNet5 model as state-of-the-art, the evolved DiaNet topology achieves the parameter reduction of 90.86% for MNIST recognition with the negligible loss of accuracy. To reduce hardware utilization, the sensitivity to the decline of computational precision and bit-width is investigated to suggest the guideline for efficient hardware implementations. Finally, the effectiveness of DiaNet is verified by the proposed re-configurable architecture on FPGA with the power reduction of 10.8% compared to state-of-the-art implementations. (c) 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
引用
收藏
页码:242 / 251
页数:10
相关论文
共 50 条
  • [1] DiaNet: An elastic neural network for effectively re-configurable implementation
    Wu, Man
    Kan, Yirong
    Erlina, Tati
    Zhang, Renyuan
    Nakashima, Yasuhiko
    [J]. Neurocomputing, 2021, 464 : 242 - 251
  • [2] DiaNet: An Efficient Multi-Grained Re-configurable Neural Network in Silicon
    Zhang, Renyuan
    Chen, Yan
    Nakada, Takashi
    Nakashima, Yasuhiko
    [J]. 32ND IEEE INTERNATIONAL SYSTEM ON CHIP CONFERENCE (IEEE SOCC 2019), 2019, : 132 - 137
  • [3] An elastic neural network toward multi-grained re-configurable accelerator
    Wu, Man
    Chen, Yan
    Kan, Yirong
    Nomura, Takeshi
    Zhang, Renyuan
    Nakashima, Yasuhiko
    [J]. NEWCAS 2020 - 18th IEEE International New Circuits and Systems Conference, Proceedings, 2020, : 218 - 221
  • [4] An Elastic Neural Network Toward Multi-Grained Re-configurable Accelerator
    Wu, Man
    Chen, Yan
    Ran, Yirong
    Nomura, Takeshi
    Zhang, Renyuan
    Nakashima, Yasuhiko
    [J]. 2020 18TH IEEE INTERNATIONAL NEW CIRCUITS AND SYSTEMS CONFERENCE (NEWCAS'20), 2020, : 218 - 221
  • [5] Re-configurable parallel Feed-Forward Neural Network implementation using FPGA
    El-Sharkawy, Mohamed
    Wael, Miran
    Mashaly, Maggie
    Azab, Eman
    [J]. INTEGRATION-THE VLSI JOURNAL, 2024, 97
  • [6] A power-efficient and re-configurable analog artificial neural network classifier
    Mohamed, Ahmed Reda
    Qi, Liang
    Wang, Guoxing
    [J]. MICROELECTRONICS JOURNAL, 2021, 111
  • [7] The re-configurable ID
    Muñoz, MG
    [J]. IEEE SYSTEMS READINESS TECHNOLOGY CONFERENCE: 2001 IEEE AUTOTESTCON PROCEEDINGS, 2001, : 757 - 769
  • [8] Re-configurable antennas
    Christodoulou, C. G.
    Anagnostou, D. A.
    Feldner, L. M.
    [J]. 2007 INTERNATIONAL WORKSHOP ON ANTI-COUNTERFEITING, SECURITY, AND IDENTIFICATION, 2007, : 9 - +
  • [9] Platform design approach for re-configurable network appliances
    Cmar, R
    Pasko, R
    Mignolet, JY
    Vanmeerbeeck, G
    Schaumont, P
    Vernalde, S
    [J]. PROCEEDINGS OF THE IEEE 2001 CUSTOM INTEGRATED CIRCUITS CONFERENCE, 2001, : 79 - 82
  • [10] Design and implementation of re-configurable transceiver for cdma2000
    Sheen, M
    Lee, S
    Kim, J
    [J]. VTC2004-FALL: 2004 IEEE 60TH VEHICULAR TECHNOLOGY CONFERENCE, VOLS 1-7: WIRELESS TECHNOLOGIES FOR GLOBAL SECURITY, 2004, : 704 - 708