Realization and Hardware Implementation of Gating Units for Long Short-Term Memory Network Using Hyperbolic Sine Functions

被引：0

作者：

Joseph, Tresa ^{[1
]}

Bindiya, T. S. ^{[1
]}

机构：

[1] Natl Inst Technol Calicut, Dept Elect & Commun Engn, Kozhikode 673601, India

来源：

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS | 2023年 / 42卷 / 12期

关键词：

Hardware; Long short term memory; Computer architecture; Recurrent neural networks; Mathematical models; Field programmable gate arrays; Convergence; Activation functions (AFs); long short-term memory (LSTM); recurrent neural networks (RNNs);

D O I：

10.1109/TCAD.2023.3293045

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

This article proposes a new activation function (AF) sinh(beta x) +sinh(-1)(beta x) called combined hyperbolic Sine (comb-H-sine) to replace existing AFS like sigma and tanh in long short-term memory (LSTM) neural networks. The comb-H-sine function is implemented using purely combinational architectures with a 16-bit data width with fixed-point representation, resulting in improved accuracy. Both software and hardware modeling are used to investigate the proposed architecture. Compared to prior works on sigma and tanh functions, the hardware for the comb-H-sine function shows significant improvements in power consumption, number of cells, cell area, and delay. The proposed LSTM architecture using comb-H-sine outperforms existing AFs in terms of power delay product and accuracy on various datasets.

引用

页码：5141 / 5145

页数：5

共 50 条

[31] Detection of Abnormal Network Traffic Using Bidirectional Long Short-Term Memory
Thi Thanh N.N.
Nguyen Q.H.
Computer Systems Science and Engineering, 2023, 46 (01): : 491 - 504
[32] Simplified Gating in Long Short-term Memory (LSTM) Recurrent Neural Networks
Lu, Yuzhen
Salem, Fathi M.
2017 IEEE 60TH INTERNATIONAL MIDWEST SYMPOSIUM ON CIRCUITS AND SYSTEMS (MWSCAS), 2017, : 1601 - 1604
[33] Short-Term Passenger Flow Prediction Using a Bus Network Graph Convolutional Long Short-Term Memory Neural Network Model
Baghbani, Asiye
Bouguila, Nizar
Patterson, Zachary
TRANSPORTATION RESEARCH RECORD, 2023, 2677 (02) : 1331 - 1340
[34] Short-term Load Forecasting of Distribution Network Based on Combination of Siamese Network and Long Short-term Memory Network
Ge L.
Zhao K.
Sun Y.
Wang Y.
Niu F.
Dianli Xitong Zidonghua/Automation of Electric Power Systems, 2021, 45 (23): : 41 - 50
[35] Predicting Short-term Traffic Flow by Long Short-Term Memory Recurrent Neural Network
Tian, Yongxue
Pan, Li
2015 IEEE INTERNATIONAL CONFERENCE ON SMART CITY/SOCIALCOM/SUSTAINCOM (SMARTCITY), 2015, : 153 - 158
[36] Short-term Elevator Traffic Flow Estimation with Hybrid Long Short-Term Memory Network
Zheng, Qi
Zhao, Chunhui
2020 CHINESE AUTOMATION CONGRESS (CAC 2020), 2020, : 7141 - 7146
[37] Adaptive Clustering Long Short-Term Memory Network for Short-Term Power Load Forecasting
Qi, Yuanhang
Luo, Haoyu
Luo, Yuhui
Liao, Rixu
Ye, Liwei
ENERGIES, 2023, 16 (17)
[38] Short-term runoff forecasting in an alpine catchment with a long short-term memory neural network
Frank, Corinna
Russwurm, Marc
Fluixa-Sanmartin, Javier
Tuia, Devis
FRONTIERS IN WATER, 2023, 5
[39] Effective Implementation of Convolutional Long Short-Term Memory (ConvLSTM) Network in Forecasting Solar Irradiance
Obiora, Chibuzor N.
Ali, Ahmed
IECON 2021 - 47TH ANNUAL CONFERENCE OF THE IEEE INDUSTRIAL ELECTRONICS SOCIETY, 2021,
[40] Long Short-term Memory Neural Network for Network Traffic Prediction
Zhuo, Qinzheng
Li, Qianmu
Yan, Han
Qi, Yong
2017 12TH INTERNATIONAL CONFERENCE ON INTELLIGENT SYSTEMS AND KNOWLEDGE ENGINEERING (IEEE ISKE), 2017,

← 1 2 3 4 5 →