Realization and Hardware Implementation of Gating Units for Long Short-Term Memory Network Using Hyperbolic Sine Functions

This article proposes a new activation function (AF) sinh(beta x) +sinh(-1)(beta x) called combined hyperbolic Sine (comb-H-sine) to replace existing AFS like sigma and tanh in long short-term memory (LSTM) neural networks. The comb-H-sine function is implemented using purely combinational architectures with a 16-bit data width with fixed-point representation, resulting in improved accuracy. Both software and hardware modeling are used to investigate the proposed architecture. Compared to prior works on sigma and tanh functions, the hardware for the comb-H-sine function shows significant improvements in power consumption, number of cells, cell area, and delay. The proposed LSTM architecture using comb-H-sine outperforms existing AFs in terms of power delay product and accuracy on various datasets.