A long-term memory enhanced echo state network and its optimization

This paper proposes a new type of echo state network called Long-Term Memory Enhanced Echo State Network (LTME-ESN). By extending the ideas of the forget gate and the input gate from the LSTM, LTME-ESN modifies the equation for updating the state in a Leaky Integral Echo State Network (Leaky-ESN). By regulating the accumulation of information, the construction of extended periods of dependency between states may be delayed, allowing for adaptive management of the data transported from the previous state to the present one. In order to optimize parameters by using Stochastic Gradient Descent (SGD), this research presents a necessary condition for LTME-ESN to meet the properties of the echo state network. The study uses low-frequency sinusoidal, high-frequency sinusoidal, and chaotic time series to demonstrate the model's efficiency. Simulations show that LTME-ESN outperforms Leaky-ESN in prediction accuracy and variation. This study provides long-term memory enhanced Echo State Networks (LTME-ESN), a novel and improved Leaky-ESN model. The basic concept is to update the state of neurons in the reservoir by integrating the LSTM's input gate and forget gate concepts into the echo state network. According to the results of all simulation experiments, LTME-ESN model has better prediction accuracy and lower volatility.image