Neural Network Structure for Spatio-Temporal Long-Term Memory

This paper proposes a neural network structure for spatio-temporal learning and recognition inspired by the long-term memory (LTM) model of the human cortex. Our structure is able to process real-valued and multidimensional sequences. This capability is attained by addressing three critical problems in sequential learning, namely the error tolerance, the significance of sequence elements and memory forgetting. We demonstrate the potential of the framework with a series of synthetic simulations and the Australian sign language (ASL) dataset. Results show that our LTM model is robust to different types of distortions. Second, our LTM model outperforms other sequential processing models in a classification task for the ASL dataset.