Learning and retrieving spatio-temporal sequences with any static associative neural network

The purpose of the present paper is to report on the design of a general system that is capable of learning and retrieving spatio-temporal sequences using any static associative neural networks (ANN's), including both autoassociative and heteroassociative neural networks. This artificial neural system has three major components: a voting network, a parallel array of ANN's, and delayed feedback lines from the output of the system to the ANN layer. The system has separate primary and pairing input channels. During learning, pairs of sequences of spatial patterns are presented to the primary and the pairing input channels simultaneously. During retrieving, a cue sequence, which is presented to the primary input channel only, and which may have spatial imperfections and temporal gaps, causes the system to output the stored spatio-temporal sequence. As a demonstration of the applicability of the present general system, we present an implementation using a specific neural network, that is, our dynamically generated variant of the counterpropagation network. This system shows computational advantages such as fast and accurate learning and retrieving, and the ability to store a large number of sequences consisting of nonorthogonal spatial patterns.