A CMOS realizable recurrent neural network for signal identification

The architecture of an analog recurrent neural network that can learn a continuous-time trajectory is presented. The proposed learning circuit does not distinguish parameters based on a presumed model of the signal or system for identification. The synaptic weights are modeled as variable gain cells that can be implemented with a few MOS transistors. The network output consists primarily of neuron signals which portray the periodic characteristics of the input signal in unsupervised mode. For the specific purpose of demonstrating the trajectory learning capabilities, a periodic signal with varying characteristics is used. The developed architecture, however, allows for more general learning tasks typical in applications of identification and control. The periodicity of the input signal ensures consistency in the outcome of the error and convergence speed at different instances in time. While alternative on-line versions of the synaptic update measures can be formulated, which allow for faster learning speed and better convergence behavior, the architecture of the analog RNN used here is easier to implement while still allowing to demonstrate the general principle. Because the architecture depends on the network generating a stable limit cycle, and consequently a periodic solution which is robust over an interval of parameter uncertainties, we currently place the restriction of a periodic format for the input signals. The simulated network contains interconnected recurrent neurons with continuous-time dynamics. The system basically performs random-direction descent of the error as a multidimensional extension to the stochastic approximation. To achieve unsupervised learning in recurrent dynamical systems we propose a synapse circuit which has a very simple structure and is suitable for implementation in VLSI.