ESSM: Extended Synaptic Sampling Machine With Stochastic Echo State Neuro-Memristive Circuits

Synaptic stochasticity is an important feature of biological neural networks that is not widely explored in analog memristor networks. Synaptic Sampling Machine (SSM) is one of the recent models of the neural network that explores the importance of the synaptic stochasticity. In this paper, we present a memristive Echo State Network (ESN) with Extended-SSM (ESSM). The circuit-level design of the single synaptic sampling cell that can introduce stochasticity to the neural network is presented. The architecture of synaptic sampling cells is proposed that have the ability to adaptively reprogram the arrays and respond to stimuli of various strengths. The effect of stochasticity is achieved by randomly blocking the input with the probability that follows Bernoulli distribution, and can lead to the reduction of the memory capacity requirements. The blocking signals are randomly generated using Circular Shift Registers (CSRs). The network processing is handled in analog domain and the training is performed offline. The performance of the neural network is analyzed with a view to benchmark for hardware performance without compromising the system performance. The neural system was tested on ECG, MNIST, Fashion MNIST and CIFAR10 dataset for classification problem. The advantage of memristive CSR in comparison with conventional CMOS based CSR is presented. The ESSM-ESN performance is evaluated with the effect of device variations like resistance variations, noise and quantization. The advantage of ESSM-ESN is demonstrated in terms of performance and power requirements in comparison with other neural architectures.