Recurrent neural networks with nonlinear synapses for solving optimization problems

Artificial neural network models typically assume that the synapses are linear. This assumption is not well founded in biological fact, but serves to simplify the analysis and design of artificial neural networks for a variety of tasks, including learning and optimization. In this paper we consider a new form of pulse coupled neural network, where synaptic action is comprised of two components: a nonlinear action which depends on both pre-synaptic and post-synaptic activity levels, and a linear component which is associated with the post-synaptic circuitry alone. We show that interestingly, networks comprising such neurons can be used to efficiently solve difficult optimization problems. Such tasks may arise in the processing of signals in the auditory and visual pathway, but the proposed approach can be used for other applications as well. The assumption of a nonlinear synapse leads to analog neural networks for optimization, which are compact in terms of the number of neurons and interconnections required for a task, in comparison with traditional approaches. It also motivates the design of all-digital neural networks for the same applications. These can be realized in a simple manner on inexpensive programmable digital hardware.