Recurrent Neural Networks for statistical Pattern Recognition

When Recurrent Neural Networks (RNN) are going to be used as Pattern Recognition systems. one of the problem to be solved, is the how to improve the poor capacity so as how, to avoid the spurious states. The classical solution to impose fixed points by means of the synaptic matrix W is the Hebb's law, which states that the synaptic weight w(y) should increase whenever neurons i and j have simultaneously the same activity level and it should decrease in the opposite case. When the pattern xi (mu), is acquired by the net, the updating of the weights, in the classical approach is given by Deltaw(y) = eta.xi (mu)(i).xi (mu)(j) (being eta a positive learning factor) The mathematical advantage of this interpretation lies in the fact that when the prototype xi (mu) is acquired, the synaptic weight w(y) should increase if neurons i and j receive a similar sign: in other words if xi (mu)(i).xi (mu)(j) is positive. On the other hand w(y) should decrease if xi (mu)(i).xi (mu)(j) is negative. In this paper we propose a modification of the procedure for updating the weights. Instead of the product, we will add the components, in other words, our weight updating in the training stage will be given by Deltaw(y) = eta.(xi (mu)(i) + xi (mu)(j)) This, at first slightly modification. will be do possible to obtain the statistical distribution of the prototypes which may be used in the retrieving procedure, The weak point in the classical deterministic approach comes from the fact that it does not have the appropriate tools to deal efficiently with the correlation between the state vectors and the prototype vectors. The capacity of the net is very poor because we can only know if one given vector is adequately correlated with the prototypes or not and we are not able to know what is the exact correlation degree between one given component vector and the prototypes. The algorithm. here proposed, makes possible to take a set of statistical parameter that control the probability of a pattern to belong to the basin of attraction of the prototypes. The capacity of the net is improved and the spurious states are reduced. In order to clarify and corroborate the theoretical results, together with the formal theory. an application is presented.