DYNAMICS OF LEARNING AND GENERALIZATION IN PERCEPTRONS WITH CONSTRAINTS

Depending on the kind of constraints imposed on the weights of a perceptron, learning can be a combinatorially hard problem. As an example of this type, I discuss a perceptron with binary weights comparing results obtained from replica theory, dynamic mean field theory and simulated annealing. Contrary to the replica calculation, dynamics yields information about the performance of a polynomial algorithm in a situation where the best solution cannot be found in polynomial time. I also discuss improved learning algorithms and results for finite size perceptrons.