DYNAMICS OF LEARNING AND GENERALIZATION IN A BINARY PERCEPTRON MODEL

Learning from examples by perception with N binary synaptic couplings is investigated within dynamic mean field theory. This applies to learning by simulated annealing, which is a polynomial algorithm for finite cooling rates. For examples created by a teacher perceptron of the same type, a discontinuous freezing transition occurs for a training set of size alpha-N with alpha < 1.58 at a temperature where the entropy is still positive. The resulting perceptrons have finite training and generalization error. For alpha > 1.58 the couplings of the teacher are found by the above process. This work extends previous investigations on a binary perceptron trained with random patterns.