OPTIMAL STORAGE CAPACITY OF NEURAL NETWORKS AT FINITE TEMPERATURES

Gardner's analysis of the optimal storage capacity of neural networks is extended to study finite-temperature effects. The typical volume of the space of interactions is calculated for strongly diluted networks as a function of the storage ratio alpha, temperature T and the tolerance parameter m, from which the optimal storage capacity alpha(c) is obtained as a function of T and m. At zero temperature it is found that alpha(c) = 2 regardless of m while alpha(c) in general increases with the tolerance at finite temperatures. We show how the best performance for given alpha and T is obtained, which reveals a first-order transition from high-quality performance to a low-quality one at low temperatures. An approximate criterion for recalling, which is valid near m = 1, is also discussed.