Fuzzy training for neural networks

High-speed convergence training algorithms in neural networks may render inferior generalization, when compared to some lower-speed methods, e.g., back-propagation (BP). In this paper, a method is proposed in order to improve the generalization of such high-speed techniques, mainly the Levenberg-Marquardt (LM) method, by introducing fuzz (noise) at the early stages of the training. The amount and the duration of this fuzz are shown to be very crucial in the efficacy of the proposed method. These values are shown to depend on the shape of the learning curve. In this preliminary study, a step-function curve-fitting scheme is adopted as the benchmark problem, due to the fact that this problem is highly sensitive to over-training and under-training. It is shown that adding a variable amount of fuzz to the neural network training algorithm, in the case of this problem, can considerably improve the learning performance.