The generalization error of the symmetric and scaled support vector machines

It is generally believed that the support vector machine (SVM) optimizes the generalization error and outperforms other learning machines. We show analytically, by concrete examples in the one dimensional case, that the SVM does improve the mean and standard deviation of the generalization error by a constant factor, compared to the worst learning machine. Our approach is in terms of extreme value theory and both the mean and variance of the generalization error are calculated exactly for all cases considered. We propose a new version of the SVM (scaled SVM) which can further reduce the mean of the generalization error of the SVM.