Multi-step Training of a Generalized Linear Classifier

We propose a multi-step training method for designing generalized linear classifiers. First, an initial multi-class linear classifier is found through regression. Then validation error is minimized by pruning of unnecessary inputs. Simultaneously, desired outputs are improved via a method similar to the Ho-Kashyap rule. Next, the output discriminants are scaled to be net functions of sigmoidal output units in a generalized linear classifier. This classifier is trained via Newton’s algorithm. Performance gains are demonstrated at each step. Using widely available datasets, the final network’s tenfold testing error is shown to be less than that of several other linear and generalized linear classifiers reported in the literature.