Regression by support vector machines and its applications to engineering design

Support vector machine (SVM) has been recognized as a powerful machine learning technique SVM was originally developed for pattern classification and later extended to regression (Vapnik et al 1995) In pattern classification problems with two class sets. it generalizes linear classifiers into high dimensional feature spaces through nonlinear mappings defined implicitly by kernels in the Hilbert space so that it may produce nonlinear classifiers in the original data space. Linear classifiers then are optimized to give the maximal margin separation between the classes. This task is performed by solving some type of mathematical programming such as quadratic programming (QP) or linear programming (LP). On the other hand. from a viewpoint of mathematical programming for machine learning, the idea of maximal margin separation was employed in the multi-surface method (MSM) suggested by Mangasarian in 1960's. Also. linear classifiers using goal programming were developed extensively in 1980's The authors have developed several varieties of SVM using multi-objective programming and goal programming (MOP/GP) techniques. This paper extends the family of SVM for classification to regression. and discusses their characteristics and abilities through numerical experiments in engineering design problems.