SVM Based Feature Selection: Why Are We Using the Dual?

Most Support Vector Machines (SVM) implementations are based on solving the dual optimization problem. Of course, feature selection algorithms based on SVM are not different and, in particular, the most used method in the area, Guyon et al.'s Recursive Feature Elimination (SVM-RFE) is also based on the dual problem. However, this is just one of the options available to find a solution to the original SVM optimization problem. In this work we discuss some potential problems that arise when ranking features with the dual-based version of SVM-RFE and propose a primal-based version of this well-known method, PSVM-RFE. We show that our new method is able to produce a better detection of relevant features, in particular in situations involving non-linear decision boundaries. Using several artificial and real-world datasets we compare both versions of SVM-RFE, finding that PSVM-RFE is preferable in most situations.