Random approximated greedy search for feature subset selection

We propose a sequential approach called Random Approximated Greedy Search (RAGS) in this paper and apply it to the feature subset selection for regression. It is an extension of GRASP/Super-heuristics approach to complex stochastic combinatorial optimization problems, where performance estimation is very expensive. The key points of RAGS are from the methodology of Ordinal Optimization (00). We soften the goal and define success as good enough but not necessarily optimal. In this way, we use more crude estimation model, and treat the performance estimation error as randomness, so it can provide random perturbations mandated by the GRASP/Super-heuristics approach directly and save a lot of computation effort at the same time. By the multiple independent running of RAGS, we show that we obtain better solutions than standard greedy search under the comparable computation effort.