Orthogonal Least Squares Based on Singular Value Decomposition for Spare Basis Selection

This paper proposes an improved orthogonal least square algorithm based on Singular Value Decomposition for spare basis selection of the linear-in-the-weights regression models. The improved algorithm is based on the idea of reducing meaningless calculation of the selection process through the improvement of orthogonal least square by using the Singular Value Decomposition. This is achieved by dividing the original candidate bases into several parts to avoid comparing among poor candidate regressors. The computation is further simplified by utilizing the Singular Value Decomposition to each sub-block and replacing every sub-candidate bases with the obtained left singular matrix, which is a unitary matrix with lower dimension. It can avoid the computation burden of the repeated orthogonalisation process before each optimal regressor is determined. This algorithm is applied to the linear-in-the-weights regression models with the predicted residual sums of squares (PRESS) statistic and minimizes it in an incremental manner. For several real and benchmark examples, the present results indicate that the proposed algorithm can relieve the load of the heave calculation and achieve a spare model with good performance.