Robust regularized extreme learning machine for regression using iteratively reweighted least squares

Extreme learning machine (ELM) for regression has been used in many fields because of its easy-implementation, fast training speed and good generalization performance. However, basic ELM with l(2)-norm loss function is sensitive to outliers. Recently, l(1)-norm loss function and Huber loss function have been used in ELM to enhance the robustness. However, the l(1)-norm loss function and the Huber loss function can also be effected by outliers because of their linear correlation with the errors. Moreover, existing robust ELM methods only use l(2)-norm regularization or have no regularization term. In this study, we propose a unified model for robust regularized ELM regression using iteratively reweighted least squares (IRLS), and call it RELM-IRLS. We perform a comprehensive study on the robust loss function and regularization term for robust ELM regression. Four loss functions (i.e., l(1)-norm, Huber, Bisquare and Welsch) are used to enhance the robustness, and two types of regularization (l(2)-norm and l(1)-norm) are used to avoid overfitting. Experiments show that our proposed RELM-IRLS with l(2)-norm and l(1)-norm regularization is stable and accurate for data with 0 similar to 40% outlier levels, and that RELM-IRLS with l(1)-norm regularization can obtain a compact network because of the highly sparse output weights of the network.