Active Learning by Extreme Learning Machine with Considering Exploration and Exploitation Simultaneously

As an important machine learning paradigm, active learning has been widely applied to scenarios in which it is easy to acquire a large number of instances but labeling them is expensive and/or time-consuming. In such scenario, active learning can significantly reduce the cost of labeling the instances. Extreme learning machine (ELM) is a popular supervised learning model that has the structure of a single-hidden-layer feed-forward network, and has such merits as low computational cost, high training speed, and high generalization ability. Previous studies have shown that the integration of active learning with the ELM can yield effective and efficient results. However, the currently used method of integration considers only the capability for exploitation neglecting that for exploration, further increasing the risk of the results falling into local optima in context of a cold start. To address this problem, we propose an improved algorithm called the AL-SNN-ELM in this paper. It contains two sub-procedures for a sequential query: The exploration strategy, which uses the shared nearest neighbor (SNN) clustering algorithm, takes charge of exploring the sample space to query representative instances, and the exploitation strategy is responsible for transforming the actual outputs of the ELM into posterior probabilities to query uncertain instances. That is to say, the exploration sub-procedure helps roughly locate the decision boundary for sound classification by observing the global distribution of the data, while the exploitation sub-procedure subtly tunes this decision boundary by observing the distribution of local instances surrounding it. In addition, to reduce the time-complexity of active learning, online-sequential extreme learning machine is also adopted to replace the traditional ELM. The results of experiments on 20 UCI benchmark datasets and two real-world datasets show that the proposed AL-SNN-ELM algorithm can yield a significant improvement in performance in comparison with the traditional AL-ELM algorithm, indicating that it is useful to consider the exploration and exploitation simultaneously in the framework of active learning.