ONLINE SELECTIVE TRAINING FOR FASTER NEURAL NETWORK LEARNING

Training neural networks is a computationally challenging problem that requires significant time efforts. In this paper, we propose two approaches that improve efficiency of this task by actively selecting most relevant points from a training data set. The first approach forms a batch that maximizes the reduction of the estimator's entropy, while the second approach only trains on datapoints whose predicted probability is below a predetermined threshold. Both techniques rely on data metrics to speed up training while retaining the epoch-based neural network training framework. The results demonstrate that the proposed methods enable significant reduction of training time in experiments on the CIFAR10 dataset without compromising the accuracy.