SeNPIS: Sequential Network Pruning by class-wise Importance Score

In the last decade, pattern recognition and decision making from images has mainly focused on the development of deep learning architectures, with different types of networks such as sequential, residual and parallel. Although the depth and size varies between models, they all have in common that they can contain multiple filters or neurons that are not important for the purpose of prediction, and that do negatively impact the size of the model and their inference times. Therefore, it is advantageous to use pruning methods that, while largely maintaining the initial performance of the classifier, significantly reduce its size and FLOPs. In parameter reduction, the decision rule is generally based on mathematical criteria, e.g. the amplitude of the weights, but not on the actual impact of the filter or neuron on the classifier performance for each of the classes. Therefore, we propose SeNPIS as a method that involves both filter and neuron selection based on a class-wise importance score, and network resizing to increase parameter reduction and FLOPs in sequential CNN networks. Several tests were performed to compare SeNPIS with other representative state-of-the-art methods, for the CIFAR-10 and Scene-15 datasets. It was found that for similar values of accuracy, and even in some cases with a slight increase in accuracy, SeNPIS significantly reduces the number of parameters by up to an additional 23.5% (i.e., a 51.05% reduction with SeNPIS versus a 27.53% reduction with Gradient) and FLOPs by up to an additional 26.6% (i.e., a 74.82% reduction with SeNPIS versus a 48.16% reduction with Weight) compared to the Weight, Taylor, Gradient and LRP methods.(c) 2022 Elsevier B.V. All rights reserved.