A Multi-objective Optimization Model for Redundancy Reduction in Convolutional Neural Networks

Until now, convolutional neural networks (CNNs) still among the powerful and robust deep neural networks that proved its efficiency through several real applications. However, their functioning requires a large number of parameters which in turn lead to some undesired effects such as the overparametrization, overfitting and the high consumption of computational resources. To deal effectively with these issues, we propose in this paper a new multi-objective optimization model for redundancy reduction in CNNs. The suggested model named MoRR-CNN allows to eliminate the unwanted parameters (kernels and weights) as well as to speeding up the CNN evaluation process. It consists of two objectives, the first one is related to the training task where the solution is the optimal parameters. These parameters are combined with a set of decision variables that controlling their contribution in the training process, making at the end a redundancy-related objective function. Both of the objectives are optimized using the non dominated sorting genetic algorithm NSGA-II. The robustness of MoRR-CNN has been demonstrated through different experimentation applied on three benchmark datasets including MNIST, Fashion-MNIST and CIFAR and using three of the most known CNNs such as VGG-19, Net-in-Net and VGG-16.