Adaptive Channel Pruning for Trainability Protection

Pruning is a widely used method for compressing neural networks, reducing their computational requirements by removing unimportant connections. However, many existing pruning methods prune pre-trained models by using the same pruning rate for each layer, neglecting the protection of model trainability and damaging accuracy. Additionally, the number of redundant parameters per layer in complex models varies, necessitating adjustment of the pruning rate according to model structure and training data. To overcome these issues, we propose a trainability-preserving adaptive channel pruning method that prunes during training. Our approach utilizes a model weight-based similarity calculation module to eliminate unnecessary channels while protecting model trainability and correcting output feature maps. An adaptive sparsity control module assigns pruning rates for each layer according to a preset target and aids network training. We performed experiments on CIFAR-10 and Imagenet classification datasets using networks of various structures. Our technique outperformed comparison methods at different pruning rates. Additionally, we confirmed the effectiveness of our technique on the object detection datasets VOC and COCO.