Dynamic channel pruning via activation gates

Dynamic channel pruning has been proved to be an effective method by dynamically adjusting the inference path to reduce the computing costs. However, in most existing work, the classification performance decreases rapidly with the increase of pruning rate because their pruning strategy weakens the representation ability of the model to a certain extent. To resolve this problem, a dynamic channel pruning method based on activation gate (DCPAG) is proposed, which can better maintain the classification performance while reducing the computing costs. First, a pipeline aiming for generating pruning strategy, namely channel pruning auxiliary (CPA) is proposed, which considers both the representation ability and computing costs. Second, the pruning strategy generated by CPA is embedded into dynamic rectifying linear unit (DyReLU) to form the embedded dynamic rectifying linear unit (EB-DyReLU), which achieves dynamic channel pruning while maintaining the representation capability. Third, each input sample was self-classified according to its identification difficulty during pruning, and additional training was given to hard samples to achieve better classification performance. Finally, some experiments are carried out on CIFAR-10 and ImageNet respectively to verify the effectiveness of DCPAG in accuracy and floating point of per second (FLOPs). The results show that the proposed method achieves better performance than other similar channel-based methods at the same pruning rate. Specifically, this method not only achieves 0.5-1.5% improvement in classification accuracy, but also reduces the computational costs by 5%-20%.