Robustness analysis for compact neural networks

Deep neural networks (DNNs) have achieved comparable performance to humans on many tasks. However, there are two problems in deploying a DDN to terminal devices of a real scene. First, DNNs consume huge computing resources and memory; therefore, it is difficult to directly apply DNNs to resource-constrained terminal devices. Second, real-world data are often affected by noise; accordingly, the model deployed in the real scene needs to have good robustness. Recently, many compression methods for DNNs have been adapted to resource-constrained terminal devices, and robustness analyses of these compression models have increasingly received attention. In this paper, the adversarial and corruption robustness of several compression algorithms, such as pruning, quantization, and knowledge distillation, is first analyzed. Then, several studies of compact networks with improved robustness are summarized. Finally, several challenges are discussed, and possible research directions for compact robustness networks are proposed. © 2022, Science China Press. All right reserved.