Adversarial Robustness of Multi-bit Convolutional Neural Networks

Deploying convolutional neural networks (CNNs) on resource-constrained, embedded hardware constitutes challenges in balancing task-related accuracy and resource-efficiency. For safety-critical applications, a third optimization objective is crucial, namely the robustness of CNNs. To address these challenges, this paper investigates the tripartite optimization problem of task-related accuracy, resource-efficiency, and adversarial robustness of CNNs by utilizing multi-bit networks (MBNs). To better navigate the tripartite optimization space, this work thoroughly studies the design space of MBNs by varying the number of weight and activation bases. First, the pro-active defensive model MBN3x1 is identified, by conducting a systematic evaluation of the design space. This model achieves better adversarial accuracy (+10.3pp) against the first-order attack PGD-20 and has 1.3x lower bit-operations, with a slight degradation of natural accuracy (-2.4pp) when compared to a 2-bit fixed-point quantized implementation of ResNet-20 on CIFAR-10. Similar observations hold for deeper and wider ResNets trained on different datasets, such as CIFAR-100 and ImageNet. Second, this work shows that the defensive capability of MBNs can be increased by adopting a state-of-the-art adversarial training (AT) method. This results in an improvement of adversarial accuracy (+13.6pp) for MBN3 x 3, with a slight degradation in natural accuracy (-2.4pp) compared to the costly full-precision ResNet-56 on CIFAR-10, which has 7x more bit-operations. To the best of our knowledge, this is the first paper highlighting the improved robustness of differently configured MBNs and providing an analysis on their gradient flows.