GAAT: Group Adaptive Adversarial Training to Improve the Trade-Off Between Robustness and Accuracy

Adversarial training is by far one of the most effective methods to improve the robustness of deep neural networks against adversarial examples. However, the trade-off between robustness and accuracy is still a challenge in adversarial training. Previous methods used adversarial examples with a fixed perturbation budget or specific perturbation budgets for each example, which is inefficient in improving the trade-off and lacks the ability to control the trade-off flexibly. In this paper, we show that the largest element of logit, zmax, can roughly represent the minimum distance between an example and its neighboring decision boundary. Thus, we propose group adaptive adversarial training (GAAT) that divides the training dataset into several groups based on zmax and develops a binary search algorithm to determine the group perturbation budgets for each group. Using the group perturbation budgets to perform adversarial training can fine-tune the trade-off between robustness and accuracy. Extensive experiments conducted on CIFAR-10 and ImageNet-30 show that our GAAT can achieve a more perfect trade-off than TRADES, MMA, and MART.