Improving Robustness of Deep Transfer Model by Double Transfer Learning

In recent years, deep learning models have been widely adopted for transfer learning tasks. The deep models, however, were shown to be easily attacked by adversarial examples, which is generated from original samples with the carefully designed small perturbations. Thus, the transfer learning models based on deep networks will also face this problem. Because of the particularity of transfer learning tasks, using the conventional adversarial training to improve the robustness of deep transfer models is very difficult. In this paper, we propose a novel Double Transfer Learning with Adversarial Training (DTLAT) method to enhance the robustness of deep transfer learning models. Our intuition is using the instances of the source domain and target domain to help with adversarial training. At the same time, we design a reverse transfer learning method to weaken the effect of attack methods and improve the performance of deep transfer models of target domain. We regard the adversarial examples from some kind of attack method, like FGSM, as an adversarial domain while try to improve the generalization ability on other attacks method. Experiments demonstrate that DTLAT exceed many other methods about improving the robustness of the deep transfer model on several benchmark datasets.