Multiple flow-based knowledge transfer via adversarial networks

The authors propose a new knowledge transfer method coupled with a generative adversarial network (GAN) when multiple-flow-based knowledge is considered in a teacher-student framework using a residual network (ResNet). In this method, several independent discriminators adapting multilayer-perceptron-based structures were designed for flow-based knowledge transfer. The proposed GAN-based optimisation alternatively updates the multiple discriminators and a student ResNet such that the flow-based features of the student ResNet are generated as closely as possible to the real features of a teacher ResNet. The experiments demonstrate that the student ResNet trained using the proposed method more accurately captures the distribution of the flow-based teacher knowledge than the l(2)-distance-based training method. In addition, the proposed method provided better classification accuracy than the existing GAN-based knowledge transfer method.