A Reconfigurable Accelerator for Generative Adversarial Network Training Based on FPGA

In recent years, generative adversarial networks (GANs) have been widely applied in various tasks, demonstrating outstanding performance, such as image generation, style transfer, and video generation. However, due to their high computation complexity and large amount of intermediate data to be stored, the on-device learning that trains GANs on embedded platforms remains a very challenging problem. In this work, we propose an FPGA-based reconfigurable accelerator for efficient GAN training. Firstly, the cascaded fast FIR algorithm (CFFA) is optimized towards GAN training, and a fast convolution processing element (FCPE) based on the optimized algorithm is introduced to support various computation patterns during GAN training. Secondly, a well optimized architecture on the basis of FCPEs is presented, which is flexible to support forward, backward, and weight gradient phases of GAN training. Finally, training of a prevailing network (DCGAN) is implemented on Xilinx VCU108 platform with our methods. Experimental results show that our design achieves 315.18 GOPS and 83.87 GOPS/W in terms of throughput and energy efficiency, respectively. Our accelerator achieves 4.0 x improvement over the state-of-the-art design in energy efficiency.