PolicyGAN: Training generative adversarial networks using policy gradient

This paper presents PolicyGAN, a policy gradient paradigm for training Generative Adversarial Networks that views the generator as an image generation neural agent which is rewarded by another neural agent, termed as the discriminator. Rewards are higher for samples near the original data manifold. In PolicyGAN, only reward signal from the output of the discriminator is used for updating the generator network using policy gradient. This obviates the need for gradient signal to flow through the discriminator for training the generator; an intrinsic property of original GAN formulation. Given the inherent difficulty of training adversarial models, and low convergence speed of policy gradient, training GANs using policy gradient is a non-trivial problem and requires deep study. Currently GANs have used only differentiable discriminators for training. Policy-GAN germinates the possibility of using a wide variety of non-differentiable discriminator networks for training GANs, something which was not possible with the original GAN framework. Another advantage of using policy gradient is that now the generator need not produce deterministic samples, but can generate a probability distribution from which samples can be taken. PolicyGAN thus paves the path to use a variety of probabilistic models.