Rewriting a Generative Model with Out-of-Domain Patterns

Generative Adversarial Networks (GANs) can synthesize images with the same distribution as the training dataset through random vectors. However, there is still no proper way to explain the generation rules in the generator and modify those rules. Although there have been some works on rewriting the rules in generative models in recent years, these works mainly focus on rewriting the rules between the patterns it has already learned. In this paper, we cast the problem of rewriting as rewriting the rule in the generator with out-of-domain patterns. To the best of our knowledge, this is the first time this problem has been investigated. To address this problem, we propose the Re-Generator framework. Specifically, our method projects out-of-domain images into the feature space of a specific layer in the generator and designs a mechanism to enable the generator to decode that feature back to the original image as possible. Besides that, we also suggest viewing multiple layers of the model as nonlinearity associative memory and design a rewriting strategy with better performance based on this. Finally, the results of the experiments on various generative models in multiple datasets show that our method can effectively use the patterns that the model has never seen before as rules for rewriting.