Facial Image Augmentation from Sparse Line Features Using Small Training Data

Data collection is expensive in many research fields. Data augmentation from a very small dataset, such as synthesising realistic images from limited or incomplete information available from a small number of sample images, is still an enormous challenge using deep convolutional neural networks that traditionally require a large number of training data to achieve reasonable performance. For the purpose of manipulating the synthetic results with diversity, line features, which can be easily obtained through computer vision, hand-drawn lines, or customerdesigned sketches, can be utilized to provide extra details to effectively augment a small training dataset for many applications. In this paper, a novel conditional generative adversarial network (GAN) framework for synthesising photorealistic facial images using small training data and limited line features is proposed, where sparse line features are expected to simulate abstract and incomplete handdrawn sketches for introducing diversity in the augmented facial images. The proposed GAN framework can automatically recover the lost information caused by incomplete input features, which has been proved to efficiently reduce unexpected distortions but enhance data diversity with controllable sparse line features. Experimental results have demonstrated that the proposed method with a very small dataset, 50 training images only, can generate images of higher quality than the traditional translationmethods and preserve essential details to synthesise diverse but realistic facial images. Compared to the state-of-the-art methods, the proposed GAN framework can generate more photorealistic facial images using controllable sparse line features in terms of higher FID and KID scores as well as preference evaluation by human perception.