GANcoder: robust feature point matching using conditional adversarial auto-encoder

Dense and robust image matching is a crucial step for an accurate 3D object reconstruction. Various kind of textures requires different feature descriptors for high-quality image matching. Often it is not easy to choose the best feature descriptor from the wide range of algorithms available nowadays. This paper is focused on the development of a new conditional generative adversarial auto-encoder (GANcoder) based on the deep learning. We use a coder-decoder architecture with four convolutional and four deconvolutional layers as a starting point for our research. Our main contribution is a generative adversarial framework GANcoder for training the auto-encoder on the textureless data. Traditional training approaches using an L1 norm tend to converge to the mean image on the low-textured images. In contrast, we use an adversarial discriminator to provide an additional loss function that is focused on distinguishing real images from the training dataset from the auto-encoder reconstruction. We collected a large MVSGAN dataset of feature points from textured and textureless objects to train and evaluate our model and baselines. The dataset includes 512k pairs of image patches. We performed qualitative evaluation of our GANcoder and baselines for two tasks. Firstly, we compare the matching score of the our GANcoder and baselines. Secondly, we evaluate the accuracy of 3D reconstruction of low-textured objects using an SfM pipeline with stereo-matching provided by our GANcoder. The results of the evaluation are encouraging and demonstrate that our model achieves and surpasses the state of the art in the feature matching on low-textured objects.