Negative sampling in semi-supervised learning

We introduce Negative Sampling in Semi-Supervised Learning ((NSL)-L-3), a simple, fast, easy to tune algorithm for semi-supervised learning (SSL). (NSL)-L-3 is motivated by the success of negative sampling/contrastive estimation. We demonstrate that adding the (NSL)-L-3 loss to state-of-the-art SSL algorithms, such as the Virtual Adversarial Training (VAT), significantly improves upon vanilla VAT and its variant, VAT with Entropy Minimization. By adding the (NSL)-L-3 loss to Mix-Match, the current state-of-the-art approach on semi-supervised tasks, we observe significant improvements over vanilla Mix-Match. We conduct extensive experiments on the CIFAR10, CIFAR100, SVHN and STL10 benchmark datasets. Finally, we perform an ablation study for (NSL)-L-3 regarding its hyperparameter tuning.