Self-supervised Visual Feature Learning and Classification Framework: Based on Contrastive Learning

Due to the high (human) cost of image annotation, lack of large-scale annotation data prevents computer vision models from fully solving image classification tasks. On the other hand, the Bert model [1] has shown that training large scale models with non-annotation text data is possible. Inspired by this, we propose a Self-supervised Visual Feature Learning and Classification framework (SVFLC) that can be applied to large scale training data without annotation. This approach is based on the contrastive predictive learning (CPL) method. By refining CPL using special data augmentation and new contrastive learning mechanisms, learning shape-biased features can be emphasized. In the next step, these features are used to produce pseudo labels via a clustering algorithm. Inspired by the recent research on noisy labels, we proceed to employ distance of the sample from cluster centers to eliminate low-confidence labels, and use soft triplet loss and classification loss jointly for achieving robust performance in the final classification. In this unsupervised learning paradigm, on the Imagenet dataset, our framework outperforms commonly used approaches. The unsupervised performance is lower than supervised and semi-supervised learning approaches, but our proposed framework is more suitable for general cases, and serves as a baseline algorithm for future improvement to the unsupervised learning paradigm.