Explore pretraining for few-shot learning

Few-shot learning aims to learn to classify new categories with a few samples. Pretraining the model on the base class can improve the performance of the model on the new category. To further improve the pretraining performance of the model on the base class, we propose a two-stage model pretraining method. In the first stage, we conduct Simsiam contrastive learning pretraining, which can help the model learn invariant knowledge. In the second stage, we conduct multi-task pretraining for general classification tasks and rotation prediction tasks, which can help the model learn the equivalent knowledge. Two pretraining stages can significantly enhance the model's capacity to learn new categories and enhance the effectiveness of few-shot categorization. Experiments show that our method achieves State-of-the-art few-shot classification performance on the mini-ImageNet and FC100 datasets for 1-shot and 5-shot tasks.