Feature-based Perturbation Makes a Better Ensemble Learning for SSL Classification

Semi-supervised learning (SSL) poses a significant practical challenge in the field of computer vision. Pseudo Labeling methods (PL methods), as representative SSL techniques, obtain the State Of The Art (SOTA) performances in SSL. However, the error accumulation phenomenon that easily occurs during their self-training process results in high bias and variance in pseudo-labels, further hindering effective model training. To overcome this issue, we propose a feature-based perturbation method for ensemble learning method to make it more effective and efficient in SSL. By perturbing the features input to each prediction head within a multi-head ensemble structure, the method reduces the prediction correlation among prediction heads, thereby enhancing ensemble gain. It is worth emphasizing that the proposed method is highly generalizable and can be easily extended to arbitrary SSL frameworks. Experimental data demonstrate that our method outperforms the current state-of-the-art techniques on the CIFAR10 dataset and significantly improves the quality of pseudo-labels.