Robust gradient aware and reliable entropy minimization for stable test-time adaptation in dynamic scenarios

Test-time adaptation (TTA) aims to provide neural networks capable of adapting to the target domain distribution using only unlabeled test data. Most existing TTA methods have achieved success under mild conditions, such as independently sampled data from a single or multiple static domains. However, these attempts may fail in dynamic scenarios, where the test data distribution undergoes continuous changes over time. By digging into the failure cases, we find that high-entropy or noisy samples during long-term adaptation may lead to inevitable catastrophic failure. Thus, we propose a Robust Gradient Aware and Reliable entropy minimization approach, called RGAR, to further stabilize TTA from three aspects: (1) Boosting model robustness to distribution shift, we propose a dual-stream perturbation technique that enables two weak-to-strong perturbation views of the student model guided by a common strong view of the mean teacher model; (2) mitigating the impact of high-entropy samples from different scenarios, we present to minimize the reliable samples that take into account both the distribution shift and sample adaptation degree; (3) enabling the model to be insensitive to small perturbations by encouraging model weights to reach flatter minima while focusing on the maximal gradient norm. Extensive experimental results demonstrate the effectiveness of our proposed method, RGAR. We achieve state-of-the-art performance on widely used benchmark datasets, such as CIFAR10C, CIFAR100C, and ImageNet-C. Our source code is available at https://anonymous.4open.science/r/D152/.