Neural Networks for Classification and Unsupervised Segmentation of Visibility Artifacts on Monocular Camera Image

For computer vision systems of autonomous vehicles, an important task is to ensure high reliability of visual information coming from on-board cameras. Frequent problems are contamination of the camera lens, its defocusing due to mechanical damage, image motion blur in low light conditions. In our work, we propose a novel neural network approach to the classification and unsupervised segmentation of visibility artifacts on monocular camera images. It is based on the compact classification deep neural network with an integrated modification of the gradient method for class activation map and segmentation mask generating. We present a new dataset named Visibility Artifacts containing over 22 300 images including six common artifacts: complete loss of camera visibility, strong or partial contamination, rain or snow drops, motion blur, defocus. To check the quality of artifact localization, a small test set with ground truth masks is additionally labeled. It allowed us to objectively quantitatively compare various methods for constructing class activation maps (CAMERAS, FullGrad, original and modified Grad-CAM, Layer-CAM), which demonstrated image segmentation quality above 54% mIoU without any supervision. This is a promising result. Experiments with the developed dataset demonstrated the superiority of the neural network classification method ResNet-18_U (with test accuracy of 99.37%), compared to more complex convolutional (ResNet-34, ResNeXt-50, EfficientNet-B0) and transformer (ViT-Ti, DeiT-Ti) neural networks. The code of the proposed method and the dataset are publicly available at https://github.com/vdkuznetsov/CaUS_Visibility_Artifacts.