Deep Neural Network Compression for Lightweight and Accurate Fish Classification

This research demonstrates the effectiveness of compression of deep learning models in a fish species classification system. In a bid to enhance model efficiency, response-based knowledge distillation was employed, balancing model compression and accuracy retention. The underwater footage collected in the Seychelles was annotated in collaboration with the South African Institute for Aquatic Biodiversity (SAIAB), and named the Aldabra data set. The accuracies of the MobileNet and DenseNet models were 89.26% and 86.78%, respectively, in the Aldabra dataset. Although the student model (MobileNet) achieved a lower accuracy, it enabled a three-fold reduction in parameters. In particular, in the much larger Fish4Knowledge dataset, MobileNet trained via knowledge distillation exceeded ResNet's accuracy, reaching 96.64% with an 18.7-fold size reduction. This performance improvement was attributed to efficient learning facilitated by knowledge distillation. The edge computing-friendly MobileNet model assimilates knowledge from the much larger ResNet output with information gain.