Fine-Grained Fish Image Classification Based on a Bilinear Network with Spatial Transformation

Effective classification of various fish species under water has great practical and theoretical significance. Due to the extreme conditions of the ocean, underwater images have very low resolution. Since the living environment is highly complex, fish images have properties of high inter-class similarity, large intra-class variety, and are greatly affected by light, angle, posture etc. These factors make fish classification a challenging task. To cope with these challenges, a deep fine-grained fish imageclassification model is proposed. It consists of a spatial transformer network and a bilinear network. Specifically, the spatial transformer network aims at removing the complex background as an attention mechanism and selecting the region of interest in the image. The bilinear network extracts the bilinear features of the image by fusing the feature maps of two deep networks, so that it responds to the discriminative part of the target. The model can be trained in an end-to-end way. The model achieves its best performance on the public F4K dataset. The recognition accuracy was 99.36%, which was 0.56% higher than the DeepFish algorithm. In addition, a new dataset called Fish100, containing 100 categories of 6358 images, was released. Accuracy of the model is 0.98% higher than that of the bilinear convolutional neural network(BCNN)model. Experiments on several datasets verified the effectiveness and superiority of the proposed algorithm. © 2019, Editorial Board of Journal of Tianjin University(Science and Technology). All right reserved.