Segmentation and Registration of Lettuce Multispectral Image Based on Convolutional Neural Network

In view of the deviations between the channels of multi-lens multi-spectral cameras and the inapplicability of traditional segmentation methods in multi-spectral images, the image analysis and processing process often has the problem of inability to automate segmentation or low segmentation accuracy, so a phase-based algorithm was proposed. And the semantic segmentation model based on UNet performs accurate registration of each channel of the field lettuce multispectral image and realizes foreground segmentation. The Canny algorithm was used to extract the edges of the multi-spectral channel images, and then the phase correlation algorithm was used to register the multi-spectral channel images. The average processing time of a single image was 0.92 s, efficiency was increased by 40%, and the registration accuracy reached 99%, which met the requirements of subsequent images and the required accuracy of segmentation. VGG16 was used as the backbone feature extraction network, and the double up sampling was directly used to make the final output image and the input image equal in height and width, and the optimized UNet model was constructed. The experimental results showed that the image registration and image segmentation network proposed achieved 99.19% pixel accuracy and an average IoU of 94.98%. It can perform foreground segmentation on lettuce multispectral images very well, which can be used for follow-up spectral analysis to study the precise phenotype of crops. © 2021, Chinese Society of Agricultural Machinery. All right reserved.