Counting Cigar Tobacco Plants from UAV Multispectral Images via Key Points Detection Approach

Tobacco is an important industrial crop in China. The survival rate and growing status of tobacco plants after being transplanted to the field are essential for the field management and yield predictions. However, counting the number of live plants is traditionally conducted by labors, which is time consuming and expensive. Unmanned aerial vehicle is a cost-effective option for monitoring croplands and plantations. However, visual inspection for such images can be a challenging and biased task, specifically for locating and detecting plants. As tobacco plant has a characteristic center-oriented feature, a novel deep-learning algorithm was developed to locate and count tobacco plants via key points detection method, instead of using a common bounding-box object-detection approach. The proposed deep learning algorithm was tested on the cigar plants. In the algorithm, the center of each plant was firstly annotated with a point, and a Gaussian probability density was derived to provide useful information of morphological features. Secondly, different backbones and loss functions in the proposed algorithm were evaluated. Using ResNetlS as a backbone provided the most accurate prediction of the plant number (average precision higher than 99.5%). MobileNetV2 was the most efficient backbone, but the uncertainty of predictions was higher than that of ResNetlS. The combination of Focal Loss function and MSE Loss function (Union loss) reached the highest accuracy (average precision higher than 99. 5% ) while reduced the uncertainty. Finally, the evaluation of different combinations of multispectral bands showed that the combination of red-edge, red, and green bands had a better performance than using red, green, and blue bands in differentiating tobacco plants and weeds, resulting in less uncertainty in the tobacco plant detection. The proposed algorithm can accurately locate and count tobacco plant in the UAV images, providing an effective tool and a valuable data support for planting high-quality tobaccos. © 2023 Chinese Society of Agricultural Machinery. All rights reserved.