Estimation and counting of wheat ears density in field based on deep convolutional neural network

Wheat is one of the most important grain crops in the world. The stability of wheat yield is crucial to global food security. The final yield of wheat can be calculated by the number of ears per unit area. Using deep learning technology to accurately and automatically count the number of wheat ears can save a lot of manpower and material resources. The images of wheat ears from four different wheat varieties during grain filling period were collected, and 296 images with a total of 14 964 ears of wheat were selected to construct the WEE data set. The method of point labeling was used to label the wheat ears in the images. In order to quickly and accurately count the number of wheat ears in complex crowded scenes, a method of estimating the density map and counting the number of wheat ears in field wheat images was proposed. Firstly, histogram equalization and threshold segmentation were used to preprocess the collected field wheat images to reduce the influence of light and some complex backgrounds on counting. Then, according to the characteristics of dense wheat growth at grain filling period, the Congested Scene Recognition Network (CSRNet) was introduced to construct the wheat ear density map estimation model. The CSRNet network was composed of front-end and a back-end networks. The front-end network uses the pre-training model VGG16 to extract the features of the wheat images, and the back-end network uses the dilation convolution to generate the distribution density map while expanding the receptive field. In order to improve the accuracy of the model in the training process, the transfer learning method was used to pre-train the model by using the public wheat image data set, and the collected wheat image data set was used to adjust and optimize the model parameters. The trained model was used to generate the wheat ear density map of a single wheat image, and the wheat ears were counted according to the sum of all density values in the density map. Finally, according to the test data of a single wheat ear image, a wheat ear counting function was constructed to estimate the number of wheat ears in the field. Experiments were conducted on a total of 296 wheat images collected from four varieties (Annong170, Sumai 188, Lemai 608 and Ningmai 24). The Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) were 16.44 and 17.89, respectively. The correlation coefficient R2 between the ear count values and the true values of the four varieties of wheat was about 0.9, indicating that the method in this paper has a higher accuracy for counting wheat ears in a single image. In addition, the experiment of estimating the number of wheat ears in the field showed that the error of estimation of wheat ears was smaller with the increase of area. The results of this study can provide the possibility of automatic estimation of the number of wheat ears in practical application process, and also provide a reference for wheat yield estimation. © 2020, Editorial Department of the Transactions of the Chinese Society of Agricultural Engineering. All right reserved.