Image patch-based deep learning approach for crop and weed recognition

Accurate classification of weed species in crop plants plays a crucial role in precision agriculture by enabling targeted treatment. Recent studies show that artificial intelligence deep learning (DL) models achieve promising solutions. However, several challenging issues, such as lack of adequate training data, inter-class similarity between weed species and intra-class dissimilarity between the images of the same weed species at different growth stages or for other reasons (e.g., variations in lighting conditions, image capturing mechanism, agricultural field environments) limit their performance. In this research, we propose an image based weed classification pipeline where a patch of the image is considered at a time to improve the performance. We first enhance the images using generative adversarial networks. The enhanced images are divided into overlapping patches, a subset of which are used for training the DL models. For selecting the most informative patches, we use the variance of Laplacian and the mean frequency of Fast Fourier Transforms. At test time, the model's outputs are fused using a weighted majority voting technique to infer the class label of an image. The proposed pipeline was evaluated using 10 state-of-the-art DL models on four publicly available crop weed datasets: DeepWeeds, Cotton weed, Corn weed, and Cotton Tomato weed. Our pipeline achieved significant performance improvements on all four datasets. DenseNet201 achieved the top performance with F1 scores of 98.49%, 99.83% and 100% on Deepweeds, Corn weed and Cotton Tomato weed datasets, respectively. The highest F1 score on the Cotton weed dataset was 98.96%, obtained by InceptionResNetV2. Moreover, the proposed pipeline addressed the issues of intra-class dissimilarity and inter-class similarity in the DeepWeeds dataset and more accurately classified the minority weed classes in the Cotton weed dataset. This performance indicates that the proposed pipeline can be used in farming applications.