Faba bean and pea harvest index estimations using aerial-based multimodal data and machine learning algorithms

Early and high-throughput estimations of the crop harvest index (HI) are essential for crop breeding and field management in precision agriculture; however, traditional methods for measuring HI are time-consuming and labor-intensive. The development of unmanned aerial vehicles (UAVs) with onboard sensors offers an alternative strategy for crop HI research. In this study, we explored the potential of using low-cost, UAV-based multimodal data for HI estimation using red-green-blue (RGB), multispectral (MS), and thermal infrared (TIR) sensors at 4 growth stages to estimate faba bean (Vicia faba L.) and pea (Pisum sativum L.) HI values within the framework of ensemble learning. The average estimates of RGB (faba bean: coefficient of determination [R2] = 0.49, normalized root-mean-square error [NRMSE] = 15.78%; pea: R2 = 0.46, NRMSE = 20.08%) and MS (faba bean: R2 = 0.50, NRMSE = 15.16%; pea: R2 = 0.46, NRMSE = 19.43%) were superior to those of TIR (faba bean: R2 = 0.37, NRMSE = 16.47%; pea: R2 = 0.38, NRMSE = 19.71%), and the fusion of multisensor data exhibited a higher estimation accuracy than those obtained using each sensor individually. Ensemble Bayesian model averaging provided the most accurate estimations (faba bean: R2 = 0.64, NRMSE = 13.76%; pea: R2 = 0.74, NRMSE = 15.20%) for whole growth stage, and the estimation accuracy improved with advancing growth stage. These results indicate that the combination of low-cost, UAV-based multimodal data and machine learning algorithms can be used to estimate crop HI reliably, therefore highlighting a promising strategy and providing valuable insights for high spatial precision in agriculture, which can help breeders make early and efficient decisions. Multiple affordable aerial-based sensors can be used to estimate the harvest index of faba bean and pea with an ensemble Bayesian model averaging algorithm.