Spatial-spectral feature extraction for in-field chlorophyll content estimation using hyperspectral imaging

In-situ leaf chlorophyll content (LCC) estimation based on hyperspectral imaging (HSI) is crucial to track the growth status of crops for field management. However, spatial and spectral features of HSI data, suffering from interference of growth dynamic effect and soil, pose the challenge on accuracy and robustness of LCC estimation in several years and growth stages. Therefore, a joint spectral-spatial feature extraction method was proposed by cascade of three-dimensional convolutional neural network (3DCNN) and long short-term memory (LSTM) to reduce the interference for optimising the LCC estimation. Firstly, crop pixels were separated from soil with vegetation index segmentation method. Secondly, when raw images and segmented pixels were input, sensitive bands were selected by random frog (RF bands), and 3DCNN-LSTM was used to extract the joint spectral-spatial features. Finally, models established by RF bands, 3DCNN and 3DCNN-LSTM were compared, and robustness in individual years and stages was validated. Results showed that RF bands and 3DCNN obtained R P 2 of 0.76 and 0.84 when not segmented. After segmentation, performance of 3DCNN improved (RP2 P 2 = 0.85) compared to RF bands (RP2 P 2 = 0.80). Spectral-spatial features by 3DCNN reduced the interference of soil. 3DCNN-LSTM without and with segmentation obtained good performance with R P 2 of 0.95 and 0.96, and the proposed method could reduce the image segmentation process. The optimal model achieved R P 2 above 0.93 in individual years (RP2 P 2 = 0.96 in 2021, R P 2 = 0.94 in 2021) and R P 2 in the range of 0.87-0.97 at individual stages. This paper provides a method to track growth variability between soil and crop for the LCC estimation optimisation.