Time Series Remote Sensing Image Classification Using Feature Relationship Learning

Recently, time series remote sensing image (TSRSI) has been reported to be an effective resource for mapping fine land use/land cover (LULC), and deep learning, in particular, has been gaining growing attention in this field. However, existing deep learning methods often only learn features from either the temporal or spatial domain, neglecting the intercorrelation between temporal features, which may provide more information for classification, are not fully considered. To make full use of the relations between temporal features and to explore more objective features for improving classification accuracy, we proposed a feature relationship-based classification method. The method leverages the angles between features on the temporal curve to establish relationships between every pair and triplet of features, resulting in the creation of feature relationship matrices (FRMs) and feature relationship tensors (FRTs). Afterward, a 2-D-3-D multiscale convolutional neural network (2-D-3-D MSCNN) was designed to learn deep features from FRM and FRT, achieving the classification improvement of TSRSI. Our experiment was conducted on TSRSIs located in two counties, Sutter and Kings in California, USA. The experimental results indicate that compared to both deep learning and nondeep learning methods, the proposed approach achieves significant improvements in accuracy and LULC mapping, validating the effectiveness and feasibility of enhancing TSRSI classification accuracy through feature relationship learning.