Moving Ship Optimal Association for Maritime Surveillance: Fusing AIS and Sentinel-2 Data

Nowadays, a variety of different sources can be combined together to measure and monitor maritime human activities. Reliable data fusion techniques are essential to associate the targets from different systems for maritime surveillance. In particular, the fusion of data from Sentinel-2 satellites and the automatic identification system (AIS) has attracted wide attention due to their public availability and complementarity. However, most traditional methods for target association are not suitable for this particular case, due to the time lag phenomenon of Sentinel-2 data. In this study, we first construct two new datasets for the detection of moving ships and their wakes based on Sentinel-2 images. Combined with the detection results obtained by the you only look once (YOLOv5) model, the position and course information of the detected ships are first extracted. After carefully analyzing the time lag phenomenon of Sentinel-2 data, we develop a new domain adaptation-based method for target association based on the fusion of Sentinel-2 and AIS data, called moving ship optimal association (MSOA). Different from standard domain adaptation methods only for representation alignment, the proposed MSOA is able to align representation, time, and position simultaneously. A case study is provided in which the newly proposed method is tested over the Port of Long Beach, USA. Experimental results demonstrate that both moving ships and wakes are well detected. Specifically, our newly proposed MSOA exhibits more accurate and robust performance when compared to traditional methods, and the detected ships without corresponding AIS tracks can also be detected by our MSOA. Moreover, the real sensing time and time lag of Sentinel-2 data are deduced with high accuracy. Overall, it can be concluded that our MSOA provides a new perspective for accurate target association based on heterogeneous data fusion.