TRANSFER REINFORCEMENT LEARNING: FEATURE TRANSFERABILITY IN SHIP COLLISION AVOIDANCE

The integration of artificial intelligence into engineering work has become increasingly prevalent. Engineering work processes can be highly complex, and learning from scratch requires large computation resources. Transfer learning has emerged as a promising technique for improving learning efficiency by leveraging knowledge gained from related tasks to the target task. To achieve optimal performance, one of the key challenges is to figure out how transferrable the features are among different work processes and within training networks. Simulation-based ship collision avoidance is used for case studies due to its inherent complexity and diversity. Two transfer reinforcement learning methods, feature extraction, and finetuning, are implemented and evaluated against the baseline. Instead of introducing large-scaled pre-trained models as the backbone, a light CNN model pre-trained in a related base case has been proven to transfer essential features to target cases. Simplified ship dynamics is introduced into the training process to make it more realistic and applicable, and the delay caused by the large moment of inertia is addressed by modifying the model-environment interaction mechanism. Work process features for the ship collision avoidance process are concluded from crucial aspects. The effects on transferability are displayed by experimental results discussed from the feature category and similarity perspective.