TGGS network: A multi-task learning network for gradient-guided knowledge sharing

Multi-task learning (MTL) has been widely used in various fields, such as time series data prediction and image classification. Most existing deep MTL methods achieve joint learning by sharing common knowledge with task relevance. However, they rely solely on self-exploration to mine common knowledge across the tasks, which is unguided and purposeless. Unlike these methods, in this paper, we propose a novel Task Gradient Guided Sharing (TGGS) MTL network for purposefully guiding the model to share knowledge. Specifically, we define a confidence matrix for each task for selecting effective knowledge from the common knowledge to share. To realize an effective guidance, we align the confidence matrix with a class gradient mapping (CGM) computed based on the task-specific gradients. Consequently, our method effectively employs task-specific gradients to guide the model in sharing useful knowledge. More importantly, the confidence matrix explains the complex correlations among tasks in MTL. As our method exclusively focuses on guided knowledge sharing, the TGGS network can seamlessly integrate with diverse MTL sharing strategies. We conduct extensive experiments on time series datasets and image datasets. The experimental results demonstrate our method significantly improves the performance of these MTL strategies.