A Task-Aware Attention-Based Method for Improved Meta-Learning

Based on massive data, deep neural networks have been proven to have a powerful learning capability of non-linear relationships. However, training deep neural networks on limited samples is still challenging, which may lead to the over-fitting problem. To alleviate this problem, meta-learning was proposed to train a model that can rapidly adapt to a new task with only a few related examples. However, existing meta-learning approaches tend to ignore the domain gap between different tasks. For a specific task, some of the features are unrelated or even disruptive, which may cause damage to the effectiveness of meta-learning. To address this issue, in this paper, we propose a novel attention-based method that can skip the useless features and highlight the task-specific information. We design two simple but effective attention modules, which take task representation as input and produce attention weights for features from two different perspectives. Experiments conducted on four benchmarks validate that our method outperforms state-of-the-art methods, and the main idea can be applied to various existing meta-learning models.