MAC: a meta-learning approach for feature learning and recombination

Optimization-based meta-learning aims to learn a meta-initialization that can adapt quickly a new unseen task within a few gradient updates. Model Agnostic Meta-Learning (MAML) is a benchmark meta-learning algorithm comprising two optimization loops. The outer loop leads to the meta initialization and the inner loop is dedicated to learning a new task quickly. ANIL (almost no inner loop) algorithm emphasized that adaptation to new tasks reuses the meta-initialization features instead of rapidly learning changes in representations. This obviates the need for rapid learning. In this work, we propose that contrary to ANIL, learning new features may be needed during meta-testing. A new unseen task from a non-similar distribution would necessitate rapid learning in addition to the reuse and recombination of existing features. We invoke the width-depth duality of neural networks, wherein we increase the width of the network by adding additional connection units (ACUs). The ACUs enable the learning of new atomic features in the meta-testing task, and the associated increased width facilitates information propagation in the forward pass. The newly learned features combine with existing features in the last layer for meta-learning. Experimental results confirm our observations. The proposed MAC method outperformed the existing ANIL algorithm for non-similar task distribution by approximate to\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\approx$$\end{document} 12% (5-shot task setting).