Proxy Data Generation for Fast and Efficient Neural Architecture Search

Neural architecture search (NAS) gains popularity in designing best networks for various tasks. Although it shows promising results on many benchmarks and competitions, NAS still suffers from its demanding computation cost for searching in high dimensional architectural design space, and this problem becomes worse when we want to use large-scale datasets. In this paper, we propose a systematic approach to measuring the importance of each training sample on NAS process and making a reliable proxy data, which is a small subset of the original data and thus reduces the computational cost. The idea behind proxy data comes from our observation that each sample has a different impact on NAS process and most of the examples are redundant when we compare the relative accuracy of possible network configurations. Our experimental results show that we can preserve almost the same accuracy ranking among all possible network configurations with proxy data consisting of 5-10% of the original dataset. To the best of our knowledge, our proposed method is the first attempt to make a reliable proxy data for NAS in a systematic manner.