An Effective One-Shot Neural Architecture Search Method with Supernet Fine-Tuning for Image Classification

Neural architecture search (NAS) is becoming increasingly popular for its ability to automatically search for an appropriate network architecture, avoiding laborious manual designing processes, and potentially introducing novel structures. However, many NAS methods suffer from heavy computational consumption. One-shot NAS alleviates this issue by training a big supernet and allowing all the candidates to inherit weights from the supernet, avoiding training from scratch. However, the performance evaluations in one-shot methods might not always be reliable due to the weight co-adaption issue inside the supernet. This paper proposes a supernet fine-tuning strategy to allow the supernet's weights to adapt to the new focused search region along with the evolutionary process. Furthermore, a new genetic algorithm-based search method is designed to offer an effective path-sampling strategy in the search region and provide a new population generation method to preclude unfair fitness comparisons between different populations. The experimental results demonstrate the proposed method achieves promising results compared with 32 peer competitors in terms of the algorithm's computational cost and the searched architecture's performance. Specifically, the proposed method achieves classification error rates of 2.50% and 17.07% within only 0.50 and 0.92 GPU-days on CIFAR10 and CIFAR100, respectively.