Architecture evolution of convolutional neural network using monarch butterfly optimization

Designing suitable convolutional neural networks (CNNs) for different image data requires much human effort and expertise, in recent years, this process has been greatly accelerated by automatic architecture design methods. However, existing work rarely integrates macro-architecture space with depth search space, which usually leads to suboptimal architecture design results. Also, the adopted search strategy often needs to be specially customized for compatibility with architecture encoding. This paper thus proposes an automatic architecture design method based on monarch butterfly optimization (MBO). Specifically, an expressive Neural Function Unit (NFU) based architecture representation is designed, which integrates promising architectures in GoogLeNet, ResNet and DenseNet to facilitate the joint search of macro-architecture and depth of CNNs. Furthermore, a direct architecture encoding is designed to take advantage of the fast convergent MBO, which exploits evolutionary operators that have no complex computations to continuously improve the architecture population via encoding optimization. Extensive experiments conducted on eight benchmark image datasets demonstrate that our method can achieve continuously competitive performance with much less time and computational overhead.