MSNet: Structural Wired Neural Architecture Search for Internet of Things

The prosperity of Internet of Things (IoT) calls for efficient ways of designing extremely compact yet accurate DNN models. Both the cell-based neural architecture search methods and the recently proposed graph based methods fall short in finding high quality IoT models due to the search flexibility, accuracy density, and node dependency limitations. In this paper we propose a new graph-based neural architecture search methodology MSNAS for crafting highly compact yet accurate models for IoT devices. MSNAS supports flexible search space and can accumulate learned knowledge in a meta-graph to increase accuracy density. By adopting structural wiring architecture, MSNAS reduces the dependency between nodes, which allows more compact models without sacrificing accuracy. The preliminary experimental results on IoT applications demonstrate that the MSNet crafted by MSNAS outperforms MobileNetV2 and MnasNet by 3.0% in accuracy, with 20% less peak memory consumption and similar Multi-Adds.