Privacy-Preserving Neural Architecture Search Across Federated IoT Devices

While deploying on edge devices, deep learning models often encounter various strict resource constraints. Automated machine learning becomes popular in finding various neural architectures that fit diverse Internet of Things (IoT) scenarios to handle these problems with less human efforts. Recently, there is an emerging trend to integrate federated learning and Neural Architecture Search (NAS) to prevent private data leakage while enabling automated machine learning. The algorithm development is quite challenging because of the coupling of difficulties from both tenets, although promising as it may seem. Especially, it is a hard nut to efficiently search the optimal neural architecture directly from massive non-Independent and Identically Distributed (non-IID) data among IoT devices in a federated manner. In this paper, by leveraging the advances in ProxylessNAS, we propose a Federated Direct Neural Architecture Search (FDNAS) framework that allows hardware-friendly NAS from non-IID data across devices to tackle the challenge. Extensive experiments on non-IID datasets demonstrate the state-of-the-art accuracy-efficiency trade-offs achieved by proposed methods.