Network Adjustment: Channel and Block Search Guided by Resource Utilization Ratio

It is an important problem to design resource-efficient neural architectures. One solution is adjusting the number of channels in each layer and the number of blocks in each network stage. This paper presents a novel framework named network adjustment which considers accuracy as a function of the computational resource (e.g., FLOPs or parameters), so that architecture design becomes an optimization problem and can be solved with the gradient-based optimization method. The gradient is defined as the resource utilization ratio (RUR) of each changeable module (layer or block) in a network and is accurate only in a small neighborhood of the current status. Therefore, we estimate it using Dropout, a probabilistic operation, and optimize the network architecture iteratively. The computational overhead of the entire process is comparable to that of re-training the final model from scratch. We investigate two versions of RUR where the resource usage is measured by FLOPs and latency. Experiments on standard image classification datasets and a few base networks including ResNet and EfficientNet demonstrate the effectiveness of our approach, which consistently outperforms the pruning-based counterparts.