MLSNET: RESOURCE-EFFICIENT ADAPTIVE INFERENCE WITH MULTI-LEVEL SEGMENTATION NETWORKS

In this paper, we propose a multi-level convolutional network for semantic segmentation. The advantage of our network is that it allows us to adaptively control the balance between the classification accuracy and the inference speed, depending on the limited computational resource and the complexity of a given task. Our network realizes such adaptive mechanism by introducing a hierarchically-connected decoder with multilevel classifiers. The low-level classifier is used for inference when the application prioritizes low computational cost over accuracy, and the high-level classifier is utilized when more accurate prediction is required. This switching can be automatically performed by specifying only a threshold for classification performance. Besides, to boost the lower-level classifiers, we incorporate a knowledge distillation mechanism into our network. The entire network is trained in an end-to-end fashion. Experiments on semantic segmentation datasets demonstrate that our model overperforms the conventional approaches.