ESNet: An Efficient Framework for Superpixel Segmentation

Superpixel segmentation divides an original image into mid-level regions to reduce the number of computational primitives for subsequent tasks. The two-stage approaches work better but have high computational complexity among the existing deep superpixel algorithms. In contrast, the FCN style approaches cannot extract specific image features for the superpixel task. To combine the advantages of both types of methods, we propose a carefully designed framework termed Efficient Superpixel Network (ESNet) to explicitly enhance the capability of the network to describe clustering-friendly features and simultaneously preserve the simple network structure. Concretely, two points are concerned with ESNet. First, meaningful features need to be constructed for effective superpixel clustering; hence we propose the Pyramid-gradient Superpixel Generator(PSG) to decouple the ESNet into two joint parts, i.e., the feature extractor and the superpixel generator. Second, the superpixel generator is designed in an efficient manner, which performs multi-scale sampling of input images, and can work independently by replacing the introduced feature extractor with two initial convolutional layers. Extensive experiments show that our framework achieves state-of-the-art performances on multi-datasets and is 5.3x smaller on inference than the best existing one-stage FCN-based methods.