Asymmetric decoder design for efficient convolutional encoder-decoder architectures in medical image reconstruction

In recent years, significant research has been performed on developing powerful and efficient Convolutional Neural Network (CNN) architectures. To utilize these architectures in pixel-level regression tasks such as tomographic image reconstruction, a feature extraction encoder is often combined with a symmetrical decoder to generate an encoder-decoder structure, such as a U-Net. However, a more powerful decoder focusing on high-frequency features can provide higher pixel-level accuracy. In this work, we investigate the use of asymmetrical encoder-decoder architectures in medical image reconstruction tasks. The state-of-the-art EfficientNet architecture utilizes depthwise convolutions and channel attention within inverted residual bottleneck blocks to generate highly compressed features while maintaining a significant FLOPS efficiency advantage compared to regular convolutional encoders. We develop an asymmetric encoder-decoder architecture, which uses the EfficientNet as an encoder. The proposed decoder architecture combines the multi-resolution features generated by the EfficientNet encoder using an incremental feature expansion strategy, which leads to better preservation of the structural details in reconstructed images. We have tested our asymmetrical encoder-decoder approach on undersampled MRI reconstruction tasks using the Calgary Campinas multi-channel brain MR dataset. Results demonstrate that the proposed asymmetric approach vastly outperforms a symmetric Efficient U-Net, achieving a 3dB improvement in PSNR. SSIM was also improved, and the asymmetric network was found to recover small structural details more effectively. Furthermore, the proposed asymmetric Efficient U-Net provides a four-fold reduction in inference time when compared to the conventional U-Net architecture.