Input enhanced asymmetric transformer for image captioning

Image caption is a popular research direction in computer vision. It is a task that enables machines to convey the computer's perception and cognition of vision to the outside world in the form of human language. Currently, the most dominant models are Transformer-based architectures which achieve the cutting-edge performance. Inspired by the distinguished meshed-memory transformer model which uses a mesh-like connectivity at decoding stage. It let us see more possibilities in the Transformer model. With the aim to explore more possible connectivity schemas in Transformer, we propose the input enhanced asymmetric transformer (IEAT) model. It improves the connectivity between encoder layers and optimizes the generation effect of the captions. To better evaluate the final effect of our model, we conducted extensive experiments (offline evaluation, online evaluation and ablation study) on the MS-COCO benchmark and the "Karpathy" test split. And the results show that IEAT outperforms the previously proposed models to generate satisfactory image captions.