Fine-grained Image Caption based on Multi-level Attention

With the continuous development of image caption, the effects of it are getting better. However, there are still problems with the image caption. The complete description should contain different forms of information which include fine-grained information and labels. In order to cope with the above challenges, we propose a fine-grained image caption based on multi-level attention method which can complete label prediction and description of fine-grained images. (1)We firstly use the visual attention mechanism to fuse features of the global and local fine-grained features. (2)The joint attention mechanism is then used to fuse the visual feature and the label features of the image to generate a text description for a specific region of the image. (3)Finally, we use the attention-based Long Short Term Memory (LSTM), a language generation model, to generate fine-grained image caption.