Multilingual Image Corpus - Towards a Multimodal and Multilingual Dataset

One of the processing tasks for large multimodal data streams is the automatic image description (image classification, object segmentation and classification). Although the number and the diversity of image datasets is constantly expanding, there is still a huge demand for more datasets in terms of variety of domains and object classes covered. The goal of the project Multilingual Image Corpus (MIC21) is to provide a large image dataset with annotated objects and object descriptions in 25 languages. The Multilingual Image Corpus relies on an Ontology of Visual Objects (based on WordNet) and comprises a collection of thematically related images whose objects are annotated with segmentation masks and labels linked to the ontology classes. The dataset is designed both for image classification and object detection and for semantic segmentation. The main contributions of our work are: a) the provision of a large collection of high-quality images licensed for commercial and non-commercial use; b) the compilation of the Ontology of Visual Objects based on WordNet noun hierarchies; c) the automatic object segmentation within the images followed by precise manual editing and the annotation of object classes; and d) the mapping of objects and images to extended multilingual descriptions based onWordNet inner- and interlingual relations. The dataset can be used also for multilingual image caption generation, image-to-text alignment and automatic question answering for multimedia content.