Towards Object Descriptions in Natural Language from Qualitative Models

An approach is presented aimed to produce narrative descriptions of objects within digital images which are understandable by human beings. A context-free grammar is defined based on qualitative models for colour and shape description. This approach was tested using images of the MPEG-7 CE Shape-1 library and images of tile geometric pieces. The results show that a general or detailed narrative description can be obtained from different kinds of images containing one or more objects and they also support the further application of our approach to (i) teach children about shapes by giving them instructions on how play the game known as join-the-dots and (ii) help children with visual impairments to carry out visual oddity tasks in psychological tests.